Programming

Since there's zero information about what kind of company you're working at, the following is extremely generalized.

• integrating with monitoring systems, analytics DBs, ticket systems, whatever is used by management, allowing them to ask questions in natural language
• process automation using agentic workflows, e.g. pre-analysis of incoming email queue summarizing / sentiment analysis before the customer support sees it
• provide access to models and model APIs for development workflows and integration into git / ci, allowing to use llm in local development and e.g. setting up something like automated code reviews (not a replacement for human review, only as an addition)
• set up coaching, responsible use, hallucinations, etc.

Whatever you do, take security and data security especially into consideration first, not after:

• consider whether your used provider reuses your data for learning
• consider whether it's relevant where it's located (GDPR customers?)
• always set spending limits
• consider your local and your customers data protection laws and regulations that apply to your company (especially in health and financing)

If the person who put you in charge of it knew of your indifference, that's a smart manager, and you should just trust your gut on this.

If you just vibe AI, it will lull you into thinking you're doing great.

1. Define what successful AI usage looks like and include metrics and measurements — include AI usage in your story/ticketing system.
2. Define parameters for usage. Will it write documentation? Write tests? Write functions? Classes? Whole features? Will it review code? How are you making sure everyone understands the new code (i.e. you don't want devs committing code they don't understand or reviewers passing it)
3. How will you reign in costs? I had 3 devs spend over $1500 (each) in thirty days and am in the process of a explaining this to my COO. Purchasing plans instead of using API keys gives you some natural boundaries for reigning in costs.
4. Create standard processes. Don't vibe. Have a standard for how your AI-facing documentation is structured. Have standards/templates for how prompts are structured. Have standard prompts with limited scope for specific tasks. For example have a prompt just for review that specifies to review from the standpoint of security and best practices.

First, my condolences.

Second, I think something you can do as "head of AI" is push back on the benchmarks your execs are expecting you to measure. They got them straight from whoever your AI vendor is, and if your executive team is even halfway competent they should understand "these metrics are designed by the vendor to make us spend more", but a lot of exec teams won't listen. Still, you should institute your own benchmarks around code quality and delivery speed and talk about them with the exec team even if you have to shoehorn them into the discussion.

The next and probably more important thing that comes to mind is managing how your devs use their new tools. They'll be able to churn out more lines of code than ever before and "complete" some features much more quickly. Your metrics should not incentivize this if your goal is code quality and stability. I don't really have much in the way of solutions (other than "manage expectations" and "set the narrative"), because that's about where in my "head of AI" career I got laid off. Once the numbers came in that we only needed 30% of our current staff they crunched the numbers and I was in the 70%.

Good luck. You're starting from a deficit because your management team probably already has entertained the thought of trimming some expensive devs from the payroll, and that's a tough thing to argue against.

We don't even know what your company does? Does it do development? Data analysis?

Without knowing that, as head of AI, I'd first take training to understand what AI is and get advice from an external company as to what it could be good for. "AI" is being used as an umbrella term nowadays for neural networks to stochastic machines and they all have different uses. Understanding valid and invalid usecases is important before deciding when and where to use it in the company.

It depends on your company.

If you are running public-facing code, security is going to become a huge deal in coming months. So use AI to find all the security holes in your stuff before the hackers do.

If you have lots of databases and APIs for internal use, create a MCP server so agents can do anything with your data and APIs.

If your data is even a little sensitive and you don't want to send it all to the USA (read up on the CLOUD act), look into running models locally and the hardware investments you'll need to make.

Figure out what the executive's day-to-day workflow is like, then make agents to do their job. Either they'll get an understanding of how inadequate it is, or they'll be able to just sit around doing nothing useful (assuming that's not what they do already).

Damn, I'm in the same boat. I didn't attend one meeting because ut was too late in My timezone and the others chose this moment to vote me to become the lead of the AI acceleration initiative where we have half a year to come up with and implement things on each step from requirements, through architecture and development to testing on all levels.

And yeah we better come up with something useful because I already know they have to reduce the work force because there is no money and the people who will be left can't do all of it alone without help.

Nah, show them that it's an expensive waste of time by continuing to let them run wild with useless but impressive tasks

You can have it integrated with grafana, and it can quickly search logs based on errors or alerts and link them to specific commits quite quickly. My previous company did that a year ago.

Pay me 25% and I’ll advise you.

I would think in terms of security and guardrails if I were you.

I’m in like 3 AI design forums for some reason but whenever I want to start measuring productivity gains I’m ignored so try that for the least amount of effort required.

Assuming a dev company, first thing I would do is let the manegement know, in writing, that building a dependance on AI into your processes will make you depend on a tool that is currently heavily under-priced, with operating costs for all AI firms resulting in hundreds of millions of losses, that they will want to recuperate eventually.

Link articles about firms blowing the yearly budget on per-token price in a single month. Companies spending 500mil per month on tokens woth uncapped usage.

Also explain that allowing your devs to use it will lead to erpsion of their skill, building a dependency, that will then tie their productivity going forward to the whims of AI companies, that are currently operating at heavy losses.

Link any of the numerous papers how AI erodes skill of devs.

Ask them if they really want to build a dependency into this ecosystem, and if they are willing to pay the heavily increased price, once it inevitably gets expensive. That you can start replacing devs with AI, but even now, at the heavily undercut prices, it's already costing more than a developer would, and it will only get worse. There are articles for that too. Make sure to highlight that you will be giving control of your systems and development to something that will be expensive, and it will be expensive to walk back on those decisions, once it's integrated deep enough.

Get a paper trail. Make them sign off on the risks. Don't do anything unless they sign off on the risks, and make it super clear how stupid idea it is. Bring as many recipes, papers, and blogs you can find.

Make a citation for how much it would cost to run a competitive local models. How much you would have to spend on GPUs to have a local solution for all of your planned AI need, so you can continue working once third party AI is not affordable.

Also take a look at all currently unresolved GDPR and general laws around copyright, unsolved cases. Send them blog posts that imply that there might be a copyright problem with AI generated outputs down the line, once all lawsuits resolve.

Mention the google lawsuit in germany, where Google might be legally responsible for the outputs it's AI gives. Ask if that is a risk they accept, assuming it passes.

It's an extremely stupid idea to currently even consider being dependant on AI. Give the recipes and arguments, and have them directly sign off on every risk you can think off. Basically - whatever you do, do a proper long term risk analysis, and make sure the management signs off on the risks.

You will need it down the line. There is a lot of risks.

May want to consider setting up a private, on-prem system. That way, you can reliably enforce privacy/GDPR rules. You can also tweak the system to support local training, RAG, MCPs, etc.

This way, the costs can also be controlled. It's some capital investment in local hardware, plus reasonably fixed power/cooling/maintenaance ongoing expenses.

Another way is to use the major AI services for planning/brainstorming specific features, tell it not to implement or touch anything, but to generate a detailed plan for an implementer LLM. Review that plan manually, and when ready, feed it to your local system for implementation and debugging.

This doesn't work if the goal is one-shot vibe-coding. But it works really well for focused feature enhancements, test coverage, and bugfixing.

Not sure on the size of your team, but you could perhaps form a small subset of AI "champions" who can try to help drive new initiatives. Just some ideas:

1. Start out with foundational usage of AI agents, ensure people have the right mindset and ownership of using the ai agents. Ensure there is always a "human in the loop."

2. Arrange for some training for people (not just Devs) on the right way of using AI, such as not throwing sensitive information into it. Educate people that it's far from perfect

3. Get standard boundaries in place for your current and future projects, get context files (agent.md) for your projects so those who do use the agents will be doing so from the best position.

4. See if you can arrange with the higher ups to arrange some company time do a Hackathon with the use of AI to help people see what they can do with it and the challenges they'll face. You can also take some learnings from it.

The way I see it, you've been given the position of head of AI because you're indifferent. It's your job now to drive engagement (because that's what your bosses want) but in a controlled manner. If they put someone that was overly enthusiastic about AI they would end up with a huge bill and lots of AI mess to clean up.

Could you explain what feelings you have about AI, and how you see these feelings as opposing. If you are just uncertain about AI (holding no opposing views), I think you would want to research more. Maybe if we knew more about your teams at your work, and what is being developed. You could honestly just spend an hour a week working on standardizing coding agent availability and licensing/subscriptions, and leave it at that. Either they weren't looking for someone who was a machine learning engineer, or whoever promoted you is clueless to what AI actually means.

You could just quit since you clearly have no desire to succeed. Or just wait until they find out and fire you.

As a bonus you can blame "AI" either way.