We're all middle management now

I want to talk about agentic development — how it works inside Power BI, and what it costs. But first, a story about a client. Call him Daniel.

Daniel is head of BI for a mid-size services company. His team of six includes some old hands and some very old hands — people with twenty-plus years in the organisation who know the data estate backwards, know where the bodies are buried, know why the financial director likes his reports a certain way. The reporting estate is almost as old as they are: SSRS and SSAS, with one Power BI semantic model to show for their modernisation efforts. They’re not not trying; they’re just permanently at capacity with business as usual.

When the financial director asks for an Excel reporting pack to be automated into Power BI, the team can’t take it — it’s year end. So Daniel hands it to a contracting firm who know the financial system well. They deliver a lovely report pack; the financial director is delighted. A fortnight later he wants one metric changed — annualised rather than monthly. A small change, except Daniel’s team have no idea how the report was built, and learning it would take a week or more alongside their day jobs. The director wants it this month. Back to the contractors: half a day, done.

Daniel starts to like this. Why not have the contractors do the modernisation too? Take one of the big SSRS beasts and rebuild it in Fabric while his team keeps the lights on. The contractors deliver a couple of months later — and it never reaches user acceptance testing. The team take one look: a whole set of the numbers is wrong. For years, part of what the old reports showed had come from figures the team looked after themselves, outside the source system, because the source system had never handled that part properly. Those numbers existed nowhere the contractors had been pointed at, and the contractors — briefed to build from the source system — never saw them. “Why didn’t you tell me?” asks Daniel. “You didn’t ask,” says the team. “You said: here’s the model, migrate it.”

The contractors fix it (their original developer has moved on; someone new gets up to speed) and redeliver. The numbers are still wrong. Why? The team can’t say — they don’t understand the new system well enough. Daniel is now in a spiral. His team knows exactly what the numbers should be; the contractors know exactly how the new system works; there is no bridge between them. The how has become separated from the why.

Now replace “contractors” with “AI”, and Daniel’s whole operation with yourself. That’s where we are with agents and Power BI. Daniel’s contractors had lead times, invoices, and a finite headcount; with an agent, delegating another job can be as easy as typing another prompt. Agents rapidly increase our capacity to implement the hows. Handled carelessly, they erode our ability to manage the whys.

A chatbot generates each response from its context: everything said so far, plus a hidden system prompt telling it where it is and how to behave. An agent is that, plus a harness that gives the model tools. When the model emits a tool invocation — “read this file” — the harness runs it and pushes the result back into the context. That’s the whole trick. Tools can act on the outside world, but the model only ever knows what the tool passes back. Everything in agentic development — CLAUDE.md files, MCP servers, the lot — is about getting the right material into context, or giving the agent tools to fetch more of it.

Figure 1The agentic system

This matters for Power BI because of the .pbip format, which unwraps the old .pbix binary into plain text files describing the report and the semantic model. Text files are something an agent can read and write — which means an agent can now do Power BI development.

I saved a blank report as .pbip and opened Claude Code in the folder — out of the box, no CLAUDE.md, no extra tools — with a few paragraphs asking it to invent a model, generate data, and add some charts. It wrote the files. Nobody told it what a .pbip is or how TMDL is structured; that knowledge came straight from training. When I opened Power BI it failed — but pasting each error back produced a fix, and after two rounds I had a working report built from nothing but prose. Eye-opening, if not useful: with no real scope, it invented everything.

Then, in VS Code, I asked it to “add a measure for adjusted net position that accounts for all the data in the model”. I had no idea what that meant. Neither did the agent, apparently, but it went ahead anyway. It wrote a measure that did… some stuff. Think of an agent as a very eager junior who believes you are a monster who reacts badly to clarifying questions. The fault was mine: an adjusted net position needs adjustments, and none existed in the model. Once I loaded an adjustments table and asked again, it wrote a measure that worked — and, unprompted, deleted its earlier nonsense.

In all of this, the agent had no way to know whether its output was correct, or even functional. I had to open Power BI and check, every time. What we want is an agent that validates its own output and iterates. For Power BI there are two validation questions: are the numbers right, and does it look right.

The numbers loop is the smaller one. Give the agent a tool to query the data model — Microsoft’s modelling MCP server does exactly this — and it can change the model, run a DAX query, and judge the result. But note what changes: until now the agent saw only metadata — table names, relationships, code. A query tool means the data can leave too, potentially including personal information bound for a third party you never agreed should see it — a GDPR problem. It’s manageable with effort: pin the model to infrastructure in a region you choose, and take the arrangement to your data protection officer.

The looks loop is harder — the agent has to see the page. My version: push the report to Fabric, render it via the API, screenshot it with Playwright, and hand the image back to the agent to judge. It worked; it was also flaky, slow, and produced ugly reports.

Figure 2The visual validation loop

The point is that both loops can be closed. But honestly: today it takes real effort or real risk, and validation still sits with the developer. Not impossible; one day, probably easy; right now, hard.

Even a closed loop only answers part of the question. If my car makes a funny noise at 32 miles an hour, the mechanic fixes it, I drive, no noise — done. If the repair fails, there’s a mechanic I can go back to and hold responsible for the work.

With an agent, only the implementation transfers. If it gets something horribly wrong — but plausible-looking — and my CEO loses £50 million on the back of it, nobody is sending Anthropic the bill. The buck stops with whoever managed the agent.

And what we’re delegating is not just labour but judgement. Every solution is a stack of decisions; every report is one way when it could have been another, and many of those choices are business decisions hiding as technical ones. Yes, all software delegates decisions — every abstraction layer is a choice someone made for me. But conventional software is opinionated: the DAX engine gives the same answer to the same question every time. An agent doesn’t. It decides, and next time it may decide differently. If I’m responsible for the solution, every one of its decisions is a decision I’m responsible for.

So how much should you delegate? Consider two prompts: “build me a report”, and “in this table’s M code, apply a filter after line four using these functions to remove data before this date”. They delegate wildly different amounts of decision-making. Which to use depends on the worst case if every decision goes wrong. A throwaway weekend report? Vibe the whole thing. Something the board will act on? Make the decisions yourself, up front — pages, sources, cleaning patterns, security — because every decision stated in the prompt is one the agent doesn’t make, and the result can be checked against a structure you defined rather than one it invented.

Figure 3The vibe zone

There’s a third axis: domain experience. Ask an agent to work in a technology you’ve never touched and the risk slope steepens sharply, because your ability to judge its choices collapses just when it matters.

Figure 4Domain experience

Even when delegation makes sense for the job in front of us, there’s a longer-term problem. The more time we spend letting agents make great stacks of decisions, the less capable we become of judging them. A spell checker erodes our spelling; this may erode not just our ability to do the work, but our ability to know whether it was done right. I don’t have a solution to that yet — I don’t think anyone has done enough agentic Power BI work to have truly felt the erosion. But the shape of it is visible.

Daniel wanted implementation done fast by people who knew the new technology, with his team still responsible for the result. The decisions became separated from their consequences, and the spiral followed. The only way out — and it is slow work — is for his team to take the new solution apart piece by piece until they understand every why. Ownership first; then someone, or something, else can do the implementation.

The moment we delegate, we become managers — that part isn’t optional. We’re all middle management now. The only choice is what kind: the kind who understands the work well enough to own it, or the slopey-shouldered kind who waves everything through because Claude can do it. Claude can do it. But a manager who doesn’t understand the risks, the consequences, or the decisions inside the solutions being produced for them isn’t a developer any more — just a relay.

And the agent doesn’t hold the responsibility. You do.