Every software vendor says they use AI. Most of them are lying. Not intentionally, but by omission.
What they usually mean to say is that their developers have access to AI tools. What they don’t mean is that they changed anything fundamental about how they actually deliver.
There’s a heavily marketed version of “we use AI” that means a Copilot license and a slightly faster pull request. What this article will help you recognise is that one version means the entire delivery model was rebuilt around the assumption that agents handle implementation while humans direct and review. That is a fundamentally different thing, and treating them as interchangeable is an expensive mistake.
The difference rarely shows up in a pitch deck. It surfaces when you walk through how a team actually works, phase by phase. Let’s break it down step by step.
When AI adoption doesn’t move delivery, there are almost always two root causes:
- The pieces got faster, but the flow didn’t. Individual tasks accelerated while the handoffs between phases stayed exactly where they were.
- The context AI needs to work effectively is scattered. Requirements and decisions live across Jira, Confluence, Slack and slide decks, so every new teammate and every agent starts from zero.
Every tell a delivery partner isn’t really “AI-native” traces back to one of these.
If their estimates still come from gut feel, nothing changed
Discovery and estimation is the first place to look, and the one most vendors hope you skip past.
In an AI-assisted setup, developers might use ChatGPT to do some research. But estimates are still based on experience and intuition, and the output of discovery, the decks, the Confluence docs, sits there useful to humans and completely invisible to any agent that needs to build from it.
AI-native teams treat this differently. Prototypes are cheap now, so you build one before you estimate. Agents read the prototype, verify assumptions against working code, and corrections are data-backed rather than instinct-based. The estimation method itself changes because agents work at epic level, not individual tasks. And critically, discovery output is structured so agents can consume it directly. No translation layer. No human handoff between the thing that defines the work and the thing that builds it.
If their scoping deliverable can’t be read by the tool doing the implementation, that’s a handoff that didn’t need to exist, and it’s a tell.
What to ask:
“How does AI participate in your estimation process? Can you show me an example where it changed an estimate, not just a formatted one?” And: “Can your development agents use your discovery output directly, or does someone have to translate it first?”
Every vendor’s favorite number, “50–80% faster”, is mostly irrelevant
Implementation is where every vendor leads with their headline number, and where the truth is the hardest to extract. Yes, AI speeds up coding 50-80% on well-defined tasks.
That’s technically true. It’s also nearly meaningless on its own.
AI-assisted teams write code faster. Everything around that: reviews, coordination, client feedback loops, runs at exactly the same speed it always did. The bottlenecks just got reshuffled. The org can’t absorb what agents produce, so the gain evaporates downstream.
Here’s a useful calibration: teams that measure the whole lifecycle honestly land somewhere around 25–35% — and they can tell you exactly where AI didn’t help. Anyone quoting multiples across the board is quoting their best phase and hoping you don’t ask about the rest.
In an AI-native setup, the developer role itself changed. Engineers direct agents and own quality across a broader scope with fewer people
But that only works if the inputs change too. Agents need structured specs, architecture documented in the repo, and clear acceptance criteria. Not a conversation summary and a Jira title. Teams that skip this end up with developers constantly picking up tasks with incomplete context, assuming AI will fill the gaps. It won’t. And blocked time actually increases.
What to ask: “What does a developer on your team actually do day-to-day, and how is that different from two years ago?” Then: “What does a ticket look like, how do you define work so an agent can pick it up?” And the one most teams dodge: “Where did productivity not improve, and what’s the bottleneck now?”
The question nobody thinks to ask: who can actually onboard here?
Knowledge management doesn’t sound very exciting. It’s also one of the most revealing when you’re evaluating someone’s true AI capabilities.
Every project generates knowledge: decisions made, architecture chosen, context built up over months. The question is whether that knowledge lives somewhere agents can reach, or whether it’s trapped in people’s heads and a Confluence instance nobody fully trusts.
This is the second root cause and the more decisive one: an AI’s output is capped by the context it reads. Scattered context caps AI at autocomplete. Structured context is what unlocks agents.
AI-assisted companies document in Google Docs and Confluence. New developers spend weeks getting up to speed because context is scattered and unstructured. Agents can’t touch any of it.
AI-native companies structure project knowledge so both humans and agents can use it.
New team members onboard from the knowledge base. Decisions are logged in machine-readable form. The spec isn’t just legible to a person sitting down with a coffee, it’s consumable by the tool that picks up the next task.
And this is where the gains actually compound. The context captured in month one is still working in month six, instead of evaporating at every handoff and every new joiner. Faster pieces reshuffle the bottleneck; compounding context is what moves whole-delivery speed.
What to ask: “If I join your team mid-project, how do I get up to speed? And how does an AI agent on your team get that same context?”
If those are two different answers, the knowledge layer isn’t there.
Stop asking these questions, they don’t tell you anything
Three questions that feel sharp and aren’t:
“Do you use AI?”
The answer is always yes. 99% of teams do.
“How much faster are you with AI?”
You’ll get a number from the one phase where AI performs best. It tells you nothing about whether anything else changed.
“What AI tools do you use?”
Tool names are not a delivery strategy. Copilot, Claude, Cursor, none of it matters if the process around them is identical to 2024.
The right question isn’t “do you use AI?” It’s “did AI improve the overall speed of delivery?” Everything in the chart below is that question, made concrete.
The question that actually gets you somewhere
The pattern, once you see it, is consistent.
AI-assisted companies sped up implementation and left every other phase untouched.
AI-native companies recognized that faster coding doesn’t solve the problem, it creates new ones in reviews, QA, knowledge management, and coordination, and they rebuilt around that.
One question exposes which camp you’re dealing with faster than anything else:
“Show me a project where AI made something go wrong, and what you changed because of it.”
Any team that’s genuinely pushed this has a story. If they don’t have one, they haven’t gone far enough to learn anything real yet.
Ultimately, this was never about AI
None of these questions are really about AI. They’re about what you’re actually buying: working software, estimates that hold, fewer surprises in sprint five.
And the rebuild only pays off when the people directing the agents understand your domain. In banking, wealth management, and fintech, an agent doesn’t know your compliance boundary, your regulator, or why that legacy integration can’t be touched, so the team directing it has to.
Domain knowledge is what makes agent output trustworthy; the delivery model is what makes it fast.
So judge digital transformation consultants on outcomes and domain first. Then use these questions to find out whether the AI underneath is native or just along for the ride. And if a vendor believes their own story, they’ll take the simplest test there is: run one engagement AI-native, measure it, and let the numbers settle the argument.
Frequently Asked Questions About Delivery Partners
How do I choose the right AI delivery partner for a banking or fintech project?
Judge on outcomes and domain knowledge first, not name-dropping tools. Ask how AI participates in estimation, what a ticket looks like on their team, and where productivity hasn’t improved. A partner who can’t answer that last question honestly likely hasn’t rebuilt their delivery model around AI, they’ve just given developers access to it.
Should we build AI capabilities in-house or bring in an AI-native delivery partner?
It depends on whether your team already has agents directing structured, repo-based context, not just AI tool licenses. If knowledge is still scattered across docs and Slack, an in-house build will hit the same bottleneck an AI-assisted vendor does: faster coding, unchanged delivery speed. A partner is worth it if they’ve already solved the context problem, not just the coding-speed problem.
What’s the difference between an AI transformation consulting partner and a vendor that just uses AI tools?
A transformation partner restructures the delivery model itself: specs, architecture, and knowledge become machine-readable so agents can act on them directly. A vendor that “uses AI tools” gives developers a faster editor and leaves reviews, handoffs, onboarding exactly as it was.
How do I evaluate whether an AI vendor’s estimates are actually AI-driven?
Ask for a specific example where AI changed an estimate, not just reformatted one. AI-native teams build a cheap prototype before estimating and let agents verify assumptions against working cod, while AI-assisted teams still estimate from gut feel and use AI only for research.
