Your First AI Agent: The Realistic Path From Use-Case to Production

The use-case is chosen, the budget is approved, everyone is motivated. Three months later the agent is in the pilot graveyard. Not because the technology could not do it, but because six steps were skipped. That is the pattern we see in most failed Mittelstand projects in 2026, and it almost never comes down to missing technical skills. It comes down to discipline, at exactly six points. Here is the realistic path from a chosen use-case to genuine production, honestly paced over roughly twelve weeks, without the hero stories.

Prerequisite: The Use-Case Is Set

This post starts at a very specific point: the question "which use-case do I pick first" is already answered. If you are not there yet, work through the use-case selection with the 90-day matrix first. That is where you decide which process makes a good first project, based on data quality, volume, error tolerance and value contribution. Everything that follows builds on that.

The assumption for the rest of this post: you have a clearly named process, a rough sense of its value, and at least one person who does the process manually today and can answer your questions. You need nothing more to start. What you need now is a path that does not fizzle out. And it consists of six steps that build on each other. Skip one, and you pay double later.

Step 1: Sharp Scoping (Week 1 to 2)

The most common mistake with a first agent is not too little ambition, but too much. The first agent is supposed to take over "all invoice checking" or "the entire first-level support" right away. That is too broad, and too broad means: not measurable, not acceptable, never finished.

Sharp scoping means cutting the first agent so narrowly that one person can say in a single sentence what it does and how you recognise success. Not "handles invoices", but "compares incoming supplier invoices against the matching purchase order and flags discrepancies above 100 EUR for manual review". That is narrow, that is measurable, and exactly because of that, buildable.

Three things belong in this scope document, and it fits on one page. First, the goal in one sentence, with a clear input and output. Second, the success criteria as a number: which hit rate, which maximum error rate, which processing time counts as success. "Works well" is not a criterion, "detects at least 90 percent of discrepancies with no more than 5 percent false alarms" is one. Third, the explicit boundary: what the agent expressly does not do. This negative list matters more than most think, because it prevents the creeping bloat that suffocates pilots.

Why this sharp scoping helps in a counterintuitive way we explained in the agent anatomy: agents break on ambiguous goals, not on hard ones. A narrowly defined agent is not the more modest one, it is the more reliable one. You can expand later. But you cannot expand what never went to production.

Step 2: Eval Harness From Day 1 (Week 1 to 2, in Parallel)

This is the step almost everyone skips, and it is the most expensive mistake. Before you build a single line, you need an answer to the question: how do I measure whether this agent is good enough. This measuring instrument is called an eval harness, and it is created in parallel with scoping, not at the end.

At its core, an eval harness is a collection of real cases with a known correct answer. You take thirty to a hundred actual past cases where you know what the result should be. In the invoice example: thirty real invoices, some with known discrepancies, some clean, a few edge cases like partial deliveries or cancellations. For each case it is fixed what the agent should output. That is your test bench.

The effect is fundamental. Without an eval harness, "the agent is good" is a gut-feeling statement that collapses in every steering meeting. With an eval harness it is a number: "the agent correctly detects 28 of 30 discrepancies, with two false alarms." That is acceptable, that is defensible, and with every change it shows you immediately whether you got better or worse. Without that number you build blind, and agents built blind do not go to production, because no one has the courage to sign off on them.

The rule of thumb from our workshops: whoever builds the eval harness only after the agent stands usually does not build it at all, and the pilot dies on the question "is this good enough now". The eval harness is not a luxury for data scientists, it is the precondition for your project to reach a sign-off in the first place. Half a day of collection work at the start saves you weeks at the end.

Step 3: Build the Pilot (Week 3 to 6)

Only now do you build, and here too discipline is decisive: small, with real data, with tightly limited tool access, closely observed. The pilot is not a miniature production system, it is a controlled experiment that answers one question: does this agent reach the success criteria from step 1, measured against the eval harness from step 2.

Real data from the start is non-negotiable. An agent that looks good on invented sample data says nothing about the reality where invoices are scanned crooked, order numbers are missing and suppliers use creative line-item descriptions. It is exactly this messiness that is the real test. If the pilot does not run on real data, it never runs.

Tool access belongs tightly limited in this phase. The pilot agent may read, compare, propose, but it does not touch anything that cannot be undone. No automatic sending, no automatic booking, no deletion. What the agent can and cannot do is an architecture decision, not a later configuration, and the most common pitfalls are covered in the 5 architecture failures from pilot to production.

Closely observed means: every run is logged, every decision of the agent is traceable, and a human reviews the results daily. This is where you clarify the questions still open in the scope document: how does the agent behave on edge cases, where does it hallucinate, where is it unsure, at which points does it need more context. By the end of week 6 you either have an agent that meets the eval criteria, or a clear explanation of why not, and both are a usable result. What an agent fundamentally cannot do, no matter how well the pilot runs, is laid out in what AI agents cannot do.

Step 4: Guardrails and Human-in-the-Loop (Week 5 to 7)

In parallel with the late pilot you decide where the agent may act autonomously and where not. The guiding question is not "how smart is the agent", but "how reversible is the action". This distinction is the most important safety decision in the whole project.

Reading and preparatory steps the agent can largely do autonomously: fetch data, compare, build a draft, formulate a recommendation. Irreversible actions belong behind a human approval: transferring money, sending emails to customers, deleting records, signing contracts. Human-in-the-loop here is not distrust of the technology, but clean architecture. You build a gradation per action type, not a switch set to "all or nothing".

Three guardrail mechanisms belong in every first agent. First, hallucination catching: when the agent makes a statement, it must be traceable to a source, otherwise it is flagged as uncertain instead of output as fact. Second, thresholds: above a defined uncertainty or a defined amount, the agent automatically escalates to a human instead of deciding itself. Third, the escalation path: it must be clear who a case goes to when the agent cannot proceed, and that human must receive the case with all context, not just an "I cannot do this".

The point most people think too little about: what happens in the failure case. An agent without a defined failure path either fails silently, in which case no one notices, or it does something anyway, which is worse. A good guardrail ensures that the worst case is a case escalated to a human, never an irreversible loss.

Step 5: Gradual Rollout (Week 8 to 12)

Once the agent meets the eval criteria and the guardrails stand, the rollout begins, and here one rule holds without exception: never big-bang. The agent does not go live for everyone on Monday. It goes in three stages, and each stage has a clear abort criterion.

Stage one is shadow operation. The agent runs in parallel with the human, on the same real cases, but its output is not yet acted upon. The human keeps doing the work as before, and you compare: where does the agent agree with the human, where does it diverge, and who was right. Shadow operation is the most honest trial there is, because it runs against real reality without a mistake hurting. Two to three weeks of shadow operation in one team are well invested.

Stage two is partial autonomy in one team. The agent now handles the clear cases on its own, the uncertain ones still go to the human, and this one pilot area gathers experience before it goes wider. Stage three is the expansion to further teams, one after another, carrying the experience from the first area along. First one team, then wider, never everything at once. Why the big-bang rollout fails so reliably is shown by the pilot graveyard with concrete patterns.

The reason for this staging is not timidity, but learning economics. Each stage uncovers problems that were invisible in the previous one, and every uncovered problem is cheaper to fix as long as it only affects one team. A big-bang spreads every problem to everyone at once and thereby costs trust you only have once.

Step 6: Operations Concept (From Go-Live On)

Go-live is not the end of a project, it is the start of operations. That is the thinking error that causes the most expensive late damage: the agent goes live, the project team disbands, and six weeks later no one notices the hit rate slowly slipping because the input data has changed. An AI agent is software operations, not a finished project.

Four questions the operations concept must answer before the agent goes to production. First: who monitors. You need a named operations owner, a person with a name, not "the team", who is responsible for ongoing quality and runs the eval harness from step 2 regularly. Second: how is it versioned. Changes to the agent, the prompts or the models must be traceable and reversible, otherwise after three months you no longer know why it behaves differently than in the pilot.

Third: how do you react to drift. Drift is the slow slipping of quality because the world changes: new suppliers, new invoice formats, a model update from the provider. The eval harness is your early-warning system here, because it states quality as a number at any time. Fourth: who is responsible when the agent makes an expensive mistake. This question must be answered before go-live, not in the event of damage. The ongoing operating costs of this responsibility, from monitoring through model fees to maintenance, are calculated in detail in the TCO post over 12 months. Whoever wants to set up the whole path as an engineering programme finds the larger structure in the 5-phase roadmap for engineering teams.

The 4 Places Mittelstand Firms Fail in 2026

In most failed first projects we see, it is one of these four places, and none of them is a technical problem.

First, no eval harness. Without a measurable test bench, "is the agent good" stays an opinion, and opinions do not reach a sign-off. Second, too broad a scope. The first agent is meant to do too much at once, is never finished and never acceptable. Third, big-bang rollout. The agent goes live for everyone at once, the first real problem hits everyone immediately, and the trust is gone before the agent had a chance. Fourth, no operations owner. The agent goes live, the team disbands, no one notices the drift, and after three months the agent is worse than the human it was meant to replace. The overarching patterns behind such cancellations Gartner has quantified: more than 40 percent of agentic-AI projects will be scrapped by the end of 2027 according to its press release from June 2025, and the anti-patterns behind them map almost one to one onto this list.

FAQ

How long does this realistically take?

For a narrowly cut first agent, the roughly twelve weeks described here are a realistic frame, from scoping to the first production team. That is a Sentient workshop aggregate from 40 DACH projects, not a promise. More complex use-cases take longer, simpler ones can go faster. What cannot be shortened without endangering it is shadow operation and the eval harness. Cut corners there and you save weeks but lose months.

Do I need a dedicated team for this?

For the first agent, not necessarily. What you need is a person who understands the process on the business side and can answer questions, and someone with the technical craft who can build, internal or external. What you need from go-live on is a named operations owner, and the effort for that is small, but it must exist. A dedicated AI team is best built only once the first agent has proven its value, not before.

What does the first agent cost?

That depends too heavily on the use-case for a serious flat number, and anyone who names one upfront is guessing. What can be said: model fees are usually the smallest item for a first agent, the larger effort sits in scoping, eval harness and accompanying the rollout. And ongoing operation keeps costing, even after go-live. We have broken down the realistic items over 12 months in the TCO post, so you see not only the build but also the operating costs.

What if the pilot fails?

A pilot that does not meet the eval criteria is not a lost project, provided you had the eval harness from day 1. Because then you know exactly what it fails on: the data, the scope, the use-case itself. That is usable knowledge that makes the next attempt cheaper. Lost is only the pilot that ran without a measuring instrument and died on a diffuse "somehow this is not working". That is exactly why step 2 is non-negotiable.

Is an agent even the right approach, or is classical automation enough?

A fair question that belongs before scoping. If your process has fixed rules, never changes and needs no language processing, classical automation or RPA is often cheaper and more robust. An agent only pays off when the task requires ambiguity, language or adaptation to intermediate results. The distinction in detail is in the post on AI agent vs RPA vs automation.

---

Sources:

• Sentient Dynamics workshop aggregate, 40 DACH workshops 2025-2026 (headcount 80 to 4,000)
• Gartner press release, June 2025 (over 40 percent of agentic-AI projects scrapped by the end of 2027)
• MIT NANDA Report 2025: "GenAI Divide: State of AI in Business 2025" (95 percent of GenAI pilots with no measurable P&L effect)
• McKinsey State of AI, November 2025
• Bitkom AI study 2025 (German companies with 20+ employees: 41 percent adoption; from 500 employees: 89 percent)

Next step: If you want to plan the path to production for a concrete, already chosen use-case, book 30 minutes via our demo page. We bring the six steps, an honest look at your scope and three questions, no vendor deck. If the use-case is not set yet, start with the 90-day use-case matrix, and whoever wants the full market context for 2026 finds it in the 6 developments affecting the Mittelstand in 2026.