If you are new here, here are the first three articles in this series:

1. What Is an AI FDE?

2. What Is an AI FDE Pod?

3. Five Phases of an FDE Engagement

Starting there will provide valuable context for all upcoming FDE topics.

Today, we’ll go deeper into the first two phases: technical discovery and scope.

In both, the FDE problem-finder within the pod takes the lead.

The problem-finder figures out what’s really wrong, then what to fix first.

Why is this important? Well, better AI tools with a shit stratgegy has proven futile.

Take this recent BCG study that asked ten thousand people who often use AI at work:

Did AI give you results you could measure?

When the plan wasn’t clear and tools were few, 55% said yes.

Better tools alone barely helped (60%).

A clear plan, even with fewer tools, raised that to 80%.

So it makes sense, that a part of my job, is to vet the work before we build.

A request can sound right and still miss the real problem.

What’s this look like in the real world?

Let’s use a pretend client we all know: your favorite local pizza chain.

They reached out because customers wait too long for their pizzas.

The owner wants AI to take phone orders.

We have a problem, and a proposed solution, but is it the right problem?

Phase 1: Find the right problem

This is the technical discovery phase.

At IBM, where I work, its official name is Technical Qualification.

1. Watch the real work

For the pizza parlor, I’d stand behind the counter and watch a few orders.

What do the workers do first?

Where do they write the order down?

What happens when something is unclear?

Say a worker takes a call and writes the order on a paper slip.

The slip sits on the counter until a cook grabs it.

The writing is messy, so the cook can’t read “no onions.”

The cook has to ask, but the worker is on another call.

So the cook waits.

Now we have something to look into.

Is the slow part the phone call, or everything after it?

I’d follow the pizzas that get made again too.

Say 1 in 6 pizzas come out wrong and get made again.

Most of those started as a messy slip.

Ok, so messy slips may be the real problem, but why and how do I quantify it?

2. Measure what happens today

One messy order gives us a place to start.

But in order to understand why, we still need to know how often and what it costs.

We’d figure this out by measuring three things:

• Speed: minutes from the call to a ready pizza

• Accuracy: how often a pizza comes out right the first time

• Cost: pizzas thrown away, plus the time to make them again

Those numbers are our starting point.

I’d also split working time from waiting time.

Say the call takes 3 minutes, but the slip waits 15 minutes before a cook sees it.

Then a faster phone call would barely make a dent in the total time.

Hence, barely solve the problem.

Now we have a good baseline covering easy orders.

The shop’s best workers can tell us about a few of the harder ones.

And wa-lah, we have a starting point!

But wait! Before we move any further, we all must be on the same page.

Does the pizza shop owner still think he needs to use AI to take phone orders?

If so, you still have some reframing to do.

3. Agree on the problem

The workers can show us what slows them down.

The shop’s tech person knows where orders are kept and what we can use.

But the owner (or “sponsor” as we like to call them) has to agree it’s worth fixing.

I need all three in the same room.

Say the owner wants shorter phone calls.

But the cooks say the messy slips are the real problem.

We need to sort that out before we decide what success looks like.

Good thing we have our problem-finder all over it!

Finally, once we are all in agreement, we write the problem down in a few plain lines.

That’s the problem statement.

For our pizza chain, it might read like this:

Orders get written in a hurry and sit too long
before cooks see them.
Some pizzas come out wrong and get made again.
We want pizzas ready sooner, with fewer do-overs.

The real version adds our starting numbers.

Then the owner signs it and asks the shop’s tech person to work with us.

We may still end up building an AI helper.

Or we may find that a simple screen in the kitchen fixes enough of the problem.

I’d rather learn that now than after weeks of building.

Say we pick the AI helper.

It can send clear orders straight to a screen in the kitchen.

But how do we make sure we don’t end up “agentifying” everything in that kitchen?

It can send clear orders straight to a screen in the kitchen.

Phase 2: Make it small enough to finish

A clear problem can still turn into a project that never ends.

I still remember one of the first freelance web development contracts I ever did.

Amazing project, real builders backing me, great cash money to be made.

But every time we met, another feature would be added to the road map.

Another button would need to be moved again.

Another flow, another profile, another color, another another another.

Before you know it, BOTH sides end up spending more time than its worth.

Ends up turning a winning dream project into an unrecognizable lose-lose situation.

Same goes for our pizza parlor example.

Before long, someone asks, “Can it take delivery orders too?”

Someone else wants it to take payments.

“How about that Pizza Pete chat bot to really take it to the next level?”

Those requests change the job we signed up for.

But that’s where our beloved Scope comes in.

Scope means we decide what to build first, and what waits for later.

We also agree on how we’ll grade the result. (This is KEY).

The companies that get the most from AI show this same focus.

Another BCG study surveyed more than 600 big US companies.

The biggest wins from AI went to the top 6% of those companies.

BCG says those companies don’t try AI everywhere.

They pick a few big projects and go deep.

4. Pick one piece someone can use

I’d start with pickup orders at one store.

The AI answers the phone and takes the order.

It reads the order back to the customer.

The order shows up on a screen in the kitchen.

A worker checks it before the cooks start.

We’d work with the shop’s tech person on two things:

• Which of the shop’s computer programs the AI can use

• How orders show up on the kitchen screen

Linking the AI to them is part of the job we’re agreeing to do.

Delivery, payments, and other stores wait for later.

We can add them once we know this helps.

5. Agree on what “done” means

Think of it like a test at school.

Before we build, we agree on what counts as a passing grade.

What makes an order right?

Is every topping right?

Is the size right?

How much does the worker have to fix?

At IBM, we write this down as a definition of done, with a passing score.

The owner signs it.

Why be this strict?

AI can be great at one job and bad at the next, even when the jobs look alike.

In one study, 758 BCG workers did real tasks with and without AI.

AI helped them work faster on tasks it was good at.

But on a task picked to be too hard for AI, fewer people got it right.

Without AI, 84.5% got it right.

With AI, only 60% to 70.6% did, depending on the group.

What does the study teach us? Well, simply, AI can’t do everything better.

So we test it before we trust it.

The shop’s best workers pick at least 20 real calls from the shop’s recordings.

For each one, they write down the right order.

That’s the AI’s practice test.

We help turn their know-how into a test the team can run the same way every time.

But we don’t just include that calls AI should handle.

To make a reliable system, we’d have to include calls the AI shouldn’t handle too.

A caller with a food allergy is one example.

We need to know what the AI does with those before anyone relies on it.

We need the practice test and the signed definition of done before we build.

We’d also agree how to compare the new speed with today’s numbers.

6. Decide what happens when it gets stuck

Say a caller asks something the AI can’t handle.

It should say so and pass the call to a worker.

The worker takes it from there, the usual way.

We agree on exactly what the worker sees and does next.

When the AI isn’t sure, a person takes over.

Calls include names and phone numbers.

So we decide who can see what, and we keep a record of what the AI does.

Safety and security experts help set those rules during Scope.

The people in charge of safety give their official OK while we build.

Using it with real customers still needs the owner’s OK.

A worker checking each order is one safety net.

We still need proof that the whole system is ready.

7. Take new requests seriously

Delivery orders may turn out to be worth adding.

I’d want to know the extra work first.

Then we agree with the owner on what changes: the job, the people, and the timing.

A good idea in a meeting shouldn’t quietly turn into another promise.

I want us to finish something the shop can use and see how it works.

Then that experience can guide what we take on next.

The short version

Remember my urge to jump straight to the fix?

I still have it.

Now I hold it until two things are true:

1. Everyone agrees on the real problem, and we’ve measured how things work now.

2. It’s small enough to finish, and the owner signed what “done” means.

That’s the problem-finder’s job.

Before your next project starts

Ask your team these questions before anyone builds:

1. Have we watched the real work?

2. Do we know today’s numbers?

3. Did the leader in charge sign the problem statement?

4. Are we building just one piece first?

5. Is “done” written down and signed?

6. Do we know what happens when the AI gets stuck?

7. Do new requests get a real decision?

If any box is empty, you aren’t ready to build yet.

Coming up

Before this AI works with real customers, the people in charge of safety need proof.

Gathering that proof is a big part of the governance lead’s job.

That’s the teammate who makes sure the AI is safe to use.

Next up: How an AI System Earns Trust

Disclosure: I work at IBM as an AI Forward Deployed Engineer. I write here in my personal capacity, and the views are my own. This is my interpretation of the role, not an official IBM publication. The example above is fictional and contains no client-confidential information.

But when McKinsey asked those people how far along their companies were, only 44% said AI was scaling across the whole company.

The rest said they were still testing it or running pilots.

That gap makes sense to me.

I spent years showing clients what our tech could do before they bought it.

I’ve given more demos than I can count. 😂

But the first demo is really just meant to show how an idea might work.

It can’t prove that the data will hold up, that people will use it, or that the client can run it independently.

As an AI Forward Deployed Engineer, or FDE, at IBM, that’s exactly what we aim for.

An FDE works side by side with a client to build AI and put it to work in their business.

We work in a small team called a pod.

And every project we choose to take on with a client is called an engagement.

Here’s the full path, from the first look at the work to client ownership.

Five phases, five questions

An FDE engagement moves through five phases: Technical Qualification, Scope, Build, Deploy, and Adopt.

Each phase answers one question and ends at a checkpoint.

We don't move on until the checkpoint is met.

Each phase below has five parts: what it is, how we run it, the checkpoint, what it produces, and how we measure it.

We track early signs that a phase is on track, and proof that it's done.

All the proof must be in before we move on.

1. Technical Qualification: Are we solving the right problem?

RAND talked with 65 data scientists and engineers about why AI projects fail.

The first root cause on their list: people misread, or poorly explain, the problem the AI is meant to solve.

What it is: We learn how the work really runs, not how the manual says it runs.

This phase runs on every engagement.

How we run it:

1. Shadow the real work, workarounds and all.

2. Map the whole flow: people, systems, data, decisions, handoffs, and weak spots.

3. Turn the pain we see into ideas we can test.

4. Measure today’s accuracy, speed, and cost. That’s the starting line.

5. Name the client engineer who will pair with us.

Checkpoint:

• We know the systems, the data, and how they integrate.

• The value case is in writing: the limit, the starting line, and the target.

• A small first version or workshop shows the problem is real.

What it produces:

• A written problem statement

• Today’s numbers for accuracy, speed, and cost

• A map of how the work runs today

• A named client engineer

The problem statement answers five questions. Fill it in like this:

What must improve: [task] takes [time] and goes wrong [how often].
Who will use it: [role].
Data it needs: [sources].
Limits: [rules, systems, budget].
Success: [measure] goes from [today] to [target].

How we measure it:

• On track: The sponsor meets us and we watch at least 80% of the steps firsthand.

• Done: Sponsor signs the problem statement and all three starting numbers are in.

2. Scope: What is the smallest useful win?

What it is: Scope turns what we found into work one engagement can finish.

We shrink a big goal to one workflow, one user, and one decision.

Then we write down what “done” means before anyone builds.

In our experience, skipping this step is a common reason a pilot never goes live.

How we run it:

1. Narrow to one use case: one workflow, one type of user, one decision.

2. Plan the path: how data maps, how systems connect, and what we build first.

3. Build the test set with the client. Their experts write real cases.

4. Write the definition of done: the passing score and what counts as a failure.

5. Start the approvals early. Bring in the product, security, and commercial teams.

Checkpoint:

• A build plan, step by step.

• A people plan, with both sides confirmed and free.

• A target go-live date, with what it depends on.

What it produces:

• A written definition of done

• A test set written by the client’s experts

• An agreed plan and build order

• Approvals under way with the right teams

How we measure it:

• On track: All the test cases we asked for are in and one use case is in scope.

• Done: The client signs the definition of done. The full test set is in hand.

3. Build: Can it work from end to end?

What it is: One real use case to working result on the client’s real data and systems.

Then we make it safe to run: tested, secured, watched, and easy to undo.

We end with a working system and the proof a risk team needs to approve it.

How we run it:

1. Build the thinnest path first: One slice, not many half-built parts.

2. Make it solid from day one: clean outputs, tests, and error handling.

3. Run the test set every week and share the score. The trend is the status.

4. Build the proof: docs, tested failure cases, known limits, and a plan to watch it.

5. Clear security and governance, the rules for safe AI use.

6. Plan for failure: a backup, a way to send hard cases to a person, and an off switch.

7. The client engineer writes the how-to guide, then runs the system end to end.

Checkpoint:

• A first version works on real data in a test setting.

• The test score meets the passing bar.

• Approvals are in, the undo is tested, and we can see what it’s doing.

• The client can run it without us, with a clear path to go-live.

What it produces:

• A working system on real data

• A test score at the bar

• Sign-off from security and the risk team

• A how-to guide the client wrote

• A list of product gaps and what each one costs the business

How we measure it:

• On track: The weekly test score keeps going up until it hits the bar.

• Done: The final score meets the bar. Both approvals are in.

4. Deploy: Can it hold up in real work?

What it is: We move the system into real use in steps: watch-only, then a small live group, then everyone. We also check that people really use it.

Many projects never get this far.

In an S&P Global survey, 42% of companies said they dropped most of their AI projects between a first test and real use. A year earlier, 17% said that.

How we run it:

1. Run it in watch-only mode. It answers live work, but no one acts on it.

2. Go live for one team, region, or slice of the work.

3. Switch everyone over only when those changes are steady and below the limit.

4. Track use from day one: who uses it, how often, and their workarounds.

5. Keep it steady. Watch for slipping quality, rising cost, and slow answers.

Checkpoint:

• The first version runs in the client’s live setting.

• The client signs off that it does what we agreed.

• People change its answers less than the agreed limit, and that rate holds steady.

What it produces:

• A system doing real work

• A low, steady rate of changed answers

• Use tracked from day one

• A stable system, with quality and cost under watch

How we measure it:

• On track: Watch-only mode, disagrees with people no more than the agreed limit.

• Done: Changed answers stay at or below the limit. More people use it each week.

5. Adopt: Can the client run it without us?

What it is: We hand full ownership to the client and save what we learned.

How we run it:

1. Flip the pairing. The client engineer drives. We watch and step in only if asked.

2. Prove they can run it alone. The client engineer ships one change end to end.

3. Hand over ownership: the code, the budget line, the alerts, and the test set.

4. Review value with sponsor. Compare the result to the target and the starting line.

5. Save the pattern to our shared library, and set check-ins at 30 and 90 days.

That value review matters.

In IBM’s 2025 CEO study, CEOs said only 1 in 4 of their recent AI projects paid off.

Checkpoint:

• We test it with the client on real work.

• We agree with the client on the plan to roll it out and grow it.

• It reaches all the users it’s meant for, and the client formally takes ownership.

What it produces:

• A change the client shipped and undid on their own

• Ownership formally handed to the client

• A finished value review, and a client who agrees to be a reference

• The pattern saved, with check-ins set

How we measure it:

• On track: The client ships at least one change end to end, undo included.

• Done: The client runs it live without us for an agreed time. All handover items have moved. The value against the starting line meets what is in Scope.

Putting it all together

Remember the gap we started with?

Most companies use AI. Fewer than half of those are scaling it.

These five phases are how we aim to close that gap:

1. Technical Qualification finds the right problem.

2. Scope shrinks it to one win we can finish.

3. Build gets it working safely on real data.

4. Deploy proves it holds up in real work.

5. Adopt hands it to the client for good.

Each phase ends with proof, not a guess.

A demo shows it can work. This path helps make it last.

Try this with your team

If your team is working on AI right now, ask two questions:

1. Which phase are we really in?

2. What proves we’re ready for the next one?

If you can’t answer the second question yet, you aren’t ready to move on.

I call the work in phase one problem-finding.

Next, I’ll show how we trace the work, measure today, and agree on the problem.

Next up: AI FDE Role: The Problem-Finder

Disclosure: I work at IBM as an AI Forward Deployed Engineer. I write here in my personal capacity, and the views are my own. This is my interpretation of the role, not an official IBM publication. The example above is fictional and contains no client-confidential information.

Our job was reconnaissance and security. Simply put, we gathered information about the terrain and the enemy so commanders knew what they were sending people into.

Every scout trains the same way. One Station Unit Training (OSUT) combines basic soldiering and the scout trade into one continuous 22-week program.

First, 10 weeks of the fundamentals: rifle marksmanship, land navigation, first aid, and basic communications. Then, 12 weeks of the craft: reconnaissance, vehicle operations, crew-served weapons, calling for fire, and reading terrain.

By the end, each of us could work every position in the squad and understood how they fit together.

Once we graduated and were assigned to our units, the mission set the roles. Mounted, one scout drove while another manned the gun. On foot, one took point, one carried the radio, one shouldered the M240B.

We were all trained to do every aspect of a scout’s job.

On a mission, your squad counted on you to do one.

That’s the part that feels familiar in an IBM AI FDE pod.

My OSUT platoon at Fort Benning. That’s me in the red square.

Same team, different responsibilities

A pod is the team working in the client’s environment to take an AI use case into production and help their people run it.

One FDE stays accountable for seeing the entire engagement through. The pod does the work together. IBM’s architecture, platform and product specialists can step in when the work needs them.

We come into FDE work with different backgrounds and strengths. But we all need a working understanding of four things: finding the right problem, building the system, earning trust, and keeping the engagement moving.

On each engagement, we put those strengths to work and agree who leads each one. Knowing the other jobs helps us back each other up without dropping our own.

In my pod, I lean toward the Engagement Lead role, drawing on my years in technical pre-sales at IBM. I spend a lot of time getting to the problem beneath the request, connecting technical and business teams, and making sure what we build gets used.

Nonetheless, I still need to understand what we’re building.

If an integration changes what we can deliver, I work through the tradeoffs with the engineers and explain them to the client.

Four essential skills sets, not necessarily four people.

Take the fictional claims-routing assistant from my first article. The client wants AI to send claims to the right team. Here’s how I think about the work we need to cover:

Finding the right problem:

The problem-finder leads discovery and sits with the claims team to understand where work gets stuck. Are claims going to the wrong queue, or do people lack the information to act on them? We need that answer before we decide what to build.

Building the system:

The builder leads the development work alongside the client’s engineers to connect the system, test it on real cases and make sure failures get caught and handled. They also help prepare the client’s team to run it.

Earning trust:

The governance lead brings in the client’s security and model-risk reviewers early and makes sure the build meets their requirements. Can we explain a routing decision? Who can override it? What evidence do we need before it goes live?

Keeping the engagement moving:

And finally, (my personal specialty) the engagement lead, keeps the sponsor involved, gets decisions made and checks whether the claims team actually uses the result. If the tool doesn’t add value to a process people already hate, the pod still has work to do.

Those responsibilities overlap. We work through them together, but each needs an owner the rest of the team can count on.

The work overlaps. And each responsibility has an owner.

Do your job

I like the line Bill Belichick made famous: “Do your job.”

On a football team, everyone needs to understand the play and carry out their assignment.

That’s how championships get won, missions get accomplished, and AI use cases make it to production. Everyone understands how the jobs fit together. Everyone gives relentless effort to the one that’s theirs.

In my pod, if I’m responsible for getting a decision from the sponsor, the team counts on me to get it. If I’m stuck, they need to know while there’s still time to do something about it.

And things change. A broken integration might reveal that we misunderstood the workflow. The builder needs help from whoever led discovery, and I need a decision on scope.

Understanding how our individual assignments connect helps us catch problems early and adapt together.

Everyone knows the play. Everyone runs their assignment.

First, agree on the problem

Before any of those assignments matter, we need to agree with the client on what we’re trying to accomplish. Whose work should improve? What’s getting in their way? How will we know we made it better?

Otherwise, we can all do our jobs well and still build the wrong damn thing.

That’s why commanders send scouts out first.

Finding the right problem is an AI FDE’s reconnaissance.

Next up: Five Phases of an FDE Engagement.

Disclosure: I work at IBM as an AI Forward Deployed Engineer. I write here in my personal capacity, and the views are my own. This is my interpretation of the role, not an official IBM publication. The example above is fictional and contains no client-confidential information.

AI Forward Deployed Engineer.

For anyone wondering what I did to get the job, spoiler alert: it was a complete surprise to me.

I was essentially volun-told.

Once the surprise wore off, I started figuring out what I actually signed up for. I never planned my career around becoming an AI FDE, and most people I talk to ask the same question:

What does that even mean at IBM?

After a few conversations and a lot of reading, I had a second reaction.

I was born for this shit.

The way we explain it to clients comes down to six words: from working demo to working business.

At IBM, an AI FDE works inside the client’s environment with the systems and data the business actually uses.

The goal is to take one important AI use case out of demo mode and put it into production, where people use it and the client can keep it running.

IBM saw what I had been building, trusted me to do it for clients and made it my day job.

Grateful is an understatement. It’s an honor.

Six and a half years of enterprise pre-sales at IBM, then this line changed.

What happens after the demo?

I know how convincing a good demo can be, especially when the data has already been cleaned and the person who built it is running the call.

Then the demo ends, everybody goes back to work and somebody has to make it real.

The same system now has to connect to whatever the client already runs, usually with uglier data than anyone wants to admit. Security has questions. The users are busy, and somebody still needs to own it when it breaks.

The pilot ran on a clean extract with the vendor on the call.

Production runs on a Tuesday at 4 p.m., on data nobody cleaned, with nobody from the vendor in the room.

MIT Project NANDA put a number on that gap in 2025.

Its preliminary report estimated that only about 5% of task-specific enterprise GenAI tools reached successful implementation with a sustained productivity or P&L impact.

You can read the full report here:

The GenAI Divide: State of AI in Business 2025

Production has to survive real data, real workflows and real users.

Here’s what that can look like

This example is made up, but the problems in it show up all the time.

Say an insurance team pilots an AI claims-routing assistant. It reads a clean batch of claims, sends each one to the right queue and recommends the next step. In the demo, it looks great.

Then the team tries it on real claims. One field means something different in another system, and an important note is buried in free text. Compliance asks how anybody will audit the decision.

Worse, the tool adds clicks to a process the claims team already hates.

At that point, the model is not even the main problem anymore.

That mess is the job.

The FDE sits with the people who run the systems and the people expected to use the result.

When something breaks, we follow it into the source system or the workflow and fix what is actually in the way. Then we test again with the people who will own it.

This is hands-on work.

We have to understand the tech and the room

The work changes with the client. You could spend one day tracing a broken integration and the next testing model output with someone who knows the process better than you ever will.

Sometimes the technical problem is obvious. Other times everything passes testing and the users still will not touch it.

You have to understand both situations well enough to ask whether the team is solving the right problem.

I have watched highly technical teams spend months building something nobody needed.

An FDE needs the credibility to build and the judgment to ask an uncomfortable question before the team wastes three months.

This job requires technical depth and good judgment.

How I ended up here

I did not come into this from a traditional engineering role.

I spent the last six and a half years in enterprise technical pre-sales at IBM. I gave more demos than I can count, built proofs of concept and helped close more than $25 million in business.

Most days, I was translating what the client needed into what the technology could actually do. When the work moved into production, I would bring in our Client Engineering team.

Many of those engineers are my teammates now.

My strongest lane is the engagement side. I spend a lot of time finding the problem beneath the request, getting the right people in the room and making sure what we build actually gets used.

The building side is something I have sharpened outside of work through my own projects. Now that part is becoming much bigger in my day job.

I did not walk in as the best software engineer in the room, and I will not pretend otherwise.

My advantage is that I can translate between the people building the system and the people who own the business problem.

There is not one resume that qualifies you for this work, and I will write a separate piece about becoming an AI FDE.

If you hate unclear requirements and technical systems that refuse to cooperate, this job will wear you out.

I happen to love that part.

The part I get to stay for

For years, my role often ended right when things got interesting.

I could prove the technology and help a client see the path. Then I would bring in the engineers who could take it into production.

Now I get to stay for the hard part.

I get to see what happens when the clean demo meets the client’s actual environment. I get to work through what breaks and find out whether the thing we built makes anyone’s job better.

People keep asking me what this new title means. The honest answer is that I am still learning the full shape of it.

What I know already is that it puts me closer to the work I have wanted to do for years.

The Army taught me that nobody carries a mission alone. I am already learning the same thing here.

An FDE is only as strong as the pod around them.

That pod deserves more than a quick explanation at the end of this article, so I am giving it the next one.

Volun-told or not, I would have chosen this.

Know somebody who keeps asking what the hell an AI FDE is?

Send them this.

Next up: What Is an FDE Pod?

Disclosure: I work at IBM as an AI Forward Deployed Engineer. I write here in my personal capacity, and the views are my own. This is my interpretation of the role, not an official IBM publication. The example above is fictional and contains no client-confidential information.