Now that we’ve gone over the basics, let’s get into the work itself.
If you are new here, here are the first three articles in this series:
Starting there will provide valuable context for all upcoming FDE topics.
Today, we’ll go deeper into the first two phases: technical discovery and scope.
In both, the FDE problem-finder within the pod takes the lead.
The problem-finder figures out what’s really wrong, then what to fix first.
Why is this important? Well, better AI tools with a shit stratgegy has proven futile.
Take this recent BCG study that asked ten thousand people who often use AI at work:
Did AI give you results you could measure?
When the plan wasn’t clear and tools were few, 55% said yes.
Better tools alone barely helped (60%).
A clear plan, even with fewer tools, raised that to 80%.
So it makes sense, that a part of my job, is to vet the work before we build.
A request can sound right and still miss the real problem.
What’s this look like in the real world?
Let’s use a pretend client we all know: your favorite local pizza chain.
They reached out because customers wait too long for their pizzas.
The owner wants AI to take phone orders.
We have a problem, and a proposed solution, but is it the right problem?
Phase 1: Find the right problem
This is the technical discovery phase.
At IBM, where I work, its official name is Technical Qualification.
1. Watch the real work
For the pizza parlor, I’d stand behind the counter and watch a few orders.
What do the workers do first?
Where do they write the order down?
What happens when something is unclear?
Say a worker takes a call and writes the order on a paper slip.
The slip sits on the counter until a cook grabs it.
The writing is messy, so the cook can’t read “no onions.”
The cook has to ask, but the worker is on another call.
So the cook waits.
Now we have something to look into.
Is the slow part the phone call, or everything after it?
I’d follow the pizzas that get made again too.
Say 1 in 6 pizzas come out wrong and get made again.
Most of those started as a messy slip.
Ok, so messy slips may be the real problem, but why and how do I quantify it?
2. Measure what happens today
One messy order gives us a place to start.
But in order to understand why, we still need to know how often and what it costs.
We’d figure this out by measuring three things:
Speed: minutes from the call to a ready pizza
Accuracy: how often a pizza comes out right the first time
Cost: pizzas thrown away, plus the time to make them again
Those numbers are our starting point.
I’d also split working time from waiting time.
Say the call takes 3 minutes, but the slip waits 15 minutes before a cook sees it.
Then a faster phone call would barely make a dent in the total time.
Hence, barely solve the problem.
Now we have a good baseline covering easy orders.
The shop’s best workers can tell us about a few of the harder ones.
And wa-lah, we have a starting point!
But wait! Before we move any further, we all must be on the same page.
Does the pizza shop owner still think he needs to use AI to take phone orders?
If so, you still have some reframing to do.
3. Agree on the problem
The workers can show us what slows them down.
The shop’s tech person knows where orders are kept and what we can use.
But the owner (or “sponsor” as we like to call them) has to agree it’s worth fixing.
I need all three in the same room.
Say the owner wants shorter phone calls.
But the cooks say the messy slips are the real problem.
We need to sort that out before we decide what success looks like.
Good thing we have our problem-finder all over it!
Finally, once we are all in agreement, we write the problem down in a few plain lines.
That’s the problem statement.
For our pizza chain, it might read like this:
Orders get written in a hurry and sit too long
before cooks see them.
Some pizzas come out wrong and get made again.
We want pizzas ready sooner, with fewer do-overs.The real version adds our starting numbers.
Then the owner signs it and asks the shop’s tech person to work with us.
We may still end up building an AI helper.
Or we may find that a simple screen in the kitchen fixes enough of the problem.
I’d rather learn that now than after weeks of building.
Say we pick the AI helper.
It can send clear orders straight to a screen in the kitchen.
But how do we make sure we don’t end up “agentifying” everything in that kitchen?
It can send clear orders straight to a screen in the kitchen.
Phase 2: Make it small enough to finish
A clear problem can still turn into a project that never ends.
I still remember one of the first freelance web development contracts I ever did.
Amazing project, real builders backing me, great cash money to be made.
But every time we met, another feature would be added to the road map.
Another button would need to be moved again.
Another flow, another profile, another color, another another another.
Before you know it, BOTH sides end up spending more time than its worth.
Ends up turning a winning dream project into an unrecognizable lose-lose situation.
Same goes for our pizza parlor example.
Before long, someone asks, “Can it take delivery orders too?”
Someone else wants it to take payments.
“How about that Pizza Pete chat bot to really take it to the next level?”
Those requests change the job we signed up for.
But that’s where our beloved Scope comes in.
Scope means we decide what to build first, and what waits for later.
We also agree on how we’ll grade the result. (This is KEY).
The companies that get the most from AI show this same focus.
Another BCG study surveyed more than 600 big US companies.
The biggest wins from AI went to the top 6% of those companies.
BCG says those companies don’t try AI everywhere.
They pick a few big projects and go deep.
4. Pick one piece someone can use
I’d start with pickup orders at one store.
The AI answers the phone and takes the order.
It reads the order back to the customer.
The order shows up on a screen in the kitchen.
A worker checks it before the cooks start.
We’d work with the shop’s tech person on two things:
Which of the shop’s computer programs the AI can use
How orders show up on the kitchen screen
Linking the AI to them is part of the job we’re agreeing to do.
Delivery, payments, and other stores wait for later.
We can add them once we know this helps.
5. Agree on what “done” means
Think of it like a test at school.
Before we build, we agree on what counts as a passing grade.
What makes an order right?
Is every topping right?
Is the size right?
How much does the worker have to fix?
At IBM, we write this down as a definition of done, with a passing score.
The owner signs it.
Why be this strict?
AI can be great at one job and bad at the next, even when the jobs look alike.
In one study, 758 BCG workers did real tasks with and without AI.
AI helped them work faster on tasks it was good at.
But on a task picked to be too hard for AI, fewer people got it right.
Without AI, 84.5% got it right.
With AI, only 60% to 70.6% did, depending on the group.
What does the study teach us? Well, simply, AI can’t do everything better.
So we test it before we trust it.
The shop’s best workers pick at least 20 real calls from the shop’s recordings.
For each one, they write down the right order.
That’s the AI’s practice test.
We help turn their know-how into a test the team can run the same way every time.
But we don’t just include that calls AI should handle.
To make a reliable system, we’d have to include calls the AI shouldn’t handle too.
A caller with a food allergy is one example.
We need to know what the AI does with those before anyone relies on it.
We need the practice test and the signed definition of done before we build.
We’d also agree how to compare the new speed with today’s numbers.
6. Decide what happens when it gets stuck
Say a caller asks something the AI can’t handle.
It should say so and pass the call to a worker.
The worker takes it from there, the usual way.
We agree on exactly what the worker sees and does next.
When the AI isn’t sure, a person takes over.
Calls include names and phone numbers.
So we decide who can see what, and we keep a record of what the AI does.
Safety and security experts help set those rules during Scope.
The people in charge of safety give their official OK while we build.
Using it with real customers still needs the owner’s OK.
A worker checking each order is one safety net.
We still need proof that the whole system is ready.
7. Take new requests seriously
Delivery orders may turn out to be worth adding.
I’d want to know the extra work first.
Then we agree with the owner on what changes: the job, the people, and the timing.
A good idea in a meeting shouldn’t quietly turn into another promise.
I want us to finish something the shop can use and see how it works.
Then that experience can guide what we take on next.
The short version
Remember my urge to jump straight to the fix?
I still have it.
Now I hold it until two things are true:
Everyone agrees on the real problem, and we’ve measured how things work now.
It’s small enough to finish, and the owner signed what “done” means.
That’s the problem-finder’s job.
Before your next project starts
Ask your team these questions before anyone builds:
Have we watched the real work?
Do we know today’s numbers?
Did the leader in charge sign the problem statement?
Are we building just one piece first?
Is “done” written down and signed?
Do we know what happens when the AI gets stuck?
Do new requests get a real decision?
If any box is empty, you aren’t ready to build yet.
Coming up
Before this AI works with real customers, the people in charge of safety need proof.
Gathering that proof is a big part of the governance lead’s job.
That’s the teammate who makes sure the AI is safe to use.
Next up: How an AI System Earns Trust
Disclosure: I work at IBM as an AI Forward Deployed Engineer. I write here in my personal capacity, and the views are my own. This is my interpretation of the role, not an official IBM publication. The example above is fictional and contains no client-confidential information.