Not in general terms. Specifically.

Not "it will save time." How much time? For whom? Measured against what baseline?

Not "it will improve outcomes." Which outcomes? By how much? Compared to what?

Not "it will be more efficient." Define efficient. Define the measurement. Define who is looking at the number and on what cadence.

If you cannot answer those questions before the pilot begins, you do not have a pilot. You have an experiment with no hypothesis, and an experiment with no hypothesis cannot tell you whether the idea worked, which means it cannot tell you whether to stop, continue, or scale.

The answer your organization needs before any AI pilot starts is simple: if this works, here is exactly what we will see, and here is exactly how we will know we are seeing it.

If you cannot write that sentence, the pilot is not ready.

This is the question that gets skipped most often. And it is not one question. It is three.

Is the underlying strategy or process ready to be measured or automated?

I worked with a nonprofit that wanted to build a dashboard connecting monthly ad spend to sessions scheduled. Completely doable. The data existed. The technical build was straightforward.

But the organization did not have a cohesive ad campaign that leadership had truly committed to. So partway through the build, they stepped back to reconsider their strategy. Again.

The dashboard was measuring chaos. The tool worked. The conditions for using it did not exist.

Automating a broken process does not fix the process. It scales the problem.

Does this pilot fit how the organization actually operates, not how you imagine it operates?

A vendor proposed a 40-agent AI workflow to automate appointment scheduling via social media apps using trained chatbots. The contract was signed without internal oversight. The technical and product organizations were told this was happening. The CEO said go.

The operations team pointed out that the organization runs on Calendly. The prototype was built around a different calendar tool entirely. The VP of Operations said no, we run on what we run on, and that is not going to change.

There was an existing portal handling the current workflow. The vendor was building a new portal integrated with their agentic AI tool. Then a salesperson pointed out that the organization was contractually obligated to display and sell certain things to end customers, which meant the new system could not legally replace the old one.

So now there were two portals, two calendar systems, a contractual wall nobody had read, a VP of Operations who had said no from the beginning, and a vendor who kept building because they were being paid to.

And the VP of Innovation was telling everyone they had to "show that something was getting built".

The calendar tool was not the real problem. That is not a technology failure. That is what happens when a pilot starts without governance.

Before any pilot starts, someone needs to map what actually happens operationally today, confirm the pilot fits that reality, and have the authority to stop it if it does not. Not the ideal state. The current state. And not after three months of building.

Is there a real, named, allocated owner for build, run, and support?

This is the one that lands on the existing team every time.

The 40-agent workflow goes live. Someone needs to manage what the chatbot is answering, handle the operational demands it creates, support the people it is supposed to serve, and monitor it when it does something unexpected.

That someone does not appear from nowhere. If they were not named and allocated before the pilot launched, they will exist after it launches, doing it on top of everything else, on a team already at 120 percent capacity.

Name the owner before you build. If you cannot, the pilot is not ready.

Every pilot needs one before it starts, not after things go sideways.

A clear timeline. A named decision point. A specific person who has the authority and the responsibility to look at the data and say: this worked, this did not, here is what happens next.

Without an off-ramp, pilots do not end. They become permanent fixtures that nobody formally evaluated, quietly consuming staff capacity and organizational attention while everyone waits for someone else to make the call.

Define the off-ramp before you build. What date does the pilot end? Who owns the decision? What data will inform it? What are the three possible outcomes, and what does each one mean?

If you cannot answer those questions, you are not running a pilot. You are making a commitment you have not admitted to yet.

Get the Free Toolkit