What is the difference between a prototype, a pilot and production?
A prototype explores whether an approach can produce a useful output in a limited setting. A pilot evaluates a defined use case under agreed controls with a defined group. A production workflow has ongoing ownership, monitoring, support and appropriate review for its real operating context.
These are working distinctions for planning. Your organization may use different gates or terminology. The critical point is to make the transition explicit instead of treating a successful workshop demonstration as permission to deploy.
Start with intended use and the human decision.
Write down who will use the output and what they will do with it. A draft internal summary and a recommendation that influences a borrower outcome create different evaluation questions.
Name the person who reviews the output, what evidence they should check and how exceptions are handled. A statement that a person is “in the loop” is incomplete until the review task is concrete and realistic.
In a hypothetical document-triage pilot, the output might be a list of apparently missing items. The reviewer would compare those flags with the underlying package and correct them before requesting information. The pilot would measure both missed items and unnecessary requests.
Build an evaluation you can learn from.
Collect representative test cases that the relevant owners have approved for this use. Include incomplete inputs, conflicting information and cases the system should escalate. Agree on what counts as a satisfactory answer before comparing results.
- Quality: Is the output accurate, complete and traceable to the input?
- Operating effort: How much time goes into preparation, review and correction?
- Exceptions: Does the workflow recognize uncertainty and route it appropriately?
- Change: How will a new model, prompt, policy or data source be re-evaluated?
Report failures alongside successful examples. Averages can hide an exception that changes the decision about whether to proceed.
Involve the people who own the operating constraints.
Discuss the proposed use with the relevant technology, security, privacy, risk and business owners. Clarify where data goes, who can access it, what the provider retains, and how the workflow connects to existing systems. This is a set of planning questions, not a substitute for your organization’s review process.
For Fannie Mae seller/servicers, the April 8, 2026 lender letter provides an AI/ML governance framework for origination and servicing practices. Review the current source with the people responsible for its applicability in your organization.
Use a 30/60/90 plan as a decision sequence.
- First 30 days: define the use case, ownership, baseline and evaluation questions.
- By 60 days: complete an agreed evaluation and review the findings with the relevant stakeholders.
- By 90 days: decide whether the evidence supports a controlled pilot, a revised approach or stopping the work.
This is an illustrative sequence, not a deployment deadline. The pace depends on your workflow and review requirements. The value of the plan is that someone owns the next decision and knows what evidence it needs.
The AI Lending Lab uses hands-on learning to help executives see these questions sooner. Explore what to look for in executive AI education or read about the Orlando workshop on November 10, 2026.
Sources and scope
Examples are illustrative. This guide is educational and does not replace the review required for a specific lending use case.