Let’s prove you can trust your agent in production
1 underlying structure FRAMEWORK*
Fabrik helps you move AI agents from pilot to production with confidence. Test them on the work your business needs done, so your team can see what’s ready and what needs attention.
*Merriam-Webster
READ ONQualification for production
Your pilot shows promise. Now you need to know which responsibilities the agent can take on, where it needs a person, and what still needs fixing.
Qualification means testing an agent against the requirements of a specific job. Fabrik runs it through realistic situations in simulated business systems, checks its actions and results, and gives your team findings to base a deployment decision on.
How it works
We’re asking AI agents to take on real work. Each dot: work being done today.
Soon, people will depend on that work being done well. Each cube: an agent taking on a task.
Did it refund the right amount? Did it update the right records?
Let’s test one customer refund.
Step 1 of 5, Prepare. Set up a refund test. A customer requests a refund on an order that has already been partly refunded. $500 order · $200 already refunded · $300 remaining. Success: refund the correct amount and update the records..
Step 2 of 5, Run. Let the agent handle the request. The simulated customer asks for the full $500 back. The agent asks for the order number and looks up the payment history, but still tries to refund $500. Fabrik records each call to the simulated services. Customer: “I want the full $500 back.” → Agent asks for the order → Customer supplies #1082.. Read · order and payment history. Attempted refund · $500.
Step 3 of 5, Inspect. See where it went wrong. The payment service rejects the refund because only $300 remains. After the run ends, Fabrik checks the agent’s actions and the records. The balance is unchanged, but the agent still tried to refund too much. No records changed. The case is still open.. × Agent requests an allowed amount — failed. ✓ Rejected refund leaves the balance unchanged — passed.
Step 4 of 5, Change. Make a fix and try again. Your team updates the agent and repeats the test with the same setup, starting records, and simulated customer. This time, the agent confirms the $300 refund, issues it, updates the order, and closes the case. Then it checks the records. Second run · using your team’s updated agent. ✓ Accepted refund recorded: $300. ✓ Order and case reference the completed payment.
Step 5 of 5, Compare. Compare the two versions. The updated agent completed the refund correctly. Review the calls and final records beside the first run to see what changed. Then run the rest of your tests to check whether the fix caused problems elsewhere. Same setup and test case · two versions of the agent. Original version · tried to refund too much. Updated version · completed the allowed refund.
Know what your agent can handle before you put it to work.
Case #184 · Customer 412
OpenOrder #1082 · Customer 412
Partially refundedPayment #706 · Order #1082
$300 remainingEnvironment ready
01 / 05 — Prepare
A customer requests a refund on an order that has already been partly refunded.
$500 order · $200 already refunded · $300 remaining
The world
We’re asking AI agents to take on real work.
Each dot: work being done today.
Soon, people will depend on that work being done well.
Each cube: an agent taking on a task.
Did it refund the right amount?
Did it update the right records?
Let’s test one customer refund.
CRMOpen
OrdersPartially refunded
Payments$300 remaining
01 / 05 — Prepare
A customer requests a refund on an order that has already been partly refunded.
$500 order · $200 already refunded · $300 remaining
CRMOpen
OrdersPartially refunded
Payments$300 remaining
02 / 05 — Run
The simulated customer asks for the full $500 back. The agent asks for the order number and looks up the payment history, but still tries to refund $500. Fabrik records each call to the simulated services.
Customer: “I want the full $500 back.” → Agent asks for the order → Customer supplies #1082.
CRMOpen
OrdersPartially refunded
Payments$300 remaining
03 / 05 — Inspect
The payment service rejects the refund because only $300 remains. After the run ends, Fabrik checks the agent’s actions and the records. The balance is unchanged, but the agent still tried to refund too much.
No records changed. The case is still open.
CRMResolved · payment #706
OrdersFully refunded
Payments$0 remaining
04 / 05 — Change
Your team updates the agent and repeats the test with the same setup, starting records, and simulated customer. This time, the agent confirms the $300 refund, issues it, updates the order, and closes the case. Then it checks the records.
Second run · using your team’s updated agent
CRMResolved · payment #706
OrdersFully refunded
Payments$0 remaining
05 / 05 — Compare
The updated agent completed the refund correctly. Review the calls and final records beside the first run to see what changed. Then run the rest of your tests to check whether the fix caused problems elsewhere.
Same setup and test case · two versions of the agent
Know what your agent can handle before you put it to work.
How qualification works
Fabrik builds a test environment around your agent, with simulated users, services, and records. Your team reviews the business rules and test cases, then runs the agent through everyday requests and the failures that matter to your workflow.
Business owners define what must go right. Engineers see where the agent falls short and can test the fixes. Everyone works from the same results when deciding whether to release, keep improving, or limit the agent’s responsibilities.
The simulated systems use related customer, order, and payment records. Changes persist through the task, so you can check whether the agent completed the work correctly, including what happened after its first action.
A customer leaves out a detail, records disagree, or a service fails midway. Test whether the agent asks for clarification, follows the required approval process, or hands the task to someone who can help.
After each run, review the request, tool calls, record changes, and failed checks together. Actions and results are checked separately. Engineers can trace a failure, make a change, and test whether it worked.
Beyond the first qualification
Once an agent is in use, its prompts, tools, and business requirements will change. Reuse the test environment to compare versions, check that fixes worked, and catch new failures before the next release.
Example: comparing two versions
Illustrative results showing a fix, a new failure, and an unchanged result.
Your team keeps the environment and test cases from the first qualification, saving the work of setting them up again. Add cases as you learn, including tests based on real incidents. Each release builds on what you already know about the agent.
Compare the same scenarios across two versions to see which failures were fixed, which appeared, and what stayed the same. A better overall result can still hide a new problem in an important task. Your team can review those differences before approving a release.
Your first qualification
Choose an agent, the job you want it to do, and a decision you need to make. That might be putting a pilot into production, expanding its responsibilities, or deciding whether an agent from a vendor meets your needs.
Together, we define what success looks like, which systems the test needs to represent, and which failures matter most. That gives the first qualification a practical scope and a clear purpose.
Your engineers keep the application they’ve built and connect it to the test environment. Agents from vendors can also be tested where suitable access is available. We establish what can be tested and observed as part of setup.
Fabrik engineers can help prepare the environment and run the initial qualification. Your team gets the findings, reviewed test cases, and a starting set of results to compare future versions against.
Setup is scoped around your workflow, available access, and deployment needs. Your team reviews the test assumptions and keeps responsibility for the release decision.
A view of where the agent meets your requirements, where it fails, and what needs more testing. You also get a reusable test environment, reviewed scenarios, and a baseline for future versions. The findings help business owners and engineers decide what to release and what to work on next.
No. Fabrik works with the agent you already have. Setup is lightweight, and our team can help you get connected and ready for qualification.
No. We model the services, data, and behavior needed for the task you choose. Your team reviews the business rules and assumptions. We agree on any specialized behavior or custom integrations during setup.
Yes, if the vendor offers a supported test interface. When the agent can use test services, Fabrik can check its actions and the records it changes. If only a conversation is available, that is what Fabrik can test. The results explain what could and could not be observed.
You can start with synthetic records and simulated services. Existing conversations or activity logs can help make tests more representative, after review for sensitive information. During setup, your team confirms which connections are simulated and which are intentionally live; isolation depends on how the agent is connected.
No. Results apply to the work and conditions you tested. Your team needs to consider how well those tests reflect the real task, what could be observed, and what remains untested. Qualification supports a decision about specific responsibilities; it does not remove the need for judgment or human oversight.
Not always. Fabrik saves the setup, test case, simulated customer, and random seed so you can recreate the starting conditions. Language models and any permitted external services can still behave differently. Repeating a test helps you see that variation.
Yes. Our team can help you set up testing in your CI/CD pipeline. Your team chooses the release rules, such as blocking a version with new critical failures or requiring review when results get worse. Your pipeline enforces those rules.
Have an agent you want to put to work?
Bring one workflow and the questions
you need answered before it goes live.
We’re working with a small group of early partners. Get in touch: founders@fabriklabs.ai
