DEV Community: Kevin Li

How to Put Guardrails Around an AI Workflow

Kevin Li — Sun, 11 Oct 2026 16:17:52 +0000

An AI workflow becomes useful when it can operate inside a boundary the business understands. The difficult part is rarely the first prompt. It is deciding what the system may read, propose, change, and stop.

When I review an automation for a small team, I look for six guardrails.

1. Define the input and output contract

Write down what starts a case and what a completed case looks like. “Read incoming requests and help the team” is too broad. “When a request arrives in the shared inbox, extract the customer, product, requested date, and missing information, then prepare a draft for review” is a boundary.

The contract should also name the source of truth. If the workflow can read three spreadsheets, an inbox, and a CRM, someone needs to decide which record wins when they disagree.

2. Separate AI judgment from deterministic checks

AI can classify, summarize, extract, match, or draft. It should not quietly replace rules that need to be exact.

Keep permissions, required fields, calculations, duplicate checks, date rules, and routing conditions deterministic wherever possible. A model can suggest that two records look similar; a controlled rule should decide whether the system is allowed to update one.

This separation makes failures easier to diagnose. If the output is wrong, the team can ask whether the issue came from interpretation, source data, an integration, or a business rule.

3. Give the workflow the least authority it needs

A workflow that drafts a response does not need permission to send one. A workflow that proposes an inventory change may not need permission to write directly to the inventory system.

Start with read access and recommendations. Add write or send authority only after the team has evidence that the narrower boundary works. For customer commitments, financial actions, access changes, or sensitive records, keep an explicit approval state.

4. Make exceptions visible

A review step is not a disclaimer. It is an operating state.

The reviewer should see the source evidence, the proposed result, the reason for uncertainty, missing or conflicting information, and the actions they are allowed to take. “Please check this” is not enough if the person has to reconstruct the case from several systems.

Define what happens when the reviewer rejects the result, when required information is missing, and when the system cannot reach a dependency. Every exception needs an owner and a next state.

5. Log enough to replay the decision

A useful log records the input reference, relevant source versions, workflow version, proposed output, validation results, human decision, final action, and downstream outcome.

This is not only for incident response. It lets the team compare the proposed path with the old path and see where time, corrections, and rework actually moved. Without that history, a workflow can feel faster while simply shifting effort to review.

6. Measure the completed business result

Model accuracy is only one signal. Measure the whole path:

elapsed time from trigger to completed result;
employee touch time and review time;
accept, edit, reject, and escalation rates;
missing-information and exception rates;
duplicate work and downstream rework;
system failures, retries, and recovery;
cost per completed case; and
customer or operational outcome.

A workflow should advance because the evidence supports a defined boundary, not because the demo looked impressive.

My background is in AI engineering at a global technology company. I helped a nine-person engineering team automate parts of software development and increase measured development velocity by approximately 100%. That was a prior engineering environment, not a promise of client savings.

For a fuller implementation sequence, I keep a field guide on moving an AI pilot into controlled production. The same principle applies at every stage: define the boundary, expose the failure modes, keep a safe fallback, and expand one dimension at a time.

Why AI Workflow Projects Fail at the System Handoff

Kevin Li — Thu, 08 Oct 2026 04:59:54 +0000

The model call is rarely the hardest part of an AI workflow. The failure usually appears one step later: when a draft, classification, or extracted field has to cross a boundary into another system.

A useful way to think about the design is to treat every handoff as a contract.

1. Define the handoff before choosing the model

For each step, write down:

the trigger
the context the step is allowed to see
the exact output schema
the checks that must pass
the system that owns the next action
the person who handles exceptions

For example, an invoice workflow might accept an uploaded document, extract a small set of fields, validate the vendor and amount, route uncertain cases to a queue, and only then create a draft record in the accounting system.

The important detail is that the model is not the owner of the entire process. It performs a bounded interpretation step.

2. Keep deterministic checks outside the model

Use code or system rules for things that should not be persuasive:

required fields
allowed status transitions
duplicate detection
amount thresholds
date formats
permission checks
whether an approval is required

This makes the workflow easier to test and easier to explain when something goes wrong.

3. Make the exception path visible

A workflow is not production-ready if it only describes the happy path. Define what happens when:

the source document is incomplete
two systems disagree
confidence is low
a customer asks for an exception
the requested action is irreversible

In those cases, the correct output may simply be a review item with the source context attached. That is a useful result, not a failure.

4. Measure rework, not just model accuracy

A high extraction score can still produce a bad workflow if people spend time correcting records or checking every result manually.

I would track:

percentage of items completed without correction
exception rate
average review time
duplicate or rejected actions
time from trigger to completed handoff
percentage of work that still requires re-entry

These measures tell you whether the workflow is reducing operational friction.

5. Start with one narrow boundary

The first pilot should usually connect one trigger to one structured outcome. For example: receipt received → fields extracted → review queue → reimbursement draft.

I wrote a practical example of this kind of workflow, including where validation and human review belong: https://kelenai.com/insights/expense-report-automation/

The broader lesson is simple: the most valuable AI workflow is often not the one with the most impressive model. It is the one whose handoffs are explicit, reversible, and easy for a person to supervise.

Human Review Is a Workflow State, Not a Disclaimer

Kevin Li — Sun, 04 Oct 2026 18:51:09 +0000

“A human will review it” is a useful principle, but it is not a workflow design.

When a small business adds AI to a sales, document, or customer-service process, the risky part is usually not the model call. It is the handoff around the model: what enters the system, what evidence is available, who can approve the next action, and what happens when the output is uncertain.

I think of human review as a workflow state.

Start with a business event

Do not begin with “where can we add an agent?” Begin with an event the team already recognizes:

a quote has received no response;
an invoice contains missing or contradictory fields;
a customer request needs a policy decision;
an order document does not match the expected format;
an email attachment needs to be entered into an existing system.

This gives the automation a bounded starting point and a real owner. It also gives you something measurable before any model is selected.

Keep the model inside a bounded step

A practical workflow often looks like this:

A known event triggers the workflow.
The system collects the relevant context.
AI drafts an interpretation or proposed action.
Deterministic checks validate required fields and business rules.
A human reviews the cases that meet an escalation condition.
The approved action is written back to the system.
The result and any exception are logged.

The model should not silently own the whole process. It should perform the part where flexible language understanding is useful. Systems, rules, and people should retain authority over the parts that need consistency, accountability, or reversibility.

Make the review state explicit

A review queue should answer five questions without forcing the reviewer to reconstruct the case:

What did the system receive?
What did the model infer?
Which rule or threshold caused escalation?
What can the reviewer change?
Where does the decision go next?

“Please check this” is not enough. A useful review state might show the original document, extracted fields, validation warnings, confidence signals, and the proposed next action. The reviewer should be able to approve, edit, reject, or route the item for more information.

For consequential actions, approval should happen before the write-back or outbound message—not after it.

Measure the workflow, not the demo

A demo can prove that an API call works. It cannot prove that a process improved.

For a first pilot, I would track:

time from trigger to completed action;
percentage of items reaching a human review queue;
correction and rejection rate;
follow-up completion rate;
exception reasons;
work that still happens outside the workflow.

These measures reveal whether the system removes work or merely moves it to a different screen. They also make it easier to decide whether AI is actually the right mechanism. Sometimes a deterministic rule, a better form, or an existing platform feature is the better solution.

What I would ship first

I would start with one workflow and one owner. For example, when a quote goes quiet, the system can assemble the existing customer and quote context, draft a follow-up, require approval, send it, and record the outcome.

That is a better first production boundary than a general-purpose “AI assistant” that can touch every system but has no clear definition of success.

The pattern is simple:

trigger → context → draft → validation → human approval → action → outcome

I wrote a longer field guide on designing human review as an explicit workflow state here: https://kelenai.com/insights/human-review-ai-workflows/

I’m building KelenAI around this kind of practical workflow implementation for U.S. small businesses. The goal is not to add AI everywhere; it is to make one important process more observable, controlled, and useful.