The Rulebook

Definition. A Rulebook is the maintained set of instructions, limits and responsibilities for a particular AI process.

A policy document may say that a system should be safe. A working Rulebook says which decisions it may make, which actions are out of bounds, when it must ask a person and who can change those conditions. It should be short enough for the people running the process to use and specific enough to test.

Three kinds of rule

Red lines are actions the system must not take. For example, an agent that drafts customer replies may be forbidden from sending one without approval. Where the system can execute an action, enforce the prohibition in the workflow or application as well as in its instructions. Test the control before release and after changes that could affect it. An instruction in a prompt is not a reliable access control.

Guardrails define the conditions under which routine work can proceed: which requests are in scope, which data is required, what counts as an exception and where an exception goes. An internal support agent might route tickets for known categories but send unclear requests to a person. The operating limits should match the autonomy level assigned to that decision type.

Recommendations help people and the system work consistently without pretending every preference is a hard rule. They may cover response style, how to record a correction or how to investigate a recurring error. They can change as the team learns from use.

Not every recommendation needs an automated test. For a red line or an important guardrail, choose the strongest check available: an execution control, a test case, monitoring, human approval or a combination. Record how a failure is detected and who responds.

Build it from real work

Start with one process and its owner. Walk through recent routine decisions and difficult cases. For each rule, write down:

• The decision or action it applies to, and any exceptions.
• Its owner and the source of authority for it.
• How the system or a person checks it.
• What happens if the check fails or the required information is missing.
• What change would prompt a review.

Test cases where two rules could lead to different answers. If two rules lead to different answers, the process owner must set a priority or an escalation path. Do not expect the model to resolve a business conflict by guessing.

The Rulebook should live alongside the working process, with a version history and a named person responsible for keeping it current. If a separate policy or audit record is required, reconcile it with the operating version. A signed document that describes old behaviour gives a false picture of what the system does now.

Review after change

A new model, a different data source, a changed approval workflow or a new type of request can invalidate an earlier rule or test. Check the affected boundaries before broadening use. A Decision Log can surface disagreements, errors and exceptions for review; it does not change the rules automatically.

When a rule fails, contain the affected action, investigate the cause and decide whether the fix belongs in the instructions, a system control, the data or the human workflow. Then add a case that would catch the same problem next time. The owner decides when the process can resume or gain more authority.

The Rulebook is useful when it changes how the process runs. If nobody can point to a decision it governs, a check it triggers or a person who updates it, the team has a document rather than a control.