Write a testable AI automation project brief

A useful opening might be: “Approved service enquiries from the central mailbox are matched to the correct customer, created as CRM tasks and presented with a reply draft for review.” That sentence identifies intake, matching, the destination and a human decision.

“We want to modernise communication with AI” describes an intention but not an acceptable deliverable. Suppliers might interpret it as a chatbot, writing tool, email classifier or new platform. Specify what work should actually happen differently after delivery.

An earlier Fraunhofer IAO methodology article emphasises concrete use cases, clear objectives and checking simpler process improvements. It dates from 2019; we use those planning principles rather than its historical market or technology assumptions. Fraunhofer IAO on AI pilot projects, reviewed again on 26 September 2026.

Process and volume. Outline the existing process. How many cases occur in an ordinary period, which variants are common, and where do questions, duplicate entry or late handoffs arise? Identify the people and systems involved.

Measured or estimated. Separate measured figures from estimates. “Last month we recorded 420 enquiries” differs from “probably 500 to 1,000.” Both can appear in an initial brief, but label them appropriately. Unknown data should not silently become a fixed pricing assumption.

Examples. Provide representative examples, including an ordinary input, an incomplete case and a difficult exception. Use approved anonymised or synthetic information. A supplier needs the structure of the problem, not necessarily complete customer records during an initial discussion.

Start and end. Describe the starting point and business completion. If work begins with an enquiry and ends with a reviewed CRM task, subsequent quote preparation is not automatically included. State such boundaries to avoid different expectations later.

Initial scope. An initial scope might cover one mailbox, one CRM account, two enquiry types and one language. Other mailboxes, historical migration and automated contractual commitments can remain outside it. This makes the first assignment assessable rather than signalling technical weakness.

Dependencies. List dependencies too. Internal API approval, an unresolved software tier or necessary data cleaning may affect the start. Give each prerequisite an owner and a decision for what happens if it is late. A schedule ignoring these dependencies is weak evidence of delivery readiness.

Four fields help for each significant requirement: identifier, desired behaviour, priority and acceptance evidence. Avoid saying the system should be intelligent and user-friendly without defining how anyone will assess that.

The following table shows six synthetic requirements for a service-enquiry workflow, each with an identifier and its acceptance evidence.

Use mandatory requirements deliberately. A preferred dashboard differs from preventing a message to the wrong recipient. If every visual preference becomes mandatory, suppliers and reviewers lose sight of the important boundaries.

Specify who can accept a deviation. The person reviewing a technical test may not have authority to change a business requirement. Record that decision path before delivery pressure makes informal compromises tempting.

Operations. List sources, destinations, object types and actual operations. “Connect to Microsoft 365” is too broad. Identify the approved mailbox, required reading operation and where a draft should be saved. A listed integration does not establish every desired capability.

Field meaning. Fields need meaning and authority. Which identifier defines a customer? Which source may change a phone number? Does a missing value mean unknown or delete? These differences determine correct behaviour and should not emerge accidentally during implementation.

Supplier assumptions. Ask suppliers to separate confirmed support, custom adaptations and unresolved prerequisites. A planned custom API connection differs from an existing standard capability. Either can be appropriate, but the assumption must remain visible in the offer.

Model task. Explain what the model does: summarise text, suggest values, select a category or prepare a response. Then describe separately which system action may follow. Producing a correct draft does not grant permission to send it.

Writing actions. Every writing action needs a destination, permitted parameters and an approval rule. Specify what the reviewer sees and what follows rejection. A period without a response must not silently become consent.

Business rules. Even with an agent, explicit business rules should be checked outside free-form output. Customer identity, valid status transitions and duplicate prevention should not exist only as wishes in a prompt. Ask where the supplier will implement the relevant technical and organisational controls.

Expected results. Define expected results before testing. Include ordinary cases and deliberately difficult variants. Perfect inputs alone can support a convincing demonstration without representing everyday operation.

Error types. Distinguish error types. Awkward wording differs from incorrect customer matching. Assess important fields and actions separately. A general percentage labelled accuracy means little unless the counted outcome and treatment of serious errors are defined.

Repeated checks. Repeated checks may be useful for generated content because outputs can vary. Record the model, configuration and examples used. One correct response does not establish broad reliability. Also require uncertainty to remain visible instead of forcing an apparently complete answer.

Framework. The voluntary NIST AI Risk Management Framework connects context, measurement and ongoing risk handling. It offers methodological guidance rather than automatic acceptance or a legal compliance guarantee. NIST AI RMF.

The following table lists the five tests of this synthetic acceptance sequence, each with its input or change and the expected result.

These five cases assess different properties and should not disappear inside one overall demo result.

Record observation, deviation and decision for every case. If a difference is accepted, explain its consequence. An unresolved issue should be corrected, intentionally removed from scope or accepted as a documented limitation, rather than hidden under a broad completed status.

Keep enough evidence for later retesting. A screenshot alone may not show which input or configuration produced the result. Store the approved test case, relevant settings and observed destination state using the project’s agreed access and retention rules.

Operation. Identify who handles alerts, renews credentials, reviews changes and catches up unfinished cases after launch. Distinguish platform availability from successful business processing. A reachable server does not prove that every enquiry arrived in the CRM.

Handover. Handover should cover configuration, field rules, known limits, test cases and responsibilities. Agree a short recovery procedure and manual fallback. A second person should be able to understand a typical failure using those records.

Changes. Changes need an orderly route: description, effort estimate, impact review and approval. A new field or additional mailbox may alter the original assignment. Keep the current agreed version of the brief.

Ask every supplier to respond under the same headings:

• process understanding
• confirmed scope
• assumptions
• technical approach
• internal participation
• acceptance
• operation
• handover
• costs

Separate one-off, recurring and usage-based charges.

Ask what remains your responsibility. An apparently complete offer may assume substantial data preparation or daily exception handling. Include that work in comparison even when it does not appear on the supplier’s invoice.

Assess mandatory requirements first, then implementation quality, operation and costs. A good presentation cannot replace a missing core capability. A supplier should also be able to discuss limitations without automatically losing to an unqualified promise.

You do not need a lengthy catalogue to start. Describe one concrete case, collect representative examples and identify the person who will accept the business outcome. Build the project brief from that foundation so proposals become comparable and the eventual decision remains understandable.

Finally, agree how unmet acceptance criteria will be handled. Define the correction cycle, which cases will be tested again and who may approve reduced scope. Reaching the planned date does not, by itself, make a technical demonstration successful. Record remaining limitations in a form the eventual operating team can actually use.

The following pages and articles cover related decisions about AI automation projects.

• AI automation: tasks, selection and implementation
• AI automation costs: a first-year budget for SMEs
• Introducing AI: software, an internal team or an agency?
• AI workflow approvals: who checks which action?

For a workflow review, start with the task, systems involved and typical exceptions. Discuss it with modernagents.