Testing an AI phone assistant in Austria: acceptance guide

Write down what the assistant is allowed to complete. “Reduce telephone workload” is too vague. A testable scope might be: answer opening-hours questions from an approved source, record callbacks with a confirmed number, and transfer certain requests to a staffed group.

For appointments, distinguish intake, proposing a slot and creating a confirmed booking. If only callback capture has been configured, a test should not accidentally expect a completed reservation. Conversely, a purchased booking integration should not pass merely because it sends an email.

Record prohibited commitments as well, such as bespoke prices, unverified delivery dates or professional advice outside the approved knowledge. Testers need the same rules as the person configuring the assistant. Otherwise, a disagreement about scope can look like a product defect.

Number and provider. Identify the number customers dial and the provider that routes it to the assistant. Obtaining a new destination number, forwarding an existing number and porting to another provider are different changes. Setting up the AI product does not automatically change the existing telephone contract.

Numbering rules. Austria's RTR explains geographic numbers and local numbering areas. Confirm with the telephone provider how number provision and use apply to your location. Do not assume instructions written for a foreign number apply unchanged to an Austrian service.

Routing tests. Test routing during office hours, when the line is busy and after closing separately. Record possible additional connection charges and how forwarding can be reversed. The person with administrative access to the phone system should be available during the change.

Use a small group of speakers reasonably reflecting the area you serve. Ask some to use regional speech and others standard German. Include background noise and corrections made during the conversation. For a business with international callers, include a language switch.

Score important information individually: name, street, building number, date, time and callback number. A pleasant reply must not compensate for an incorrect address. Preserve the expected values in the test record so that assessment does not rely on a tester's memory.

When an expression is unclear, the assistant should ask a specific clarification or offer a suitable alternative. Test that reaction as carefully as the normal success path. A system that makes uncertainty visible can be more operationally useful than one that always continues fluently.

The table lists twelve acceptance cases for an AI phone assistant, each with the expected behaviour and the evidence to look for in the working system.

This is a starting set. Add the cases specific to your industry instead of treating a generic checklist as a complete approval. Sensitive or professionally consequential processes need expectations defined by the responsible qualified people.

Call. The caller says: “I need an appointment next Thursday, preferably around three. Please call me back on the number I am about to give you.” During intake, the caller corrects one digit and then changes Thursday to Friday.

Run 1: normal completion. A good flow confirms the actual calendar date, preferred time and corrected callback number. If only appointment intake is authorised, it records that request. If direct booking is authorised, it checks availability and makes a definite commitment only after the destination system confirms storage.

Run 2: calendar failure. For a second run, deliberately disable the calendar connection. The expected result is a clearly labelled request for human follow-up, not an optimistic promise. The two runs test the same customer need under different technical conditions, making the failure behaviour directly comparable.

Run 3: interrupted submission. A third useful variation is to end the call immediately after the booking request is submitted. Inspect whether the booking was created, whether a confirmation is pending and whether staff can reconcile the result. The assistant should not need the caller to remain connected for a backend action to be traceable.

Record fields. For each case, record its identifier, configuration version, time, call route, expected action, observed action and deviation. Add the identifier of any created work item. Synthetic records should be clearly recognisable as test data and removable afterwards.

Error severity. Classify errors by their consequence. Awkward wording is different from a false appointment confirmation or a message disclosed to the wrong person. Severe failures must remain visible even when most conversations appear satisfactory.

Acceptance criteria. Agree acceptance criteria before the demonstration. An organisation might require every privacy-critical case to follow its approved path and prohibit any binding transaction from being declared complete without a confirmed system result. These are proposed acceptance rules, not a general guarantee of future performance.

No statistical claims. Avoid claiming statistical certainty from a small set. Twenty passed calls show that twenty particular scenarios passed under recorded conditions. They are useful evidence for a bounded release, but not proof of a universal success percentage across future traffic.

The greeting should make it understandable that an AI assistant is answering. The European Commission explains the transparency obligations applying to certain AI systems from August 2026. Also check which information is collected, whether recording occurs and how required privacy information is supplied.

A transcript, audio recording and callback task are separate data objects. The GDPR must be assessed against the actual processing; a general supplier label is insufficient. Use synthetic information for acceptance testing wherever possible and avoid real health or customer data.

Document the permitted tasks, unresolved limits, responsible people and route back to the previous telephone arrangement. Identify who checks the first production outcomes and how problems are reported. Repeat the relevant acceptance set when routing, knowledge or model configuration changes.

The method remains useful in Germany and Switzerland, but numbering, language, contracts and legal assessment need local adaptation. An Austrian acceptance test establishes evidence for the configuration actually tested. Keeping that boundary explicit makes the record more useful for decisions and later troubleshooting.

These pages cover the service overview and the related decisions around testing an AI phone assistant:

• AI phone assistants: functions and boundaries
• Configure an AI phone assistant: rules and knowledge
• Human handover and callbacks for AI phone assistants
• Measure AI phone assistant quality in operation

Discuss it with modernagents.