Launch and operate a chatbot with clear release criteria

Scope. For each request type, state what the customer supplies and what the bot may deliver. “Automate customer service” is too broad. “Answer a general delivery-time question from the approved market policy” can be assessed. An individual delivery enquiry additionally requires current order data and authorisation.

Ending. Specify the ending: answered question, confirmed action, accepted handover or unresolved case with an explicit alternative. A closed chat is not sufficient evidence of completion. Identify decisions that remain with staff.

Baseline. Record the existing process before release. How many eligible requests arrive, how long does handling take and which exceptions create work? Without a baseline, the team may struggle to tell whether the bot saves effort or simply spreads contacts across several channels.

Five responsibilities need an owner before launch: service lead, knowledge owner, system operator, privacy and security owner, and quality reviewer; the table gives the operational task and the coverage question for each.

One person may hold several roles, but responsibilities and substitutes still need to be explicit. A monthly agency invoice does not establish who notices a failed callback process on Friday evening and corrects customer communication.

Published system. Test the actual customer-facing system, rather than relying solely on an editor preview. For example, Microsoft documents how Copilot Studio publishing affects connected channels, and updated content may not immediately reach an existing session. Source checked on 26 September 2026.

Test record. Record version, knowledge revision, channel and test time. Open a new customer session and separately inspect an existing conversation. Correct text in an editor is not evidence that every customer receives it.

Channels. Include channel presentation: a button, attachment or selection may behave differently across channels. Website, WhatsApp and internal applications each need a suitable complete test, even if they share a knowledge collection.

Prepare release cases for the chatbot that cover these situations:

• normal requests
• missing information
• incorrect identity
• an explicit request for a person
• an unavailable destination system

For each, record expected behaviour, observed result and accountable approval. Use suitable test records and document real findings rather than simply checking boxes.

Blocking criteria. Set blocking criteria before reviewing outcomes. Another customer's data, invented payment confirmations or an unavailable required service route cannot be offset by many correct standard replies. Less serious findings need an owner and a reviewable remediation date.

Release scope. A meaningful release states which tasks are permitted in which channels. New write operations and new data sources are separate changes with their own effects to assess. A stable bot name does not mean that its capability scope has stayed unchanged.

Incident. A fictional bot answers general product questions and creates callback requests. In the afternoon, its ticket connection fails. Information answers continue to work, but callback requests can no longer be confirmed.

Response. The planned response disables the affected action specifically. The chat explains the available alternative while an operator investigates. Requests with uncertain delivery are reconciled against the helpdesk using their existing references. They are not all resubmitted because some may already have arrived.

Recovery. After recovery, the team tests a normal callback, a repeated request and a lost acknowledgement. The action resumes only after confirmed results. Unresolved customer cases receive the agreed follow-up. A green connection indicator does not complete the business reconciliation.

Events and review. Changes to prices, opening hours, products and ownership should actively reach the knowledge owner. Supplement these events with regular review of failed or uncertain conversations. A monthly meeting should not leave information that became invalid today in circulation for several weeks.

Causes. Distinguish missing knowledge, retrieval errors, system failures and process gaps. A new document can help with missing information, but cannot repair an expired integration permission. Associate every improvement with its cause and a repeatable acceptance case.

Change record. Keep a record of what changed and why. If results deteriorate, that record helps distinguish a content issue from a model, channel or integration change. Avoid changing several unrelated components during the same investigation unless the dependency requires it.

Measures and alerts. Track completed requests, genuinely accepted transfers, repeat contacts and remaining staff time. Add technical signals such as failed calls and unusual spend. A useful alert identifies an action and an owner, rather than accumulating in an unattended mailbox.

Low volumes. For low volumes, regular case review may be more informative than an apparently precise percentage. Record changes in request mix, such as a delivery incident or seasonal questions. Otherwise a difficult month may be misread as a model deterioration.

Usage limits. Check what happens when an allowance or usage limit is reached. The team should know whether automation stops, charges increase or cases move to staff, and whether that outcome is acceptable during a busy period.

The operating handover for a chatbot should include these items:

• the source register
• access ownership
• approvals
• unresolved findings
• incident procedures
• supported contact routes

Verify that a substitute can reconstruct a case and disable an affected capability using those materials.

Introduction is complete when the team can operate the agreed service, identify faults and approve changes deliberately. Further automation can then be justified by actual cases. It needs the same clarity about task, authority and result as the first approved workflow.

These pages cover the service overview and the related decisions for launching and operating a chatbot:

• AI chatbots for customer service
• Customer-service AI chatbots: plan the human handover
• Maintain customer-chatbot prices, products and policies
• Measure resolved chatbot requests and remaining work

For a workflow review, start with the task, systems involved and typical exceptions. Discuss it with modernagents.