---
title: "After launch: a routine for improving your bot"
description: "A practical routine for the weeks after your bot goes live - find bad answers in your conversations, fix the cause, check the fix and track what customers ask."
canonical_url: "https://chatthing.ai/docs/improve/after-launch"
last_updated: "2026-09-25"
---

# After launch: a routine for improving your bot

Your bot will never be as good on launch day as it is a month later. Real customers ask things you didn't expect, word questions differently from your help pages, and find the gaps in your content. A short, regular review turns those conversations into better answers. This guide gives you a routine for the first few weeks, then a lighter weekly one.

## What you'll use, by plan

| Tool | What it's for in this routine | Plans |
| --- | --- | --- |
| [Conversations](https://chatthing.ai/docs/manage/conversations) | Reading chats, with search and filters for bot, date, state, unread and feedback | All plans |
| Thumbs up and down | Letting customers flag bad answers | All plans |
| [Otto](https://chatthing.ai/docs/get-started/otto) | Diagnosing a bad answer and applying the fix | All plans (owners and admins) |
| AI conversation **Summary** | Skimming a chat without reading it all | Standard, Pro, Enterprise |
| [Auto-syncing](https://chatthing.ai/docs/knowledge/keep-it-up-to-date#sync-on-a-schedule) | Keeping knowledge fresh without thinking about it | Standard, Pro, Enterprise |
| [Analytics](https://chatthing.ai/docs/manage/analytics) | Topics, sentiment, common questions and recommendations | Pro, Enterprise |
| [Test cases](https://chatthing.ai/docs/bot-settings/test-your-bot) | Re-checking answers you've fixed, automatically | Enterprise |

On the Free and Basic plans the routine still works: you just read conversations yourself instead of relying on summaries and analytics.

## Before you go live

Two settings make the rest of the routine much easier. Both are in your bot's **General settings** tab.

- Turn on **Allow users to provide feedback on responses**. Customers then see thumbs up and down buttons on each answer, and you can filter for thumbs down.
- Turn on **Include sources for responses** while you're reviewing. Seeing which pages an answer came from is the quickest way to tell a knowledge gap from a retrieval problem. You can turn it off again later if you don't want customers to see links.

If you haven't already, work through [Check your bot is ready to launch](https://chatthing.ai/docs/get-started/launch-checklist).

## 1. Find the conversations worth reading

You don't need to read every chat. Open **Conversations**, choose your bot, and use **Filters** to go straight to the ones most likely to show a problem:

- **Feedback** set to thumbs down: customers telling you an answer was wrong.
- **State** set to **Escalated**: the bot handed off, which often means it couldn't help.
- **Sentiment** set to **Negative**, or **Emotions** such as **Frustrated** or **Confused** (Pro and Enterprise).
- **Search** for phrases your bot uses when it's stuck, such as "sorry" or "don't know", and for words like "human", "person" or "agent".

In the first week, look daily. After that, a weekly look is usually enough. The **Summary** in each conversation's side panel (Standard plan and above) tells you what a chat was about without reading it all; summaries appear about 10 minutes after the last message.

## 2. Work out why each answer went wrong

For each bad answer, find the cause before changing anything. Otherwise you risk fixing the wrong thing, for example rewriting your instructions when the page simply never synced.

- **Ask Otto.** Open the conversation and click **Ask Otto about this chat**. Otto reads the exchange and the sources the bot used, explains the cause and proposes a fix. See [Diagnose a bad answer](https://chatthing.ai/docs/get-started/otto#diagnose-a-bad-answer).
- **Or check it yourself** with the symptom table in [How your bot answers](https://chatthing.ai/docs/improve/how-your-bot-answers#match-the-symptom-to-the-fix). If the answer listed no sources, the bot didn't find anything relevant; if it listed the wrong ones, the search matched the wrong content.

Keep a simple list: the question, what went wrong and the cause. Patterns show up quickly, and three chats with the same cause need one fix, not three.

## 3. Fix the cause

Most fixes fall into one of these:

- **Missing knowledge.** Add the page to your website data source, or write the answer as a [manual entry](https://chatthing.ai/docs/knowledge/manual) in your customers' own words. Manual entries are quick to add and easy to find later.
- **Out-of-date knowledge.** Re-sync the data source (**Sync** on its card, or **Synchronise** inside it), and turn on **Auto-syncing** if your content changes often. New pages on a website aren't found by a re-sync: add them. See [Keep your bot's knowledge up to date](https://chatthing.ai/docs/knowledge/keep-it-up-to-date).
- **Instructions.** Add a rule to the **Prompt**, such as what to do when the answer isn't in the knowledge, how long answers should be, or which topics are off-limits. Change one thing at a time. See [Write your bot's instructions](https://chatthing.ai/docs/bot-settings/instructions).
- **Retrieval settings.** Lower **Document relevance** or raise **Max context documents** if the bot misses content that's there; turn on **Enhanced knowledge retrieval** if it loses track of follow-up questions. See [Retrieval settings](https://chatthing.ai/docs/knowledge/retrieval-settings).
- **No way out.** If customers get stuck, add [Talk to a human](https://chatthing.ai/docs/power-ups/talk-to-a-human) or a contact form, so the bot can pass them on.

For the full list of fixes by problem, see [Improve your bot's answers](https://chatthing.ai/docs/improve/improve-answers).

## 4. Check the fix worked

- Click **Test** on your bot's page and ask the original question, plus two or three rewordings. Chats you start here use message tokens like any other.
- On the Enterprise plan, save each fixed question as a [test case](https://chatthing.ai/docs/bot-settings/test-your-bot) with a check on the answer. Run your tests after every change so an old problem can't quietly come back.

## 5. Look at the bigger picture

Once you have a few weeks of conversations, step back from individual chats. On the Pro and Enterprise plans, your bot's **Overview** tab shows:

- **Common Questions**: the questions customers ask most. Ask each one yourself and check the answer. A frequent question with a weak answer is your most valuable fix.
- **Topic Analysis**: what people ask about, and how that changes week by week. A growing topic may need more content.
- **Sentiment Analysis**: whether customers are leaving happier or more frustrated over time.
- **Actionable Recommendations**: suggestions for your knowledge and instructions, based on patterns in your conversations.

Team members also get a weekly analytics email summarising each active bot. See [Analytics](https://chatthing.ai/docs/manage/analytics).

On other plans, get a similar picture by downloading your conversations (**Download all** on the **Conversations** page gives a CSV of everything matching your filters) and skimming the questions.

## Your weekly checklist

Once the first busy weeks are over, this takes 20 to 30 minutes a week:

1. In **Conversations**, filter for thumbs down, **Escalated** and (on Pro and Enterprise) negative sentiment since last week. Read each one.
2. Diagnose each bad answer with **Ask Otto about this chat** or the symptom table.
3. Fix the causes: add or update knowledge, adjust the **Prompt** or a retrieval setting.
4. Re-ask the questions you fixed with **Test**, or run your test cases on Enterprise.
5. Check your data sources synced recently and that no sync failed. Add any new pages from your site.
6. On Pro and Enterprise, scan **Common Questions** and **Actionable Recommendations** on the **Overview** tab.
7. Check your usage on the **Overview** tab so you don't run out of [message tokens](https://chatthing.ai/docs/account/message-tokens) before the month ends.

Your bot doesn't learn from conversations on its own. Every improvement comes from a change you make to its knowledge, instructions or settings, which is why this routine matters.

## Next steps

- [How your bot answers](https://chatthing.ai/docs/improve/how-your-bot-answers): what goes into each reply, and the symptom table.
- [Improve your bot's answers](https://chatthing.ai/docs/improve/improve-answers): step-by-step fixes by problem.
- [Recipes for AI agents](https://chatthing.ai/docs/mcp/agent-recipes#review-the-last-weeks-conversations-and-suggest-instruction-changes): have your AI assistant run the weekly review for you.

## Related

- [Review customer conversations](https://chatthing.ai/docs/manage/conversations): Find, filter and read every conversation your bots have with customers, see AI summaries and user details, and share or download transcripts.
- [See what customers ask with analytics](https://chatthing.ai/docs/manage/analytics): Use your bot's Overview tab to see usage, conversation topics, customer sentiment, engagement, the most common questions and AI recommendations.
- [Improve your bot's answers](https://chatthing.ai/docs/improve/improve-answers): Fix a bot that says "I don't know", gives wrong or made-up answers, goes off-topic, rambles, forgets context or replies in the wrong language.