Log in

Recipes for AI agents using the Chat Thing MCP server

This page is written for AI agents (Claude, ChatGPT, Cursor, Codex and others) connected to the Chat Thing MCP server, and for people who want to see what their agent will do. Each recipe lists the tools to call, in order, and the checks that prove the job worked. Tool names match the tools reference.

If you're a person, you don't need to call these tools yourself. Paste a recipe's goal into your assistant, for example "Set up a support bot from https://help.example.com", and it will follow the steps.

Rules that apply to every recipe

  • Check access first. Call list_teams. Only teams where canWrite is true (the Owner or Admin role) allow tools that change something. Read tools work in any team the user belongs to.
  • Read before you write. Call get_bot, list_power_ups or get_web_channel before changing them. update_power_up replaces the whole config, so write back every field you want to keep.
  • Fetch the schema before configuring. Power-ups, webhooks and the website widget follow list_*_types, then get_*_schema, then create or update. A rejected write lists the fields that failed plus the full schema, so fix those fields and retry once.
  • Don't retry creates blindly. create_power_up and create_hook aren't deduplicated. If a call may have succeeded, check list_power_ups or list_hooks before calling it again.
  • Test chats are real chats. A chat opened with start_chat appears in the team's Conversations, can fire the Conversation started webhook, runs the bot's power-ups for real (a handoff sends a real email) and spends the team's message tokens. Tell the user before testing a bot that has live power-ups or webhooks.
  • Only v2 bots can be changed. get_bot returns version. Legacy v1 bots are read-only over MCP; ask the user to migrate them in the dashboard.
  • Ask before destructive calls. delete_bot, delete_data_source, delete_power_up and delete_hook can't be undone. Get the user's explicit confirmation first.
  • Check the docs when unsure. search_docs and get_doc return these pages, so answer questions about plans, limits and settings from the docs rather than memory.

Set up a support bot from a website URL

Goal: a bot that answers from the user's help pages and cites them.

  1. list_teams, and pick a team where canWrite is true. Ask the user if there's more than one.
  2. create_bot with a name and a systemMessage that says what the bot is for, who it talks to, its tone, and what to do when the answer isn't in its knowledge. See Write your bot's instructions. Keep the returned bot id.
  3. discover_pages with mode: "sitemap" and the site's root url (or sitemapUrl if the sitemap isn't at /sitemap.xml). If there's no sitemap, use mode: "crawl" and poll list_discovered_pages until state is ready.
  4. Read the summary. Use list_discovered_pages with a filter to confirm the important sections (for example /help or /docs) are there, and to spot pages to leave out (login pages, tag archives, legal pages).
  5. add_data_source with type: "WEB", the discoveryId, and includePatterns or excludePatterns to narrow the pages. Patterns choose which pages; contentSelector and contentExcludes choose which part of each page is read.
  6. Poll get_data_source until state is synced or sync_error.
  7. update_bot with enhancedRetrieval: true and maxContextAmount of 8 to 10. If the content is many short entries (FAQ one-liners, product rows), also set documentRelevance to about 0.5. See Retrieval settings.
  8. update_bot with includeSources: true if the user wants answers to link to the pages they came from.
  9. Add an escape hatch with the next recipe.

Checks

  • list_data_source_rows shows the pages you expected, and none you excluded.
  • start_chat, then send_message with three questions whose answers you know are on the site. Each reply is correct and its sources array lists the right page URLs.
  • send_message with a question the site doesn't answer. The bot says it doesn't know and offers a next step, rather than guessing.
  • Give the user the dashboardUrl from create_bot so they can try it, and point them to Install the chat widget.

Add a human-handoff escape hatch

Goal: when the bot can't help, or a customer asks for a person, the team gets an email with the conversation.

  1. list_power_ups for the bot. If a talkToAHuman power-up already exists, update it instead of creating a second one.
  2. list_power_up_types and confirm talkToAHuman is offered.
  3. get_power_up_schema with type: "talkToAHuman".
  4. Ask the user which address should receive handoffs. Leaving it empty sends them to the bot owner.
  5. create_power_up with type: "talkToAHuman", a description that says when to hand off (for example "Use this when the customer asks for a person, or when you can't answer after two attempts. Ask for their email address first.") and config.notificationEmail.
  6. get_bot, then update_bot with a systemMessage that adds one line telling the bot to offer a person when it can't answer. Write the rest of the existing prompt back unchanged.
  7. Optional: suggest the user turns on human takeover in the dashboard (Standard plan and above) so a teammate can reply inside the website chat. There's no MCP tool for it.

Checks

  • list_power_ups shows the power-up with enabled: true.
  • Only if the user agrees (it sends a real email): start_chat, then send_message with "Can I talk to a person?". The bot asks for an email address, and after you give a test address it confirms the handoff.
  • For a bot that also needs a form or a CRM record, see Hand off to a human and Call your API.

Review the last week's conversations and suggest instruction changes

Goal: find where the bot struggled and propose specific changes, without changing anything until the user agrees.

  1. get_bot to read the current systemMessage and retrieval settings. list_data_sources to see what the bot knows and when each source last synced.
  2. list_chats with limit: 100. Results are newest first with createdAt and messageCount. Page with offset until createdAt is older than seven days. Skip chats you started yourself for testing.
  3. get_messages for each chat with more than one message. For long reviews, sample: every chat with 4 or more messages, plus a spread of short ones.
  4. For each assistant reply, note:
    • Replies that say the bot doesn't know, or apologise.
    • Replies with an empty sources array on a question that should be in the knowledge (only meaningful when includeSources is on).
    • Customers repeating or rephrasing the same question, or asking for a person.
    • Answers that are off-topic, too long or in the wrong language.
  5. Group the problems by cause, using the table in How your bot answers: missing knowledge, retrieval too strict or too loose, instructions, or model.
  6. Report to the user: each problem, two or three example chat IDs, the likely cause and the exact change you propose (new prompt wording, a page to add, a setting value).
  7. Only after the user approves: update_bot with the new systemMessage or settings, or update_data_source to add pages.

Checks

  • Re-ask two or three of the failing questions with start_chat and send_message, and compare the new replies with the old ones.
  • On the Enterprise plan, save the failing questions as test cases (see the last recipe) so the fix is checked every time the bot changes.

get_messages doesn't include customer feedback, topics or sentiment. Those are in the dashboard: see Review customer conversations and Analytics.

Keep a website knowledge source fresh

Goal: the bot picks up edited pages, new pages and removed pages.

Re-syncing a website source fetches every page already in it again. It doesn't look for new pages, and it doesn't remove pages that have gone from the site.

  1. list_data_sources, then get_data_source for the website source. Check state and lastSync.
  2. Schedule automatic re-syncs: update_data_source with syncInterval set to day, week or month (null turns it off). Automatic syncing needs the Standard plan or above; see Keep your bot's knowledge up to date.
  3. To find new pages: discover_pages with mode: "sitemap" for the same site. Then use list_discovered_pages and list_data_source_rows (both accept a filter) to work out which discovered URLs aren't in the source yet.
  4. To remove pages that no longer exist: list_data_source_rows, find rows whose state shows an error or whose URL has gone from the sitemap. Confirm the list with the user first.
  5. One update_data_source call with addUrls set to only the new URLs and removeRowIds set to the rows to remove. Only include a field when it has items: an empty addUrls or removeRowIds rejects the whole call. Don't commit the whole discovery again: existing URLs aren't skipped, so you'd add duplicates. Adding or removing rows starts a re-sync of the whole source, so don't call sync_data_source as well; poll get_data_source until state is synced or sync_error.
  6. If nothing needed adding or removing, sync_data_source refreshes every page now. Poll get_data_source the same way.
  7. Optional, for sites that publish often: list_hook_types, get_hook_schema for StartSync, then create_hook with the data source's id. Hooks are created turned off, so call toggle_hook_enabled with enabled: true before giving the user the trigger URL to add to their CMS publish webhook. The URL works like a password.

Checks

  • get_data_source shows a recent lastSync and state: synced.
  • send_message with a question only a new or edited page answers, and check that page appears in sources.
  • Syncing uses storage tokens for changed content only. If update_data_source or sync_data_source fails with a storage token error, tell the user; see Storage tokens.

Run a test run and read the results

Goal: a repeatable check of the bot's answers. Tests are on the Enterprise plan; on other plans every test tool returns an upgrade message, so stop and tell the user.

  1. list_test_cases for the bot to see what exists.
  2. create_test_case for each question worth protecting: a question plus a checks array.
    • Answer checks: factuality (the statement is the fact the answer must agree with), similarity (the statement is an example answer), requirements (the statement lists what the answer must do) and relevance. similarity, requirements and relevance take a threshold from 0 to 1; start at 0.7.
    • Power-up checks: powerUpCalled, powerUpNotCalled and powerUpSucceeded. The statement is the power-up's id or name.
  3. start_test_run with an optional name. It needs at least one test case and returns a testRunId.
  4. Poll get_test_run until state is complete or failed. If it stays running far longer than the number of cases suggests, report it as stuck instead of polling forever.
  5. Read each case: the question, the bot's response, response time, tokens, and each check's pass result, score and reason.

Checks

  • For each failed check, read the reason. If the answer is actually fine, the statement or threshold is too strict. If the answer is wrong, diagnose it like any bad answer: see Improve your bot's answers.
  • Runs use the bot's live settings at the time they run, so run once before a change for a baseline, then again after.
  • Test runs use message tokens for the bot's answers and for grading. See Test your bot's answers.

Next steps

Related

Last updated