You've built your agent, connected your data... but the responses are just okay.
Maybe it's too verbose. Maybe it sounds like a robot reading Wikipedia. Maybe it keeps saying things you wish it wouldn't.
The good news? A few small tweaks to your system prompt can completely transform how your agent behaves. Here's what actually works.
1. Tell it who it IS, not just what it does
❌ Weak: "You are a helpful assistant that answers questions about our product."
✅ Better: "You are Mia, a friendly support agent for Acme Co. You're patient, clear, and never condescending. You explain things simply and always make the customer feel heard."
Why it works: Giving your agent a defined personality and communication style makes responses feel human. It's not just answering. It's being someone.
💡 Focus on how the agent should communicate — its tone, values, and approach — rather than a fictional backstory. "You have 3 years of experience" won't change how it writes, but "you're warm, direct, and never use jargon" will.
2. Give your agent context about who it works for
Your agent might know your documentation inside out, but does it actually know what your company does?
Adding a line or two of base context helps your agent frame answers properly and sound like it actually belongs to your team.
✅ Add context like:
"Chat Thing is a SaaS tool that helps companies build AI agents using their own data. Our customers are typically marketing teams, support teams, and developers who want to create AI agents without coding."
This helps your agent:
- Understand the bigger picture when answering questions
- Speak to your target audience in the right way
- Avoid generic responses that could apply to any company
Think of it like onboarding a new team member. You wouldn't just hand them the docs and expect them to figure out what the company does.
3. Add explicit "Don'ts"
Your agent doesn't know what you DON'T want unless you tell it. This is arguably the highest-leverage section of your entire prompt — don't skimp on it.
✅ Add rules like:
- "Never make up information. If you don't know, say so."
- "Don't use technical jargon unless the user does first."
- "Never mention competitors by name."
- "Don't apologise more than once per response."
- "Only answer questions related to [your topic]. If asked about anything else, politely redirect: 'I'm only able to help with questions about [X] — is there anything along those lines I can help with?'"
That last one is critical and most people forget it. Without a scope limit, your agent will happily answer questions about cooking, tax law, or whatever else users throw at it.
These boundaries prevent the most common agent cringe moments.
4. Set the tone with examples
The fastest way to get consistent tone? Show, don't tell.
✅ Include sample responses in your prompt:
"When greeting users, respond like this: 'Hey! 👋 How can I help you today?' NOT like this: 'Greetings. I am here to assist you with your enquiry.'"
Your agent learns tone from patterns, not descriptions.
5. Give it a "fallback" behaviour
What should your agent do when it genuinely can't help?
✅ Be specific:
"If you can't answer a question from the provided data, say: 'Hmm, I'm not sure about that one. You can reach our team at [email protected] for more help.'"
Without this, agents either hallucinate answers or give awkward non-responses.
6. Keep responses short (and control formatting)
Most users want quick answers, not essays.
✅ Add a length and format instruction:
"Keep responses concise — aim for 2-3 sentences unless the question requires more detail. Use bullet points for lists or step-by-step instructions, but stick to plain prose for simple answers. Don't use headers or markdown formatting unless the answer is long enough to need it."
This single section often has the biggest impact on user experience. Without it, agents default to walls of formatted text for even simple questions.
Quick template to steal
Here's a starter prompt structure you can adapt:
You are [Name], a [role] for [Company]. You're [personality traits — e.g. warm, direct, patient]. [Company] is [brief description of what your company does and who your customers are]. Your job is to [primary purpose] using the information provided. Rules: - Only answer questions about [your topic]. For anything else, redirect politely. - Never make up information. If you don't know, say so. - Keep responses concise (2-3 sentences for simple questions). - Use bullet points for lists or steps, plain prose for simple answers. - Use a friendly, conversational tone — no jargon unless the user uses it first. - If you can't help, direct users to [fallback contact]. [Add any specific don'ts for your use case] Example response (greeting): "[Your example here]" Example response (can't help): "[Your fallback example here]"
One last tip: your prompt is never finished
Don't treat your system prompt as a one-time task. The best way to improve it is to watch where your agent fails, then add a rule to prevent it.
Spot a hallucination? Add a don't. Responses too long? Tighten the length rule. Agent went off-topic? Add a scope limit. Each version should be a little better than the last.
Go try it
Open your agent settings, tweak your system prompt, and test the difference. Small changes here have a bigger impact than adding more data.
Got a prompt trick that works well for you? Let us know on Discord. We might feature it!
