---
title: "Gemini 4 Hydrogen, Opus 5.5, Gemini 3.8 Flash | Chat Thing"
canonical_url: "https://chatthing.ai/blog/gemini-4-hydrogen-opus-5-5-and-better-testing"
last_updated: "2026-11-13T14:43:50.055Z"
meta:
description: "Gemini 4 Hydrogen tops our customer-support benchmark. Plus Claude Opus 5.5, Gemini 4 Argon and Carbon, Gemini 3.8 Flash, and searchable test results."
"og:description": "Gemini 4 Hydrogen tops our customer-support benchmark. Plus Claude Opus 5.5, Gemini 4 Argon and Carbon, Gemini 3.8 Flash, and searchable test results."
"og:title": "Gemini 4 Hydrogen, Claude Opus 5.5 and test results you can search - Blog - Chat Thing"
"twitter:description": "Gemini 4 Hydrogen tops our customer-support benchmark. Plus Claude Opus 5.5, Gemini 4 Argon and Carbon, Gemini 3.8 Flash, and searchable test results."
"twitter:title": "Gemini 4 Hydrogen, Claude Opus 5.5 and test results you can search - Blog - Chat Thing"
---
[← Back to blog](https://chatthing.ai/blog)**Changelog**# **Gemini 4 Hydrogen, Claude Opus 5.5 and test results you can search**
Zef
13 Nov 2026
~ 6 min read
---
---
### **On this page**
1. Gemini 4 Hydrogen benchmark: first on SupportBench
2. New models: Claude Opus 5.5, Gemini 4 Argon, Gemini 4 Carbon and Gemini 3.8 Flash
1. [Models being retired](#models-being-retired)
3. [Chatbot testing: search, filter and export your test results](#chatbot-testing-search-filter-and-export-your-test-results)
4. [Review your prompt for problems](#review-your-prompt-for-problems)
5. [Smaller improvements](#smaller-improvements)
6. [FAQ](#faq)
1. [Is Gemini 4 Hydrogen good for customer support?](#is-gemini-4-hydrogen-good-for-customer-support)
2. [Which plans can use the new models?](#which-plans-can-use-the-new-models)
3. [What happens to my agent if its model is retired?](#what-happens-to-my-agent-if-its-model-is-retired)
4. [Can I test my agent before switching models?](#can-i-test-my-agent-before-switching-models)
7. [Earlier this week](#earlier-this-week)
Alongside this week's bigger releases, we've shipped:
- **Five new models**, including Gemini 4 Hydrogen, which is now first on SupportBench, our customer-support benchmark.
- **Rebuilt test results.** You can search them, filter them and export them.
- **Review for problems**, which finds what's most likely to make your agent misbehave in its prompt and fixes it for you.
- **A more accessible chat widget**, and clearer errors when a file won't upload.
## Gemini 4 Hydrogen benchmark: first on SupportBench
Google released Gemini 4 Hydrogen, the cheaper, faster model in its Gemini 4 family, on 12 November. We ran it through [SupportBench](https://chatthing.ai/models/supportbench), our test of how well a model handles real customer-support conversations. It scored 89.8, the highest of the nine models we've tested, just ahead of Gemini 3.7 Flash on 88.6.
- **It's the hardest model to trick.** It never passed on an instruction planted in a forum page, and never gave billing details to the wrong person. It made no money or privacy mistakes in 155 conversations.
- **It has the best tool use we've tested.** When an order lookup failed, it tried again every time.
- **Graders prefer its answers.** Shown two models' replies to the same conversation side by side, graders picked Hydrogen's in two out of three decided matchups against Gemini 3.7 Flash, Grok 4.6 and Claude Sonnet 5.
It isn't perfect. On one simple order-status conversation it sent an empty reply in two of five runs. It offers to bring in a person more often than other models do, even on questions it has already answered. And it costs about 5.5 times as much as Gemini 3.7 Flash per resolved conversation, and is slower. If cost or speed matter most to you, Gemini 3.7 Flash scores only 1.2 points lower.
Our full report, [Gemini 4 Hydrogen for customer support](https://chatthing.ai/models/gemini-4-hydrogen), has every score and transcript excerpts.
## New models: Claude Opus 5.5, Gemini 4 Argon, Gemini 4 Carbon and Gemini 3.8 Flash
- **Claude Opus 5.5**, Anthropic's newest Opus model.
- **Gemini 4 Argon** and **Gemini 4 Carbon**, the Gemini 4 models Google built for harder problems. Google calls Carbon its strongest model yet.
- **Gemini 3.8 Flash.**
All five can read images, use your Power-Ups and think before they answer. Choose one in your agent's model settings, on the Standard, Pro and Enterprise plans. You can switch back at any time without changing your knowledge.
Each model uses [message tokens](https://chatthing.ai/docs/account/message-tokens) at its own rate:
| Model | Input | Output |
| --- | --- | --- |
| Gemini 3.8 Flash | 1.5× | 7.5× |
| Gemini 4 Hydrogen | 3× | 15× |
| Gemini 4 Argon | 4× | 20× |
| Gemini 4 Carbon | 4× | 20× |
| Claude Opus 5.5 | 8× | 40× |
### Models being retired
Their providers are retiring these models:
- **Claude Sonnet 4.5** on 30 November
- **GPT-5, GPT-5 Mini, GPT-5 Nano and GPT-5 Pro** on 11 December
- **GLM 4.5** on 31 December
GPT-3.5 Turbo, GPT-4 Turbo, Claude Sonnet 4 and DeepSeek R1 have already gone. If one of your agents uses a model on this list, we'll move it to the newest model from the same provider before the model is retired. If you'd rather choose the replacement yourself, change it in your agent's model settings now.
## Chatbot testing: search, filter and export your test results
Test cases check your agent's answers against what you expect, so you find wrong answers before your customers do. We've rebuilt the page you see after a test run.
- **The strip at the top** shows the pass rate, how many cases passed and failed, the average response time and the tokens the run used.
- **Search** for a question, or for a phrase in your agent's answers.
- **Filter** to **Failed** or **Passed** results. The filter is part of the page address, so you can send a teammate a link to a run's failures.
- **Click a result** to see it in full beside the list: the question, the whole response and every check with the reason it passed or failed. Press **J** and **K** to move between results.
- **Fix it from there.** **Edit case** opens the test case, and **Rerun case** runs it again with your agent's current settings.
- **Export CSV** downloads the results, with one row per test case and columns for each check. If a search or filter is on, only those results are exported.
On a phone, the list and the result open as separate screens. Testing is on the Enterprise plan. [Test your agent](https://chatthing.ai/docs/bot-settings/test-your-bot) has the details.
## Review your prompt for problems
Open your agent's prompt, click **Edit with AI** and choose **Review for problems**. Otto reads the prompt and points out what's most likely to make your agent behave badly:
- rules that contradict each other;
- rules too vague to act on;
- text that repeats what a Power-Up already tells your agent;
- no guidance for when it doesn't know the answer or should hand over to a person;
- references to Power-Ups or data sources you've since deleted.
Click **Fix it** and Otto makes the change, which you **Accept** or **Discard** in the editor like any other edit. The suggestions sit beside the editor, and **Review again** checks the prompt after you've made changes. A review doesn't change anything by itself, and it uses your agent's message tokens. [Write your agent's instructions](https://chatthing.ai/docs/bot-settings/instructions) has more.
## Smaller improvements
- **A more accessible chat widget.** Every button in the widget has a name that screen readers read out, in the widget's language. Keyboard focus is clearly visible, in your brand colour when it has enough contrast. Screen readers announce sent messages and new replies, and the launcher and preview bubble work with a keyboard.
- **A hint in the message box.** The empty message box now says "Type your message…" in each visitor's language. To use your own wording, fill in **Message box placeholder** in your web chat's **Advanced features**. [Customise the widget's appearance](https://chatthing.ai/docs/channels/website/appearance#message-box-placeholder)
- **Clearer file uploads.** If a file can't be uploaded, Chat Thing tells you which file and why, and the rest of your files still upload. Before, a failed upload could fail without telling you.
## FAQ
### Is Gemini 4 Hydrogen good for customer support?
Yes. It scored 89.8 on SupportBench, the highest of the nine models we've tested, with the best safety and tool-use results. It costs more than Gemini 3.7 Flash and is slower, and it occasionally sends an empty reply, so watch your conversations when you switch. The [full report](https://chatthing.ai/models/gemini-4-hydrogen) has the details.
### Which plans can use the new models?
Choosing a model is on the Standard, Pro and Enterprise plans. On Free and Basic, agents use the default model.
### What happens to my agent if its model is retired?
Before a model is retired, we move agents that use it to the newest model from the same provider. You can choose a different model yourself at any time in your agent's model settings.
### Can I test my agent before switching models?
Yes, on the Enterprise plan. Run your test cases, switch the model, run them again, then compare the two runs' pass rates and failed results.
## Earlier this week
- [Home: all your agents on one page](https://chatthing.ai/blog/home-all-your-agents-on-one-page)
- [Turn your agent into an MCP server](https://chatthing.ai/blog/how-to-build-an-mcp-server-without-code)
- [Web search replaces Google Search](https://chatthing.ai/blog/web-search-replaces-google-search)
- [Chat Thing for Discord](https://chatthing.ai/blog/chat-thing-for-discord)
### **Related articles **
[](https://chatthing.ai/blog/chat-thing-for-discord) [Chat Thing for Discord: AI moderation by your rules and a help forum that answers itself
](https://chatthing.ai/blog/chat-thing-for-discord)
Discord AI moderation by rules you write in plain English, an agent that answers every post in your help forum and hands the rest to your team, and automations that react or reply when things happen.
[](https://chatthing.ai/blog/web-search-replaces-google-search) [Web search: a more powerful replacement for Google Search
](https://chatthing.ai/blog/web-search-replaces-google-search)
The new Web search Power-Up searches the websites you choose, reads the pages and PDFs it finds, and links to its sources. It replaces Google Search, which stops working on 1 January 2027.
[](https://chatthing.ai/blog/home-all-your-agents-on-one-page) [Home: see how all your agents are doing
](https://chatthing.ai/blog/home-all-your-agents-on-one-page)
Chat Thing now opens on Home: one page for every agent in your team, with what they handled, anything that needs a person, and shortcuts for what to set up next.
- **Previous **
- [← Chat Thing for Discord: AI moderation by your rules and a help forum that answers itself](https://chatthing.ai/blog/chat-thing-for-discord)