What are message tokens?

Message tokens are used each time you send a message to a bot and receive a response. Each message uses between 1000 and 4000 message tokens (but can be more), depending on the questions and the bot's response. This means 200,000 message tokens roughly equate to 100 messages.

Generally, as a conversation gets longer, more message tokens will be used for each message as the entire chat history is sent to the model each time.

Models and message tokens

Depending on the model you choose for your bot, the number of message tokens used will be calculated differently. This allows us to standardise the cost of the available models to our default GPT 3.5 Turbo model. The token multipliers for each model are listed below:

Input and output tokens

Many model providers charge different amounts for input tokens and output tokens. Input tokens are often cheaper and include the prompt, the message, and anything else that is sent to the model when you chat with a bot. Output tokens are often more expensive and consist of the responses generated by your bots.

What does this all mean?

In simple terms, the amount of tokens your bot consumes when used depends on the model you choose. For example, if you choose GPT-4 Turbo 128k, your bot will use 20x the input tokens and 60x the output tokens compared to GPT-3.5 Turbo. This reflects the underlying cost of GPT-4 Turbo 128k being much higher than GPT 3.5 Turbo.

Updating model multipliers

The cost of models changes frequently; they usually get cheaper, and we always aim to pass these cost savings on to customers. As the models get more affordable, we will change the multipliers to reflect the new pricing, meaning your tokens go further.