Hey, in this video, we're going to talk about the difference between chat models and reasoning models.
We'll learn about why you should use different types of models depending upon the task.
So firstly chat models.
If you've ever used something like ChatGPT, then you've used a chat model.
A chat model is basically a large language model that uses a sequence of messages in a message history
as inputs, and returns messages as outputs.
On the left here you can see we've got what is digital marketing and that is a user message.
You can also see that we've got a response from OpenAI that is an AI message.
There are three different types of messages that generally chat models expose.
There's the developer system message.
There's the user messages which are the messages that you would generally type.
And then there's the AI messages that come back from a chat model.
There are lots of large language model providers to choose from.
For example, you have OpenAI with their GPT series.
We also have anthropic, which offers Claude Sonnet and Claude Opus models.
There is also meta with their free open source models, which are the llama models.
There is also Google Gemini.
They have both a Google Gemini Pro and a Google Gemini flash series of models.
There's Mistral and many more.
Some of the characteristics that you'll find with chat models is that they are actually very quick.
They have low latency.
These models respond quickly and often stream the tokens to you word by word.
They're very good in terms of their reasoning capabilities.
For example, Claude or GPT four are able to solve easy to medium difficulty tasks.
Reasoning models are a newer type of model that uses something called test time compute scaling.
And basically you can think of test time compute scaling as basically allocating more time to thinking
and reasoning before returning the result or the response.
So in general, these models take a lot longer because they're actually allocating time at execution
to think through the problem rationally and then coming up with an overall solution.
Let's have a look at this so we can see that it's got multiple turns.
Turn one has an input.
It has a bit of reasoning.
That output then goes into the second turn which then has some input.
And then that goes into the reasoning which then produces an output.
So you can see it's a multi turn experience.
And basically that means that it's got more thinking time.
So when you ask a question we'll allocate more time before actually responding to you or your query.
So the kind of characteristics you'll get when you're using a reasoning model is increased latency.
Now OpenAI, which have released GPT five, have two modes for using this.
There is the fast mode and there is the thinking mode.
You'll find that if you use the thinking mode, it has additional latency.
It takes longer to come back and longer for you to get that response.
Now, due to this test time compute reasoning, these models will definitely outperform chat models.
If you have a very hard problem or you're not sure, or you don't mind suffering with some additional
time for the response to come back.
If you want to make sure and it's a very important piece of work, definitely use a reasoning model.
They have more time that they spend on the problem and they will give you more accurate results.
Realize that there's always a trade off here, and you are spending longer waiting for the AI model
to get back to you.
Here are some examples of when you should use a chat model versus a reasoning model.
So for example, when we're doing content, quick emails and posts, but when we might use a reasoning
model is technical documentation or research analysis.
For example, if we look at problem solving, basic calculations or simple Q&A.
But if we're using reasoning model, maybe we'd use that for multi-step debugging.
Okay, so you now know the difference between the chat model and a reasoning model.
Let's have a look at how we can do this inside of OpenAI.
Go to ChatGPT.
And then if you go and click in the top left, you can see that we have the option to use GPT five with
either auto fast thinking or if you even upgrade even further, you can get a Pro, which is research
grade intelligence.
You can see that auto will basically do model routing, where it look at your query and decide is it
going to be using a fast inference model.
So a chat model.
And also we have the option to do thinking.
So if I say thinking and then what we're going to type is what is data science.
And then what you'll see is when we then submit our query, you can see that ChatGPT is thinking if
you click on the thinking, you'll see that you can actually see the thoughts and the reasoning tokens
the ChatGPT decided to do.
Then we get our response.
So we had to wait seven seconds for the thinking model to come back with a response for us.
If we contrast this with fast and I go and load a fast model, and then I'm going to keep in the same
conversation and say, what is data science?
You'll now see that we don't have any thinking tokens.
We basically get an immediate response.
However, the model isn't thinking about how best to service our query, so that is how you can switch
now.
Obviously, if you want, you can stick on auto mode, which will look at how complicated your query
is, and it will decide whether to use a fast model or whether to use a thinking based reasoning model
for best practices.
I would recommend for tasks that require increased accuracy, or you must make sure that the error is
less.
Use reasoning models for tasks such as content generation or copywriting.
You can use a smaller chat model.
Reasoning models also take time to respond, so you can also have a reasoning model working on a task
for a couple of minutes whilst you're also using Claude or GPT four alongside that.
So don't wait 2 to 3 minutes for the task to complete.
Go and work on another task whilst you're waiting for the reasoning model to get back to you.
In the next section of the course, we'll look at how you can use standard practices such as list generation,
sentiment analysis and writing clean and clear instructions to make sure that you get the most out of
these models.