So today we're going to talk about chat model objects and LM objects.
In Linkchain.
The chat model object is often going to be our primary interface for interacting with large language
models, and it is a standard way that linkchain help us to talk to those llms like GPT four, Anthropic
Cloud, Google, Gemini, and even open source models like Llama by Facebook via llama.
So historically, many Llms just took a single string of text as an input and returned us a single string
of text.
However, modern llms are designed for conversation and interaction between a user, so they work best
when we provide them with a list of messages representing the dialogue and the history, and they return
us a message back.
And this is the core of the chat model interface.
The input is going to be a list of structured messages like a system instruction, user questions or
responses.
And the output is going to be an I message representing the response of the large language model.
However, chat model objects are very powerful and beyond generating human like text, they have very
cool capabilities.
So first of all, they have tool calling for LMS that supports tool calling.
And this allows the LM to interact with outside world.
So let's take the example of a math calculation without tool calling the LM could simply hallucinate
the answer of the math equation.
However, with tool calling, it can select and execute a math tool to provide the answer and link chain.
Provide us a standard way to bind these tools to our Are LMS, letting us build agents that can perform
actions based on the user requests like sending emails, fetching data from a database, or integrating
with any external API.
And we dive very deep on tool calling and tools and tools execution in this course.
However, tool calling is not the only thing we can do with the tool.
Calling under the hood can also be used to extract structured information from unstructured text.
And we'll be elaborating on this in the course.
And this enables us with another key capability which is called structured output.
So often we don't want the free form text from data.
We might want a specific format like JSON or a pedantic object that our application can easily process
downstream.
For example, if a user provides context detail in a sentence.
We might want to extract the name, the email and phone number into a predictable JSON or Pydantic object.
In modern jet models, can be prompted or explicitly told to respond in this kind of format with the
given scheme that we give it in.
Chain simplifies it with methods like we've structured output, which we are going to cover in this
course as well.
And let's talk now about Multi-modality.
So while text is the base for everything, models these days are increasingly capable of processing
other data types, particularly images and videos.
And you could potentially send a picture along your text prompt, for example, asking the model to
describe the content of an image or to analyze it or do other tasks on that team.
In blockchain acts as a crucial layer of abstraction over these different models and their capabilities.
So instead of writing specific code for OpenAI, then different code for anthropic and another code
for Google, we can simply use link chains based chat model interface, which will give us a consistent
way to call the models regardless of the provider.
So we'll find integrations for many popular providers like OpenAI, anthropic, Google, Azure, OpenAI,
and many more.
And they are neatly organized into packages like link chain, OpenAI, link chain, llama link chain,
vertex AI, etc. so this doesn't only standardize the interface when it comes to talking those models.
We can get also crucial features when we are developing an LLM application.
So the chat models support functionality like asynchronous operations making multiple calls concurrently.
Batch processing, so sending many requests all at once, and a robust streaming API where we get the
model's output token by token as it's generated in real time, providing us with a better user experience.
For example, if we're using a chat and all of it comes with a seamless integration with Langschmidt,
their tracing and debugging platform to help us monitor our application.
So eventually we get here.
Everything together.
All right.
So let's discuss now how can we interact with these models in link chain.
And the basic model interface which all the chat models derive from it provide us with several key methods.
And the most fundamental one is called invoke.
And invoke is going to take a list of messages and return us a single response message.
We also have the stream method, which is usually for real time application where it will yield output
chunks as they are generated.
And this is very useful if we want to build chat interfaces where we want to get the response token
by token, and if we have many prompts to process, the batch method lets us send all of them efficiently
in groups.
We also have the bind tools method, which will allow us to attach external tools to the model.
Enable tool calling and the with structured output is a convenient wrapper for getting response in a
structured format directly.
All right, I want to elaborate a bit about the input and output of those elements in link chain.
And these are structured as messages.
And these messages are typically have a role.
So we can have a system role for instructions, a human role for the user input or an assistant role
for the AI's response, and the content can be simple texts or a list of content blocks, which is where
multimodal data like images can fit in.
All right, let's talk about initializing a chat model.
And when we do that we need to configure it using various parameters.
Link chain standardizes many common ones across the model providers.
You'll almost always specify the model name like GPT four or Claude three sonnet, and the temperature
parameter controls the creativity.
A lower value like 0.0 makes the output more deterministic and more focused, while a higher value like
1.0 makes it more random and creative.
We have other values like mixed tokens, which limits the length of the response of the LLM, which
is useful for controlling costs and to optimizing output size.
We also have stop sequences, which tells the model when to stop generating text, which is helpful
in specific formatting techniques.
And we are elaborating a lot on this stop argument in this course.
We have the timeout and max retries, which are critical for robustness and handling network issues
or temporarily provider problems, which when we go and scale happens a lot.
And naturally, we'll need of course to provide the API key and potentially a base URL if you're using
some kind of cloud provided service.
So keep in mind that link chain standardize many parameters.
Some models have unique parameters specific to the provider.
So always check that specific integration documentation.
And we are going to see an example of this when we're going to work with Google's Gemini.