In this lesson, we are going to talk about what LM models, what foundation LM models will we use to
create multimodal LM applications?
As you know, the problem we have.
When we are trying to develop multi-modal applications, is that the regular LM models we have been
using until now, like ChatGPT, for example, have several limitations.
A regarding the use of images, tables and other elements except text.
It happens the same with most of the foundation LM models that are right now in the market.
So this solution we have is to use the new multimodal LM Foundation models.
These are extremely new.
They are not as mature or are as stable as the regular LM models we know.
But they are extremely, extremely interesting and have a huge potential.
The first pioneers in this field were clip, Laba and Fuyu.
These are extremely new multimodal models.
We are talking about clips around 2021 and Le-van Fuyu were created in 2023.
In the same year we had a the current leaders of the market GPT four, Vision and Gemini Pro.
As you know, GPT four vision is a paid model from OpenAI, and Gemini Pro is a paid model from Google.
Right now, the main leader is GPT four vision.
But Gemini Pro is is is doing a very good job in multimodal in the multimodal space.
The pioneers clip lava and Fuyu are all open source, and we trust that in the coming months they are
going to be in a level more similar to GPT four vision.
Let's see, GPT four vision evolves and is even far better than it is today.
Very important a this caveat here.
So when we are talking about multimodal LM applications or multimodal LM Foundation models, we need
to understand that they are in a very early stage.
The maturity of multimodal LM models is still far from regular LM models.
This is very important.
So right now is a good moment for experimentation and good moment to position yourself for my immediate
future.
But it may be still a bit early for production applications, because you will see that even the most
advanced multimodal LM models like GPT four vision have some problems.
They are not 100% stable or 100% a usable right now for production applications.
So I would say right now, very good moment for experimentation and to position yourself for the immediate
future.
The evolution of these multimodal models is really, really, uh, fast.
It's being very, very fast.
So.
We don't know when it's going to be, the moment where they are stable enough to be using production
applications, but I think we can be very, very, uh, near right now.
I'm talking, uh, in March 15th, 2024.
So probably May June, we will start seeing solid production applications in this space.
Let's see.
In the next lesson, we are going to start a studying the best multimodal LLM foundation models we have
right now in the market.
We will see how to use GPT for vision.
What are the main use cases for GPT for vision, and what are the limitations of this model?
We will see this starting in the following in the following lesson.