In this lesson, we are going to talk about the limitation that right now GPT four vision has.
So the first important limitation of GPT four vision is for medical images.
This is the first use case everybody was thinking about when they launched GPT four vision.
And they immediately a.
Warn us about this.
So GPT four vision is right now not suitable for interpreting specialized medical images like CT scans,
and shouldn't be used for medical advice.
So what they are saying is be careful.
These are powerful technology, but it has limitations.
So in fields like health, you have to be super careful using this.
Second.
GPT four vision may not perform optimally right now.
Today, when handling images with text of non-Latin alphabets such as Japanese or Korean.
Okay.
Second, limitations.
We have many students from Japan and South Korea, so pay attention to this.
If in the image you have text that is non Latin and non Latin alphabet, careful with that because the
model may not perform optimally.
Regarding a small text.
The recommendation of OpenAI is enlarge text within the image to improve readability, but avoid cropping
important details.
Okay, so careful with text that is too small in your image.
About rotation.
ChatGPT for vision may misinterpret rotated or upside down text or images.
Therefore, with that as well, if you don't have your image in the proper position.
ChatGPT for vision can be confused, so it's very important to prepare the image before working with
a GPT for vision.
Next ChatGPT.
Four.
Vision may struggle to understand graphs or text where colors or styles like solid dash or dotted lines
vary.
Okay.
So you need to understand all these scenarios where GPT four vision is not going to work well and avoid
them in order to have today an application that works, uh, in a professional level.
Next, GPT four vision may generate incorrect descriptions or captions in certain scenarios.
They don't give us more detail about that.
They, as you can see, provide a very general statement a covering themselves about possible mistakes,
possible errors.
So you will see that when we talk about the different methods we can use in order to build a multimodal
LM application, we are going to see different degrees of accuracy depending on the different methods
we use.
So just a little advance about what we are going to see later regarding accuracy.
What about the image shape?
So GPT four vision struggles with panoramic and fisheye images.
So this is very important.
Uh, if your business or your application is going to handle these kind of images, careful with these
panoramic images and fisheye images.
Also about metadata and resizing.
Right now, GPT four vision doesn't process original file names or metadata, and images are resized
before analysis, affecting their original dimensions dimensions.
Okay, so careful about this as well.
A couple of additional things.
First.
GPT four vision may give approximate counts for objects in images.
So do you remember the first example?
We saw this picture with some children and we asked GPT for vision how many children do we have here?
And he says seven.
And it was correct.
But this is not always the case.
In some cases the the counting is going to be approximate.
It's not going to be exact.
So careful with that as well.
And finally, regarding CAPTCHAs, I already told you.
OpenAI says for safety reasons, we have implemented a system to block the submission of CAPTCHAs.
You see, because ChatGPT for vision was.
Breaking all the captures.
So this is a very interesting note about the power and the potential of the the multimodal LM models.
So this is the question you may ask if GPT four vision is so good, why do we need multimodal LM applications
for.
If you remember, when we initially talk about the regular chat, GPT model and LM applications, we
also made the same question.
So let's see the answer in the next lesson.