In this video, we'll be writing our backend
endpoint that will take an uploaded image
and an optional list of preferences and ship
it to the Gemini API to return a list of travel
suggestions.
To begin, we need to get in some modules to
help us with our file upload and HTTP error
handling.
First, we're going to be using the File and
Upload File modules from FastAPI.
We've already brought that in so we're good
on that front.
Next for our image processing, we'll be importing
the Image class from our Pillow package.
We'll also import the IO module for file operations.
So let us import them.
Just below JSON, say from DIL, import image
and also import IO.
Good.
We now have every module that we need to get
started.
So let's write our endpoint.
Just below the search endpoint, scroll down,
here we're going to be writing our endpoint
and the corresponding handler.
So here we'll say at app.post,
forward slash API, forward slash suggest by
image.
And for the handler, we'll also be giving
it an async handler because we're going to
to be performing async operations.
We'll call it suggest by image.
And it's going to be taking two parameters.
First is file, which is the file we're going
to be uploading.
That instance of upload file, set that to
file.
And we'll also be getting our preferences,
which is going to be an optional string.
And default to none.
Let's come down here and add some little documentation,
say it generates travel suggestions based
on
an uploaded image.
To begin our request, we'll need to wrap it
in a try except block so let us write that
out.
Let's pass in a suitable message for our error,
use an f-string and say error processing image
and let us just give you the error.
That's good.
Now we can begin processing the user's request
by first getting our image data.
So I will say image data and set it to await
file dot read.
That's from the file that is coming in our
request.
Then our image is going to be processed using
the image package from pillow.
Good.
Next let us take care of our preferences and
build a string of user preferences that we
can embed inside our prompt.
Just going to say, pref list.
Let's get an array of what was sent.
And once we have this, we can then build a
string of comma-separated preferences.
Just as we did with the text search.
Personal preface here.
Good, now we have a preferences text that
we can embed inside our prompt.
Now speaking of the prompt, I'm also going
to be saving time by bringing the elaborate
prompt here from the exercise files of the
completed version of the course.
So in the exercise files of the course, you
get a base project, which we are building
on right now.
And you also get a branch where you see the
completed version of the course.
So you can always grab your prompt from there,
as I'm going to be doing right now.
So paste the prompt here, and I'm quickly
just going to format this.
Let's get formatting up there, to this, to
this, good.
Now let us see what we have inside our prompt.
So this prompt says analyze this landmark
or travel image and suggest 5 similar destinations
with comparable features, architecture or
atmosphere.
Then we embed our preferences text, if any,
and says for each destination, we provide
the same details we wanted it to provide when
we were doing the text-based search.
And we also give it the same data format that
we want back, at least with JSON objects of
each destination.
And we also want it to return just the JSON
array
and no additional text.
Perfect.
Now we can take this prompt and our image
and send them as a payload to our Gemini API
in our request.
So let's do that.
Let's just come down here and scroll this
into view.
And here we can say response equals
genAIClient.models.generateContent.
And for our model, we're passing our GeminiModel
constant.
And for contents, we're passing a list of
content types, which would include our prompt
and our image.
So this is all good.
Now we will need to process the response.
We need to get the response text.
We need to strip off all the markdown that
was brought with it.
We also need to make sure that we are stripping
all white spaces, passing it, and returning
our destinations.
Now we've already done all that for our previous
endpoint.
So let us just copy that.
I just scroll down here and we can simply
copy everything from here down to here.
So I'm just going to collapse this once again,
then we come down here and simply paste this.
Good.
We have our response text stripped.
We have all the markdown removed, and we strip
it once again of whitespaces, pass it as json,
and return our destinations.
Awesome.
Now let us test out this endpoint.
First let me clean up this function.
Yeah, let me also come out of it.
And then we can save this file.
To test our endpoint, we need to make sure
that our backend is running.
So let us run that.
So python main.py,
run it, make sure that your file is saved
if you don't have autosaved
turned on in your editor.
And now we can head over to our browser and
test it out in the docs page.
So here on the docs page of our API, we currently
have two endpoints.
But if we refresh that, we're now going to
have three, which is our suggest by image
endpoint.
So let's test this out.
I'm going to expand this, click Try It Out,
and as you can see, it gives me a place where
I can select a binary file, and we can also
add a string of preferences here.
So I'm going to select a file, which is a
London Eye image.
preferences let's say a string where we say
parks and shopping now we can click
to execute that and as expected we get our
results back 200 which means success
and if we look at our response body we can
see our destinations we can see that
But the Singapore Flyer also has this type
of wheel.
The High Roller in Las Vegas also has this
type of wheel.
So our API endpoint is working fine which
is just fantastic.
Now we can start our integration with our
frontend using this endpoint in the next video.