-: Alright, welcome back.
So in this video, we're gonna have a look
at how you can use streaming inside of open AI's API
and what streaming is.
The notebook that you'll be following along with
is inside the OpenAI features and functionality folder.
It's called streaming with responses API.
So open that up and at the start,
we're gonna do a PIP install on the OpenAI client.
And to give you a bit of information about streaming,
basically rather than waiting for the entire response,
you can use OpenAI streaming to get all of the changes
that are happening in real time inside of Open AI's API
and get those events streamed directly
to you via your client so that you can give that
to users immediately.
Just like you can see here when I've got some information
and I say generate an article outline,
ChatGPT won't wait until respond with everything,
it will actually stream back things in real time.
And so we'll see,
make it about digital marketing really long.
And you'll see that we don't have to wait
for the entire response to come back, right?
It's just streamed directly back to us in the UI.
And this is essentially what streaming architecture is
where rather than the entire response coming back in one go
and waiting for the end response, we can stream the changes
that happen in real time and increment those delta changes
to become a final output.
So OpenAI exposes a couple of different event types,
when the response was created,
what the output text.delta is,
when the response is completed,
and whether there is an error as well.
So let's follow along
and just have a look at this coding example.
We have a GPT 4.1.mini
and you're going to need to set your Open AI key,
so make sure to do that.
Now, you'll see there's only one change that we made
to get streaming, which is adding the stream argument
to the client.responses.create and setting that to true.
And then basically, all you do is,
you iterate over the stream
and then you can print the details.
So you can see we've got this event,
which is a response stream event
and we can see there is a type of this
and there's gonna run this now
and we'll see the stream events coming in in real time.
So you can see here, you've got responses.created,
then it had that in progress, the output item.added.
So all of these come in in real time.
So I'll just try and run it again so you can have a look
and I'll scroll back up rather quickly.
So let's go and do this one more time and have a look.
And you can see, there you go.
So these are all the events that are coming in
and you've got things like the responses created,
the responses in progress.
You can use all these different types of events
to figure out how you want to talk to your front end.
The next thing I'm gonna show you is
how you can build the text up in real time.
So we've got the same thing happening here
where we're gonna request that the model do a tongue twister
and give us some output from whilst using the streaming.
So you've got that streaming is equal to true.
We set our text to be nothing.
And then, as we iterate over the stream,
if the event type is equal
to a response.output.text.delta,
we take the original text and we apply the delta to it.
So have a look at what you get from this
when you're generating this.
So, we started with the sure,
then we added the exclamation mark, then the here,
then the it, then the is.
So basically, that's what's happening is,
we are setting streaming to true,
we're iterating over the stream.
And then what we're doing is, when the event type
is equal to this output text.delta,
we are then just basically building up this Python variable
in real time and printing it out every single time
the delta is changing.
Cool. So in summary, to you streaming, you just have
to set the stream easy equal to true parameter
in your API request.
You will need to handle the event types and the deltas.