-: Hey, welcome back.
And in this video we're gonna have a look at a technique
called parallelization.
This is basically where you have some prompt inputs.
You send that out to multiple LLM call providers.
So in this example we've got one, two, three call providers,
which are happening in parallel,
and then we're using an LLM or some type
of functional aggregator to aggregate these results
and then producing a output.
So good examples of these is where you're like asking
an LLM to generate multiple answers
and then synthesizing a new answer,
or reviewing a piece of code
for both security vulnerabilities and stylistic improvements
at the same time.
Or maybe analyzing some text for emotional tone,
intent, and potential biases,
with each aspect handled by a different dedicated LLM.
So firstly, we're gonna import both OpenAI, Pydantic,
and also nest-asyncio.
We are using Nest-asyncio here
because we're gonna run some async code inside
of the Jupyter notebook.
We also are importing Asyncio, the OS package,
typing for literal, a pedantic base model,
and an Async OpenAI client.
Then you'll just need to update your OpenAI key here.
And then we also have the client
and the model that we'll be using.
So I'm just gonna run all these imports.
Now, the first thing that we're gonna do is
we're gonna have a LLM analyze the decision
of what we're currently doing
and deciding and helping with our future direction.
So think of business coach or an executive coach.
The way that we're gonna do that is we're gonna have
a function called analyze decision,
and then we're gonna put a role, which is a string,
and we're also gonna have the context,
which will be a string.
And then what this will return is a string.
We're also gonna have an aggregator function.
We're going to call this aggregate to a final answer,
which is going to take a roll of string, context of string,
and a list of answers,
which have previously happened before,
and it's going to return a string.
Now we're also gonna have a entry point into this,
which will be an async def main function,
and we're just gonna put pass there for now.
So let's work on the analyze decision.
So we're gonna have a prompt,
and our prompt is gonna be an F-string.
And inside of our prompt, you're gonna say you, are a role.
Who's analyzing the decision?
The context is the context.
We're gonna say you are analyzing the potential
for a user and helping them like a career coach.
Then we're gonna set up our chat response
and we're going use
the await client.beta.chat completions.pass.
We're gonna pass in the model and the messages,
and then we're gonna return the content.
Now for the aggregation, we're gonna do something similar,
but we now have the answers.
So we'll say this is the context,
the answers are, and then we'll say
in some, or we'll say here,
your job is to aggregate the answers
and to provide a final answer.
And then what we need to do is then put, use this prompt.
So we're then gonna use that prompt.
After that, then we are going to have a function in main,
which is gonna provide some context.
So they're looking for a job.
The role is career coach,
and we're gonna add some additional context here.
So for example, like they currently have Next.js experience
for two years and SQL experience for three years.
They are thinking about whether to apply
to a SQL role or a Next.js role.
And then we also then need to specifically run
our main function.
So rather than just running it like this,
we are going to use asyncio.run main.
And then what's happening here is we have a context,
we have a role, and then we are using asyncio.gather
to basically run three LLM calls in parallel.
And then once all of those come back,
we are then aggregating all of those answers
into a final answer and printing that out.
So the idea here is that you have a series of steps
that are being run in parallel, not sequentially,
and then you take the outputs of all of those
and you aggregate that with another LM or another function.
And you can see here it's decided some information here
and giving us some aggregated and synthesized response.
So the idea here is that you're really trying to do
lots of separate tasks in parallel
and then synthesize those into a final output
with another LLM.
In the next video, we'll have a look at how we can use
LLMs to be orchestration workers
and to dynamically create tasks at runtime,
which will then be handled and executed.
Cool. I'll see you in the next one.