In this video, we're going to have a look at something called the mixture of agents.
We're going to look at how a layered architecture using layered amounts of LMS and then an aggregator
can potentially improve the efficiency or the outputs gained.
This works really well when you're using open source models.
And if we have a look at the diagram here and sort of the bottom left hand side of the screen, we can
see that you have a prompt which goes into a layer.
This layer of LMS will then answer that question.
We will then have a secondary layer which will aggregate the results of those LM calls, will synthesize
those responses into a single higher quality response, which will then go through into being the final
output.
We've done a proof of concept which is again referenced to Together computer.
They've put together a really great paper on mixture of mixture of experts.
So let's have a look.
So the first thing that we do is we install a bunch of packages and then we run our imports.
Now you will have to for this notebook script have an OpenAI API key, a anthropic API key and a mistral
API key.
So all of those are different platforms.
And then what you can see here is we have a user prompt like, what are three fun things to do in San
Francisco.
We have our reference models which in this case we've used ChatGPT for oh uh, Claude 33.5 sonnet and
also Mistral's API.
And we have our aggregator model that's set to being GPT four.
Oh, and then what we're doing here is we're sort of saying you you've been provided with a set of responses
from various open source models to the latest user query or task is to synthesize these responses into
a higher quality response.
So and then you get the response from the models.
And then we run each model and we put a bit of time in and we sleep those models.
and then we get the responses.
So this is to run the LM function for specifically running the LM.
And then this main function here will run for each model in the reference models.
It will run that this function here and it will gather up the results.
And then we will then use the aggregator here with the system message from the aggregator system prompt
that we were talking about earlier.
And we'll pass in all of the results and enumerate those.
And then we have our human user message, which will be the user prompt.
So let's have a look and see what happens.
So now when we run this async code or run what's actually happening is we've got multiple layers of
these llms being called.
And then that's all being aggregated again by GPT four zero to try and hopefully synthesize a better
response.
So this is a really simple way for you to integrate mixture of experts into your LM based applications.
A couple of key points here.
This will add probably extra latency to your application.
The reason for this is that the slowest API vendor is going to become your bottleneck.
And also this isn't good for a streaming based architecture.
So if you need to stream the results immediately to users, that's not probably the best use of of of
this kind of application.
So you are going to get additional latency, but it can hopefully improve your results.