-: I'm gonna talk about SAMMO,
which is a new library from Microsoft.
It's quite useful for writing prompts
and also testing and optimizing.
And I'll walk you through this.
The way that you call a model with SAMMO
is you need to import runners OpenAIChat,
this is for OpenAI.
And then you just need the generate text component
and the output component.
And the way it works is you put in your model ID,
your OpenAI API key,
which here it's getting from the environment,
and then that you also set up a cache file.
So you don't need to worry too much about this.
This basically just means every time you run,
it's gonna look in that cache file.
And if you've already run the exact same prompt before,
then it's gonna get that result.
And you can set a timeout so it does things like timeouts
and retries automatically, which is quite nice.
And I'm just gonna tell it's talk bug sheets
and then tell it in my job.
And here we go.
So we just put the output, we put in that program
that we made, which was with the generate text.
This is our prompt and then this is our system prompt.
And then we're just passing the winner,
and we get a nicely formatted output here.
Don't worry by the way, we don't have any inputs
in this case.
If we added a variable in here, then see something coming.
Okay, cool.
All right, so you're gonna see that in a second.
The other thing you can do is you can pass in history.
So if this is like a really easy way
of handling the message stream,
which is sometimes a bit of a pain.
So here we're saying write a four line poem about my job,
and we're passing the first program at the history.
And then just gonna run that.
And here we go.
It's still using the Shakespeare stuff,
which is really helpful.
So it's talking about prompt engineering
because I've told it my job in the previous message,
but it's also still talking like Shakespeare,
like the system prompt.
I didn't have to add that in.
Cool, so that's how that works.
One thing that it does, which is quite fun
and quite useful I think for debugging is
you can take the outputs, and then you can do a plot trace.
So if we just a...
Just need to...
Assign it to a variable, and then we can see,
here we go, you can see, the first text we put in here,
"Hello, my name is," and then the response.
And then we pass that into generate text,
and we have the other prompt here,
and we get the final result.
Just really helpful, especially if you have nested programs.
Just very helpful to do this, in terms of the plots,
and that's gonna make it a lot easier to debug more complex.
And the other thing which I find quite useful is being able
to run it in loops and kind of join stuff together.
One thing you can do is use this union,
port from components, port union, and then...
we can see N equals five.
We're gonna run through this five times
and say fruits equals, generate text,
what is the name of a fruit?
And just going to look through that.
We'll say randomness is 0.9, say seed equals I.
So what this is gonna do is gonna give us a different
response every time because we're setting a new seed
each time and bringing everything together with this union,
we could also assign this to a variable
which might make it a little bit easier to understand.
What we're doing is we're getting all the fruits
and we're spinning up a new API call for each one
and then the union brings them all together,
so it makes more sense if we plot the program
'cause now we've assigned the program to a variable,
we can plot that program.
So this is like the plot of the prompts
that we looked at before.
This actually shows like the inputs
and outputs and the responses.
But in this case, this is plotting the program.
And what this library is trying
to do really is just make prompts into programs.
So there's inputs, there's outputs,
so even if you haven't run it yet, you can see
how are these different things joined together.
And here we have four different generated texts.
This has a seed of four, this has a seed of three.
This has a seed of two, seed of one, seed of zero.
And then we're gonna join all them together.
And what we get is this nicely formatted output at the end.
You can see these are the responses.
Sometimes it just says apple, but then sometimes
it gives you a bit of a longer response.
Cool, okay.
Now where it starts to get a little bit more complicated,
but also more useful is in the stuff that it's done
to make it easier to run through a bunch of responses
and bring 'em all together into a batch.
The way this works, I'm just gonna import,
actually I'll copy and paste this first, quicker.
Just gonna import for each as well as a template.
And then extracting Regex.
So these are helpful little programs,
and what we're gonna do is, in this case,
we're gonna make a bunch of OpenAI models
as just as an example.
So extract Regex, generate the text, and say generate a list
Five models.
I map each model in all text.
And.
And what we want to do is run, say, model blurbs.
We're gonna create some output, we're gonna say ForEach.
So it's gonna iterate through the different models.
I'm gonna say each model.
And then we're gonna say generate why is...
good model?
Cool, what have we done here?
We are creating a, basically like a program
that iterates through all of the models
that we generate in this first prompt.
Previously we were iterating through using the same prompt,
here, what is the name of the fruit?
Or in this case we could change this.
We could say generate a name of the model from OpenAI.
So you could change this to, you want to see,
bring everything together.
Now we didn't use any variables in this print,
we just changed the seed.
So we had to do a loop, but with SAMMO's ability
to run things asynchronously, we didn't have to do a loop.
We could run all of these things at once,
and input the variables here.
This is the inputs, it's just the model.
And it's gonna bring in all
these different models into them.
And that's what we're doing with the ForEach.
What this is gonna do is, it's gonna generate
five different models, we're going to extract it
by using this ExtractRegex, that's gonna pull all
the different model tags together.
And then we're gonna generate
a description of that model for each.
So it's a little bit of a different way of doing things.
Error here, I have a problem with my Regex,
just figure this one out.
Oh I didn't actually give it the Regex.
That would help.
So if we need to pull different types
of output from from the prompt description,
and then we need to kind of tell it what the format is.
And in this case we told it to wrap it in model tags.
Actually we could say wrap each model,
and/or, that way we know that each model
that we get back is be wrapped to these tags,
and therefore we can use the Regex to look for them.
So that's gonna extract those five out.
So hopefully you understand we're gonna see
how that works now because we can do the trace.
This is potentially complicated otherwise,
but we can see it all happening.
We have, in this case, this prompt,
generate a list of five models.
Now I wrapped them in the tags, and this is what it output.
It said we've got GPT-3, GPT-4, DALL-E, Codex, and CLIP.
Then we've extracted the Regex says given as this output,
this list, 'cause these things were wrapped
in the model tags and then it worked.
And then we went through the ForEach,
and for the ForEachs we had,
so why is this model a good model?
And then this is the prompt pulling in this format here.
And then what that's gone through is
hold everything together and then for every prompt
we basically have the full response for each of them.
So hopefully that makes sense.
What we get in the end is, here we go, here's the app.
GPT-3 excels due to its vast tracking data,
advanced architecture, et cetera.
DALL-E does this, CLIP does this.
So everything is pulled at the end.
Cool, so hopefully that makes sense.
It depends on your preference in terms
of how you'd like to do these things.
But I think this is quite an interesting way to approach it
where you're getting a full kind of visual understanding
of how everything is coming together.
And you can click into these and say,
Hey look, GPT-4 is pulling in here,
and then that's giving the description here,
and then that's showing up here in the final thing.
Like I find when you're do building these things
in LangChain or DS-Pi, it's pretty hard
to understand what's actually happening in these models
or these kind of chains that you're bringing together.
So even though this is still a little bit complicated
to understand if you're not used to it,
it is so much more useful once you get the hang of it,
it's much more easy to, so part of the program
like I just did here, you can see like
how everything comes together,
and the difference between plotting the program
and plotting the prompt is that here
we've actually generated the five models,
whereas plotting the program,
it's just the structure of the program.
It's not, yeah, it hasn't actually generated the models yet,
if that makes sense.
It's just showing you what's gonna happen.
You bring everything together.
It's a little bit easier to understand.
Okay, so let's make use of some of the paralyzation.
The next thing we're gonna do is we're gonna bring
in this data table.
And what a data table lets you do is bring everything
together and run it in parallel
like I would do with Pandas data frame.
This is quite often what I do,
but we're gonna recreate the runner,
except in this case we're gonna say rate set that
as, so it's gonna run six things at a time,
and just reiterate through some numbers.
So the numbers you want to go through,
through the numbers one six.
And then let's create a variable for this output.
So the output will be, we wanna generate text,
and then template and then the template is,
it's gonna be this, only the corresponding Greek letter
in English.
And then we're gonna give it the,
sorry, it needs to be double curly braces.
Okay, what are we doing here?
Which is different from what we did before.
So what we did before was we were relying
on the output of that first prompt which gave us the models,
and then we're stuffing those models.
In this case we've created a list of numbers
and put them through into our prompt template
using the double curly braces.
So it's just a little bit different
in terms of the approach.
We're gonna have a runner, and then we're gonna create
a data table of those numbers.
So this, hopefully it works.
Yeah, here we go.
So we can see that we get the inputs and the outputs,
and it's pretty easy to understand.
Now alpha, beta, gamma, delta, epsilon
these are the Greek letters for each number.
But you can see here where it says minibatches,
it's basically running all five at the same time,
asynchronously.
So it's gonna be much faster if you're doing a lot of data.
And I think a really powerful feature that SAMMO has.
So that's like one of the major reasons
why you would go through this extra effort
to set it up in this way if you're not familiar,
because I find that quite often I'm having
to rewrite my code very early on,
when I'm using the OpenAI API,
then I'm like, oh, I have to go back
and make this asynchronous 'cause it's super slow.
But with SAMMO, it has that kind of out of the box,
and you can set up a data table,
and then you can just boom, smash through things,
which is really nice.
You can also create a data table from Pandas data frames.
And then you have also an ability to optimize
your AB test prompts with SAMMO as well,
which we're gonna get into in the next sections.