-: Okay, what is Meta LLaMA 2?
So it's a new open source model by Meta,
formerly known as Facebook,
and it's the first really viable alternative
to OpenAI that's open source.
You know, this is the Stable Diffusion model kind of moment,
but for LLMs.
So when Stable Diffusion came out
in the image generation space,
it changed a lot about what was possible,
and how many, you know, businesses you could build
on top of these AI tools,
so really exciting to see this.
It's a transformer model, just like GPT.
A lot of the work in this space has been built
on top of Google's work on Attention is All You Need
that introduced the transformer model,
as well as OpenAI's work with GPT-2, which was open source,
and the results are pretty good, right?
Initially, it was created by Meta, and then open-sourced,
but they didn't open source the weights,
only researchers got those, and then somebody leaked them.
With LLaMA 2, Meta actually made the decision
to fully open source it,
so there is possibility of a commercial license, really,
for the first time, and it's really good.
You can actually fine-tune it, so that's,
you can't currently fine-tune anything in the OpenAI API,
that's not available right now,
'cause they deprecated GPT-3,
but they say that GPT-4 and GPT-3.5 fine-tuning
is coming soon, but you've at their mercy,
whereas with LLaMA 2, people are fine-tuning it every day,
like university students are doing it,
people are just doing it in Jupyter Notebooks,
it's pretty nuts.
and one way you can try LLaMA 2
without knowing how to code is you can go to chat.nbox.ai,
so they very generously host this for free,
and you can try it out.
This is how I've been playing around with it.
The really interesting thing here is that
it needs a lot more prompt engineering, in my experience,
but here, this is a pretty robust prompt
that works well, even on GPT 3.5,
but for LLaMA 2, I tend to get a lot of hallucination,
and it keeps talking, like, it doesn't stop
at one product name, so yeah,
you can play around with it and see how it goes.
It's also possible to download it and use it locally
if you have a GPU on your computer,
like a Mac M2, or like, a gaming PC.
In terms of performance, it is, I would say,
on par with 3.5, it's not quite there,
and specifically, it's the fine-tuned version,
so the Vicuna is pretty decent.
So it has two of the top slots in the model rankings
by LMSYS on Hugging Face.
It's not quite there yet.
It is definitely one of the best open source models.
MPT-30B-Chat model is also really good,
but it depends on the ratings and the use cases,
and the really cool thing here is that often,
like a fine-tuned version of LLaMA 2
will beat GPT-4 at specific tasks
if it's been fine-tuned for that task,
so this is really great for specific use cases
rather than more of a general purpose AI.
Because it's open source, the main benefit
is it's free, right?
Like, apart from the cost of compute,
you don't have to pay any credits.
You can run it on your local computer,
you can run it on your own servers,
and you can build your own UX around it,
that's completely, you know, up to you.
You can also inspect a lot more in terms
of what's going on in the model.
The main use cases, why would you need to use LLaMA 2
when it's objectively worse than GPT-4?
I would say, one is privacy or data protection.
Many in the enterprise can't use OpenAI,
or like, their companies have banned them from doing it.
They don't want sensitive data going to the API.
OpenAI, to be honest, like they said,
that they keep the API data private.
Their startups, hard to know.
I think Microsoft is actually pushing in this space,
and offering GPT through Azure,
which maybe is a little bit more private,
and they might have some self-hosted options available,
but even in that case, you might not wanna fully trust that.
So if you're building your own enterprise version
of ChatGPT,
you might want to build on LLaMA 2.
Also, fine-tuning, if you have less than 200 examples,
typically, prompt engineering just beats fine-tuning
for any given task,
but once you get more than 200 examples,
so once you start to get more data from your users
of what's good, what's bad for a specific task,
then it starts to become worth exploring fine-tuning,
you know, that's only really possible
on LLaMA 2 at this date.
You can also build a business on LLaMA 2.
Because it's open source, you can run it on your own server,
you can have all the code yourself,
you're not subject to any limits, right?
You don't have, you can't be shut down by OpenAI
because your users put some spam requests in there,
or you can't run up a huge bill in terms of your credits,
apart from just the cost of running your own servers.
It's worth checking out.
I would say, I'm cautiously optimistic.
I really want open source to work,
because in the image generation space,
I'm a heavy user of Stable Diffusion,
and I want there to be the same choice
in the large language model space as well,
so, really excited for this.