-: Hey guys, Eden here, I hope you joined the course
and I would really appreciate if you can write me a review.
This really, really helps.
Now I want to take a recap of this entire course
and what we did so far and how to go next.
So the main idea for this course
was to give you an introduction
and an elaborate introduction
for LLM application development.
We covered two super important patterns
when it comes to developing LLM applications.
The first one is using agents
and leveraging the reasoning capabilities of the LLM,
creating agents which can execute non-deterministic actions,
so this was the first part.
And the second part was dedicated
for retrieval augmentation, for utilizing vector stores
and semantic search and embeddings
in order to chat over our proprietary data
and leveraging this capability.
So every LLM application is either using this pattern
or that pattern or simply making a call to the LLM directly,
and you have now the tools
in order to create those applications.
So now let's talk about what's next
and what do I suggest you're doing in your learning journey
of LLM application development.
So if you've been following along this course,
you probably noticed that developing an LLM application
is not that simple.
Coming up with this perfect prompt,
which would get the best response for the LLM,
requires a lot of work and investing a lot of time.
And what happens if the LLM suddenly changes
and now the prompt is not working as expected?
And what if we want to consider other LLMs in other models
which are faster, maybe cheaper, or maybe more secure?
So we would need to take this prompt
and adjust it to the other LLM.
So prompt management is a really important issue
when it comes to LLM application development.
We also need to monitor our LLMs.
So we want to see how fast are we getting our response,
what's the latency?
How much does each request cost us
and how much are we going to pay to the LLM vendors?
We also have debugging, where something goes wrong,
and we want to understand why the LLM
is not returning us the correct response
and the response that we want.
And that can be a challenge,
especially when we're dealing with agents
and of course, evaluation.
Even if we use the LLM,
how can we know that the response
that we get from the LLM is good?
So we want to have tools
that help us automate all those processes,
because if we're going to evaluate
the LLM response manually,
this would take a lot of time and it doesn't scale.
So all of those problems are being bundled together
into a new evolving field called LLMOps, LLM operations.
And there are several of popular tools
which can help us solve
and to perform better LLM operations.
One famous one is LangSmith by LangChain.
And LangSmith is a unified platform
that enables developers
to build production-grade LLM application.
And it helps with debugging, testing, evaluating,
and monitoring all of the applications,
and allowing us developers
to have quick and efficient development life cycles.
So LangSmith currently is not open source,
so this is something you should consider,
but if you want open source alternative,
you can use Pezzo
and Pezzo really helps with prompt management and tracing
and monitoring our LLM operations.
So we should also check out Pezzo, in my opinion.
I want to talk about LLM security for a moment.
So in this course,
we haven't really talked about this subject.
So there's a difference
when we develop an LLM application locally,
between deploying it into production,
because once it's deployed in production
and we have real customers using it,
then we need to make sure that our LLM application
is threat safe.
And that's a pretty challenging thing to do
because large language models and usage of them
bring a lot of new attack vectors.
So things like prompt injection and agents accessing data
that they shouldn't access.
So those things are real world problems
that we need to take account of.
So recently, LangChain made a huge repository change
where all the code that was not safe
and held new vulnerabilities
was moved to this experimental directory.
So overall, security is something important in my opinion,
and maybe I'm just mentioning it
because I come from a security background,
but I think it's important to know and to explore,
especially when we are dealing with LLMs.
Now let's talk about resources and where do I suggest you go
to find new information
when it comes to generative AI application developments,
and LangChain, LangSmith, other frameworks, all of that?
So really the two things I highly suggest you do,
and in my opinion, it would cover most of it,
is one to follow the LangChain blogs,
so every week they release new blog,
and new ideas, new implementations about generative AI
and gen AI application development.
And I think it's an excellent source of information
for you to keep going.
And the other thing I highly suggest you joining Twitter
and reading about generative AI in there,
there are a lot of people that post things
about researchers and about new applications,
new use cases, and new ways to optimize things.
So I highly recommend you joining Twitter
and follow it there because all the news,
all of that are being directly streamed into Twitter.
So that's pretty much it.
I do update this course frequently.
So when something new and important comes out,
I do update this course.
And I hope you enjoyed this course,
and if you do want to support me,
I'd appreciate a Udemy review.
This really helps me to spread around the world,
and thank you so much.