Luka: Hello, everyone, and welcome back.
In this video,
we are going to talk about ChatGPT
and how to use it to implement the same algorithm for Doom
that we are going to use in the course,
convolutional deep Q-learning.
So in the next couple of minutes,
we are going to walk through the way that I actually pointed
and asked ChatGPT for us to get our model.
So let me help you understand how I did it
and why I did certain prompts in the way I did.
So the initial prompt was pretty well-crafted from my end.
I said:
"You are a Senior Machine Learning Expert
"with years of experience working on projects
"applying machine learning to simulations and robotics."
This is the first part,
setting the stage, giving them a role.
So basically, the vector space of the whole ChatGPT
will be prompted more on machine learning topics.
Now I want to bring into my context, Doom.
And actually, how to reach that point,
I said, "Okay, I hired you to help me
"with implementing a Convolutional Deep Q-Learning algorithm
"for the Doom environment.
"The goal is to have
"a fully-functional reinforcement learning algorithm
"that can play Doom, a game environment.
"I have implemented classes
"for experience_replay and memory and image processing.
"I will provide you that
"so you can incorporate that into the final code.
This is really important, which most people misses.
You need to have some way of telling him,
or telling ChatGPT,
that you already have something
and you want to use that code in the final implementation.
So if I didn't provide this as a context,
it will generate some code,
some implementation of convolutional deep Q-learning
that will not consider using experience_replay
and our image processing function,
but we want that.
So, then I continue in saying:
"Your goal is to implement the full solution using Python,
"Torch for model,
"and optimize the code to be executed in Google Colab
"so that students going through the code
"can execute those with not so strong computers."
This is basically telling the ChatGPT
what is the format of the final solution
that you'd like to have.
And then, finally, "ask clarifying questions if needed"
will provide it with more context.
And, "Eh, if you don't know,
"if you don't have enough context, please ask me."
And finally, asked, "Got it?"
I wanted to confirm that it got.
This is a clever way of allowing it to reflect on this
and not strictly go to generate next prompts, next answers.
If you don't do this,
if you don't ask "Got it?" or "Clear?"
or "Do you understand?" or something like that,
it will completely ignore some of the parts
and start generating.
Since you have the limit of how many tokens
you have in your questions and in the answer,
you would like to break it so you can get a refresh
on the amount of tokens that you can generate.
So the first answer is:
"I understood the task.
"I'm going to do exactly what I'm going to ask.
"But before I proceed, I have a few questions."
And this is where this "if you have any questions, ask me",
it asked me pretty good questions.
First off, do you have a custom environment,
or do you use something that is already there?
So OpenAI, VizDoom, and stuff like that.
Then we move to more like,
are there any specific performance metrics
that you'd like me to follow?
Are there requirements on the model itself?
What versions of Python and PyTorch?
And finally,
are there any specific constraints on the deadline?
This is from mostly business plans that it was trained on,
but for it, like, it doesn't really,
like this is not relevant for it, but okay.
I provided:
"Yes, I will use OpenAI Gym, and here is how to load it."
This is from the course.
I said, okay, follow average reward,
no specific requirements, be creative as long as it works,
use latest models or versions of the libraries,
and what is the deadline.
I just wrote "ASAP".
Additionally, here are these classes I mentioned.
So I just pasted the classes that we have in the course.
So experience replay, everything,
and finally, replay memory, image processing,
and that's it.
And with this, I build the context,
and now it started generating.
So it generated the code.
It basically started by pre-processing the image,
then created the deep Q-network,
which is pretty similar to what we have.
Then, it started by initializing that,
and then training code.
Everything so far worked, and then it breaks here.
If you want to continue to generate something longer,
I just wrote "continue".
And it said, "Continue? Yeah, certainly."
It continues generating,
and bam, we have the full code.
Amazing.
It stated how to use it, frequency,
what to change, and stuff like that.
It helped me with this as well.
And let me bring you to my Google Colab.
So I started by pasting the code here,
pasted our pre-processing function.
I'll jump to changes in this pre-processing function
in a second.
Then, I pasted this experience replay,
some of the installation from my end.
We'll explain this in a second.
I played with his deep Q-network and training code.
So there are definitely ways to improve this.
Especially if you have local environment, it will work.
If you are using this in Google Colab, it might break.
So let me bring you to that.
So I did it, and I got an error.
I said, "Hey, my initial code that it provided did not work.
I just pasted the code, this error message,
and saying, "I can't resize an images."
And if you take a look at our code from the course,
it requires to resize the image with this.
But the newer version of SciPy is actually removing this.
So it said, "Oh, yeah,
"it appears that this may have been removed."
"Here is the new version with Pillow.
And actually, it didn't require that.
It basically rewrote the whole pre-processing image class
with Pillow instead of SciPy.
So I used it, and it's working actually.
And then I said, "Okay,
"I have a problem with Environment using Google Colab.
"How to solve it? This is the error."
And it said, "Oh, yeah, apologize.
"It seems that this particular environment is not working.
"Here is what you do.
"You basically install and do this."
I got an error, another error.
And this error actually indicated
that rendering is not found in this particular version.
And it said, "Okay, install this version."
I did. Another error.
And it said, "Oh, yeah, yeah,
"Google Colab does not support rendering, OpenGL libraries,"
which means, for us, that we need to do,
instead of Google Colab,
something else for visualization purposes.
And it basically gave me a step-by-step,
leading on what to do, how to change that,
and stuff like that.
And if you do this locally, you will solve the problem.
But this is basically now teaching you
how to use properly ChatGPT.
Like, right here, you saw that I got some errors,
I prompted with those errors, and so on.
And now, you have,
like, if you scroll back,
you have full-functioning code with a deep Q-network,
with a training code,
and you can go and ask it to generate a test code,
and it will.
Amazing, right?
So that's it.
This is how to use ChatGPT to solve the deep Q,
or convolutional deep Q-network for Doom.
Thanks. Enjoy the course!