So if you take a closer look at the login request inside of Chrome or inside of Postman, well, we
can see there is lots of cookies being set.
And one of these cookies is called the Csrf cookie name.
And this value in here is exactly the same as the one we have in the form that we send for the login.
Now, this csrf cookie is already been set as we go into the front page of this site.
So the question now is how do we get this csrf cookie from request and use it inside of the form Now,
because we don't have this form data, then the request is going to say 4 or 3 forbidden.
Let me show you how we can do that.
So first, we need to make a separate cookie jar so that we can get the cookies out from request.
Because as it is now, we can't say something like request dot, get cookies.
There's no method on that.
Like that on request, we have to make a separate cookie jar.
So const cookie jar.
Equals.
Actually, let me put it down here.
Request Char.
And when you call it like that, it just makes a cookie char for you to use inside of a request.
So then we have to say again, request equals request defaults and then say char equals cookie char.
And we have to change this to be a let instead of const.
And let's remove these defaults here because we set them down here instead.
Okay, so now we have a separate cookie jar where we can do something like cookie jar, get cookie string,
get cookies, set cookies and so on.
Now let's try and let's try and see what happens after we visit the first page so we can say cookie
jar, get cookie string.
And with this method we have to put in the site that we want to get all our cookies that has been set
for this site.
So we say get cookie string put in the URL.
And let's say console.log just to see what we have in here.
Let's comment this out and let's try and run Node Index.js and see what happens in here.
So here we have.
We have a CSF cookie name with the value here and this is the value we need to pass inside of the form.
We can see here the csrf test name inside of the form.
We need to use that value inside there.
So we need to do some string splitting here in order to get this exact cookie value or cookie key and
value.
So we can say const, let's take this one and put it inside the cookie string.
Valuable.
Very valuable.
I can't even say that.
So then we can say const, split it.
By CSR Cookie name.
And we can say cookie string and split it by.
CSR of cooking.
There's really no easier way of doing this when you're just using the default request cookie jar if
you want to do more serious cookie manipulation and stuff like that.
You can use an external cookie jar, but for now I'm just using the default one we have inside of NodeJS
request.
And trust me, it's going to work fine.
Now let's try and run, say run and debug just to test out and see what we have here and have a breakpoint
here at the bottom.
Let's see.
We have cookie string.
Then we have split it by cookie.
Csrf.
Cookie name.
So this one, the first one we have here is the value we have right after we have Csrf cookie name.
You can see here, this is our cookie name for AC three and then we split it by that we get the number
two item in this split array to be the CSR of cookie name value.
So now we need to split it again.
Const c sf.
Let's see what the form value name was.
CSF test name, let's call it that.
And we say split it by cookie name and we say split again.
Let's see what we got after this.
Come on.
Oh, is it making an error?
Maybe I have an error here.
A cookie name split is not a list because I have to choose the second one of the split array.
So we have this one.
And the second item in the array is the cookie value.
Then we have this one where we split by the semicolon just to get the value only, and this is the first
item in the array.
So let's just go through it again.
Here we have the cookie string.
It looks like this.
We want to get only this key and value.
So we we split it by this name.
And then we get this here where the second item in the array is right after the splitted value that
we took.
So it's this one.
That's the value we want to get.
And now to get rid of these semicolon and space and the other values, I say split again with the semicolon
and we just get the, um, we actually need to get the.
This the first item in this array.
So then we can say that zero after here.
And then we have the exact value we are looking for.
Which is this one.
Nine, eight, nine.
A58.
We can see that matches with our raw cookie string in here.
Okay.
So that's not something I usually do when I have to automate the login of a website.
Only in this case where they have something with csrf and they pass it into the form that's not I don't
think that's sometimes forms do that with the values, but it's not always you have to do these things.
Sometimes you can just get away with enabling cookies and you can just authenticate away.
But this one is a little more tricky.
But I think we got it now.
Let's see.
So now we pass in the CSF in the form.
Let's try and run Node Index.js.
Let's also do a console log of the login result and see what we got.
Login result.
And here we can see it says Success, True success page dashboard.
That means we are now logged in and now the server has saved in the session that this cookie is authenticated
and we can now scrape the rest of the pages that is behind the authentication.
So we can go into this URL which is matching preferences and we can just say something like const matches,
await request get and put it in the URL.
Notice we don't have to put anything else in because we already have the cookie set inside of the inside
of request.
And now let's try and save this page to see if we get the right page from the request.
Let's say.
Um, let's say const FS to save the file inside of NodeJS.
F.
S and we can say f s right file sync and we could put in matches dot HTML with the matches we get from
NodeJS and see what we have in here.
Node Index.js.
Now, let's check it out inside of Chrome.
So open with Google Chrome.
And indeed we can see we now have all of the internship matches inside of Google Chrome that we downloaded
from NodeJS request.
And then we can basically just go ahead and do any sort of regular web scraping or we can get all of
the items in this website.
And I think I've already shown you that inside of the course.
So I'm not going to show you with this example.
This was more to show you how we can log in and authenticate ourselves in a website that uses something
like cookies and session authentication and a little bit of csrf sessions or tokens.
So I hope you got a lot out of this.
And if you have any questions or suggestions, please let me know.
And yeah, I'll see you around.