So now we looked inside of the network tab in Chrome Developer Tools.
We figured out which request in this log had the login request and we also saw the request headers and
the form data for this request to log in.
Now let's see if we can replicate this request inside of Postman instead of inside of the browser.
So Postman is sort of the next step before we move on to using request inside of NodeJS.
It makes it easier for us to debug how to actually pass this log in request and get ourselves logged
in.
So let me show you how we do this.
First, let's go and take the request URL from the login request.
We have this post request.
Let's copy that.
And paste it in here.
Now make sure to put the request method.
This one to be the same as we have in here.
So make sure to say it as post, which is the same as we had inside chrome dev tab.
And now we need to pass in some kind of data or form data.
To do that.
Go into the body tab here and usually the forms data is in CSV form, Urlencoded that is basically as
a web form or HTML form.
There's also form data in raw binary.
Most of the forms is in this.
Some, for example, for APIs can also be in this data, but it's not so often that you see it.
So make sure to select this one if you want to be really sure.
What kind of data is being passed to this endpoint, you can check it inside of the Chrome developer
tools.
So if you go inside of the let's see, here we have.
The request headers.
So this is the request that you send to the server.
And here we have the response headers.
So that's the response back from the server.
And if we look inside of the request headers, we can see a content type of application form URL encoded.
So we can, in good confidence, set this data to be the same.
Now we need to pass in some data.
And the first data I usually try is the most obvious one, which is the email and the password.
I'm going to show you sort of my process as I try to figure out how to actually get this login working.
I already know how we can get it working, but I'm going to show you how my process is for getting it
working.
So by that I mean basically we copy this form data here.
For example, input email handle.
And we pass it in here as a key in the body tab and then write in the value we have also.
So this is my email.
And then we also need a password here.
It's called input password.
And notice that these values can be different based basically on based on what side you are on.
So maybe Amazon just calls it email or they just call it password.
It depends on how they made this site up.
But for Craigslist, they just call it input, email handle and input password.
So copy password or input password key as well, and then copy the value as well we have here.
Paste it in.
And now let's try and do the request and see what response we get from Craigslist.
So you can see here we get a 404 not found, which means that our oops, which means that our request
was not successful.
It says Page not found.
So something went wrong.
We didn't give Craigslist all the data that they wanted.
They don't they are not just happy with just the email and password, it seems.
So then what I would do if this doesn't work at first, then I would try other values, maybe inside
of the form data, maybe we can try the browser info data we have here.
Let's try and paste it in inside the value then say browser.
Info.
Just like that.
And then I would try again to see does it work now?
And we can see it still doesn't work.
It still says four four not found.
Then I could try other ones like this one.
It's called TI.
Maybe that's something they really want to have with the form.
So I tried that one also.
And it still doesn't work.
And now maybe I try all of these data as well.
Also this one.
Maybe I try them all step by step and see is there any point where I suddenly can be enabled to log
in?
Can see it still doesn't work.
And then if I try all of the form data and try to basically copy all of that inside of Postman and it
still doesn't work.
Then the next step maybe is to look at your request headers.
Make sure that you set your request headers inside postman the same as we have in here.
So I can see there's a content type.
We already have that because we selected this one here.
So Postman is already setting that for us.
Then I can see there's one with cookie.
There's a cookie being set when we are logged into Craigslist just without logging in.
I mean, just as you visit the site, Craigslist is going to set a cookie.
Maybe it could be that.
But let's save that for later.
There's also an accept header.
Maybe we can try that.
So I'm going to copy the value here.
And then inside the headers tab, I will write accept.
And I paste in the value.
Let's see if that works now.
And it still doesn't work.
So I just keep on trying all of these things.
I mean, some things are more sensible or reasonable to make a difference.
It's something you learn by practice basically on different sites.
Maybe the origin site, the origin header information could work.
So let's try that.
Origin and then set the value in here from the Chrome developer tab.
And then let's try again.
It still says four for one not found.
So that's pretty much the process as I tried to find out how to how can I log in?
User agent is also a really good thing.
You could try to see if that works.
So let's try and set the user agent.
Then paste that in.
Well, it still doesn't work.
And then finally, at some point, you arrive at something that just works.
So let's try.
Refer.
Refer means.
Well, what side have been are we getting referred from?
I mean, you are on this side and then you get a refer from that side and you send that to this endpoint
so it knows where does this request come from?
And let's try again.
And now it says 200.
Okay, so now I've actually logged in.
And let's see, what does the site say?
It says here, Home of Stephan Hulshof.
That's my name.
Which means that I have logged in.
It also says log out here.
So now I have actually logged in using Postman.
You can see in here in the cookies also, we have a session cookie being set with this value here and
it has a expiry date which is really far out in the future, 2068.
So if you are logged in to Craigslist, you are really logged in for a long time.
But that's how it works.
We use simply try a lot of things like the form data and the headers.
Just try a lot of things until it simply works, until you find out what is actually needed to make
to make yourself logged in.
And then once I finally find something that actually makes me logged in, I try to disable other values.
I've set until now, so I don't have to set a lot of values when I'm using a request.
For example, let's try and just take off all of these extra form data we tried before.
Let's see if our request still works and I can see it still works.
I still get a 200 okay response.
So I am still logged in.
Now let's try and remove some of the header values we have here.
Let's try and remove the exact origin and user agent.
And now I can see I still get a 200.
Okay.
Now let's try and remove the referrer again.
And here we can see I get a 404 not found.
So the referrer header was apparently the key to getting this request work.
And this can really be different between what sort of.
So here you can see it says four four not found.
Sometimes it does say 4 or 4 not found.
I found out when I'm debugging this and you just have to try again, it seems, and then it logging
is logging you in.
But what I was trying to say is that.
Finding out what is important when you log in and stuff like this is very different from site to site
based on how they programmed or created the login functionality.
They make different steps necessary, so to say different values.
Sometimes you have to pass, you have to put all of the form data in as it is from Google Chrome.
Sometimes you can get away with just the email and password.
Sometimes you don't have to set any headers at all.
Sometimes you have to set lots of headers and weird hitters also.
So you have to look inside of your chrome developer developer tab, look at all the values and try them
one by one inside of Postman until you get something that just clicks and works.
And then once you get something that clicks and works, you tick off all of the boxes until you find
the minimum required to make it work, which is going to make your life a lot easier as we in the next
section make the request inside of NodeJS.
So I'll see you in the next section where we are going to use request in NodeJS.