The left side is my editor, Visual Studio Code, which I use to edit the code.
And on the right side, we have the browser where I navigate around in the page that I want to scrape.
So this enables me to look around in the elements, select elements, find out how to select them,
and then write the code.
On the other side.
So first let's get the request module.
Say require.
Request promise.
And this enables us to download pages, which we can then use chario to select different elements.
Just like jQuery.
Okay, so let's create our main function for the web scraper, which is going to be an async function.
And if you're not familiar with async, it's a feature in the new JavaScript Es7, which enables us
to use keywords such as Await.
And I'll explain that later.
Now we'll make the call for the function down here.
So it enables us to use features such as Await so we can say await for every asynchronous call we make.
So let me show you in practice how this works.
We will get the HTML from the page first.
So we say request dot get and this downloads basically any URL we paste into request so we get the quicktest
page here.
With all the jobs, I will make a URL variable.
Going to paste this URL down in the description as well.
So if you want to get it.
You look in the description or the resources for the Udemy lecture.
And then we can use the await keyword to wait for requests to finish this request and then we can do
something with the result.
Like the old fashioned way to do it would be using something like request dot, get URL and then then
and then do something with the result.
And maybe we have a catch clause here.
But I think that using the await keyword in the new version of JavaScript is a lot cleaner than having
the chain then clauses and so on.
So that was just a short intro to await, in case you're not used to that.
Then I'm also going to paste in a try catch clause so we catch any errors we have in the code and we
can paste it out to our console.
So now let's try and just see in the console what request is actually getting from the Craigslist page
so you can see it in practice.
We'll write console log result and then we'll write Node Index.js, which has our web scraper.
And then you can see inside of the console that it's actually just the basic HTML page you're getting
in a string format.
Now we can't really do so much with this page as it is right now.
That's what we need Cherry-o for.