So now we created a selector inside of Chrome tools.
We tested that it worked on the website and got all of the data inside the table.
Now we need to move on to NodeJS and get our data inside NodeJS.
So let's first make a folder where we can have our project.
Inside this case, I'm going to call it Scraping Tables with NodeJS.
Just like that.
And then let's go inside the directory.
And then let's initialize NPM so that we can get some NPM packages that we're going to need for our
project.
So npm init dash dash gs.
And then let's go and add the Cheerio and request promise package.
So I'm going to use yarn because yarn is faster than NPM, I think, but you can also use NPM if you
prefer to use NPM.
So yarn add request.
Request promise and then cheerio.
There we go.
Now, all of the packages have been added to the project.
Now let's open it up inside Visual Studio code.
So here I have the project opened up in Visual Studio code.
We have just the node modules folder which has the packages that we added, the cerio and request packages.
And then we have just some empty files for keeping track of the packages that we added.
Let's create the first file now let's call it index.js.
And in here we're going to write out a web scraper.
So let's first import the packages inside the code that we imported with yarn before.
So const request require.
Request promise.
And then.
Cheerio.
Choir.
Cheerio.
Okay, so now let's write out the main function where we have our code inside.
So I'm going to use async because it's a it's a nice clean syntax you get when you have async request
inside NodeJS.
So I'm going to use async function and let's call it main.
And in here we have our main code to write the web scraper inside.
Then I'm executing the function down here in the bottom of the code and run it this way.
So let's first make the request for the website to get the HTML of this page so we can do something
like const result await and we can say request get, and then we type in the page for the the table
that we want to scrape.
In this case https coding with Stefan.
And then we say table example.
Okay, so this just gets the HTML from the page.
It's sort of like curl if you know, curl or wget get.
If you're from the Linux world, it simply just gets the web page.
It's just a Http request we're doing.
So we can't pass the site yet.
We can't select elements from it yet, but that's what we're going to use Cherry-o for.
So Cherry-o is sort of like the jQuery interaction that we did on the page before with the extension.
So we can simply say const and then we assign it to a dollar sign variable just for easy readability.
So we say const cherry-o load and we pass in the HTML string that we get.
In this case the HTML string is inside the result variable that we got from request.
So there we go.
Now we can select elements on the page.
In this case, let's use the selector that we have from our chrome tools and paste it in here.
So we can actually just copy this code that we have inside Chrome tools, the one where we get a line
for each name and just copy that and let's just paste it inside the code here in NodeJS.
And now let's try and run the code inside NodeJS and see if we get the same result as we did in Chrome
Tools with the name on each line.
So I'm going to go into the terminal in Visual Studio code, new terminal here, and we're going to
say.
Node Index.js.
And there we go.
We can see all of the names just like we have inside of Chrome tools.
Okay.
So let's do a quick recap on what this code is doing here.
So first we import the packages request and cheerio request is simply doing a actp request of a given
URL.
It just simply fetches the data.
It doesn't do anything.
But Cheerio is enabling us to select elements and parse the HTML so we can select elements on it.
In this case, we want to get the table data and that's what we can use.
Cheerio to.
It loads the HTML page, parses it, and then we get this jQuery selector as a result, just like we
have inside of Chrome tools.
And we can simply paste in the same code that we have from the Chrome tools and run it inside NodeJS.
Now, the reason why we want to have this inside of NodeJS is because then we can make an API that people
can request from to get scraped results.
We can do it in an automated way instead of pasting in code in chrome tools.
We can save our data inside CSV files or we can save it in a database and so on.
There's a lot more possibilities when we are sitting with the scraping process inside NodeJS rather
than just inside the Chrome browser.
Okay.
So now in the next section, let's move on to how we can put this data inside of the data structure
that we talked before about in the earlier lessons.