Okay, everyone in this section or lecture, we're just going to build a really simple scraper and we're
going to save the scraper data to a MongoDB database.
Specifically, we're going to use the free.
MongoDB database that's called Mlab.
So let's take a look at how this works.
So let's build a folder first for our scraper.
Let's call it the Reddit scraper.
And then inside of this Reddit scraper, as usual, we are going to run npm init dash dash js.
And then we're going to open the folder inside of Visual Studio code.
So.
So now I have the basically empty folder with the package.json file inside.
And the first thing we need to do is we are going to add Cheerio request and request promise to build
our super simple scraper.
And then we need to add Mongoose.
So let me just try and enlarge this text so you guys can see what I'm actually getting.
So.
Cheerio.
Request.
Request promise Mongoose.
And of course I need to type in yarn add and you can also type NPM install if you prefer to use NPM.
So now I added the packages and see them here.
So let's write our index.js file.
And we type in.
A mongoose.
And.
Cheerio.
And request.
Just like that.
And then we will write out our main sort of function to scrape inside.
And we're going to scrape Reddit, but it's going to be a super simple scraper.
So in case you don't know Reddit, why is it so slow?
Okay, so we're just going to get the headlines of each of these articles and um, yeah, that's going
to be it for now.
And then we're going to save this data to our MongoDB database.
So let's see.
So each of these headlines here is a H2 tag, so it's easy for us to select it.
So let's go ahead and try to do that.
So const HTML await request get.
Read it.
Then we have.
Cheerio.
So we load in the HTML so we can pass it and then we simply select all the titles.
By doing a H2 selection and then we can go through each of these titles.
And do something with them.
Uh.
Mac keyboards.
Um.
So let's just type console.
Now I want to save the title.
So we have the title.
Which is the element.
Text.
And then I just want to see if we are getting the title of all of these Reddit articles.
So that is it.
Then we call the main function here.
Let's try and run it.
Okay, so now we get all of the titles of these different Reddit articles.
Um, so you can see, for example, this one with Vladimir Putin in it.
You see the same article in here.
So now we get that that is a super simple scraper that we're able to build here.
But, um, now we need to be able to save it to our MongoDB DB database, and that's going to happen
in the next lecture.