Hello, everyone.
In this section we're going to look at how to deploy a web scraper onto Heroku.
Now, how to make it run automatically every ten minutes or any other given interval you may want.
So we have a site here.
We have Craigslist with the cars.
So I went just to look in the cars and trucks section and we're going to scrape this side or this page,
the first page every ten minutes on Heroku.
The the the site is being updated pretty frequently.
Pretty frequently.
So that's why it's pretty interesting to do this kind of periodic scraping on it.
Um, so Heroku is a place for you to deploy your NodeJS apps, so you basically deploy it to the cloud
and you can either have a API up and running on Heroku or you can also just have a simple scraping script
like we will have that you can run in every ten minutes or every day or every hour and so on.
And then it's going to save all of our results onto a MongoDB database we have running on Mlab.
And of course, you can choose any other provider you want to, but it's I'm using mlab and it's going
to save all of the results here.
And if there's any new results every time it runs, it's going to save a new a new object onto the database.
So, so far since I've been running it for a couple of hours, it's been scraping 310 cars.
And remind you, there's 120 cars on this first page.
So it's just getting new cars every ten minutes or so.
So now let's take a look into the code itself and we'll talk a little more about the code, how it works,
its principles we had before in the course.
But I'm just going to show it now so you understand what it's doing.