So I got a question about how to deploy a puppeteer web scraper inside a Heroku dyno, which turns out
to be a little more complicated than I originally thought it would be.
Now, we already covered a section where I show you how to deploy a scraper on Heroku or Google Cloud.
And this works fine for a regular web scraper with cherry and request.
But if we start using Puppeteer, then you need to add the so-called Buildpack to your Heroku dyno.
This is beginning to sound terribly complicated already, but don't worry.
I'm going to show you step by step how to deploy a puppeteer web scraper to Heroku.
Okay.
Step one is to get my GitHub repository with the minimal puppeteer project and go ahead and clone this
git repository.
So let's go inside the console and say git clone and put in the URL.
And then let's open up this project inside Visual Studio code.
So now that we have the project open, let me just give you a brief intro to how the project is.
So we have this very simple file here, Index.js, which has the property dependency.
We set the headers to be true and we pass in this no sandbox argument.
It's very important that you have the no sandbox argument or else it is not going to work.
And then we go to the example.com page.
We simply get the text from the page and then we do a console output to make sure that our puppeteer
browser is working as it should be inside of Heroku.
So with that being said, let's go ahead inside of Heroku and create a new app that we can deploy this
app to.
Now inside Heroku, let's go ahead and make a new app.
This is step two.
So we say here I will call mine Puppeteer Udemy.
And let's say reaching is Europe because that's where I'm located.
Now we need to use the Heroku CLI in order to add this repository.
So we go and take the existing git repository and we simply copy this command and go inside of the repository
inside the terminal and we add it to our Heroku app.
Just like that.
Now we set the remote to be Heroku.
Now we need to publish or push this app so we can say Git push, Heroku master.
Now, it is important that you do this push first before we add the buildpacks.
Because I found out by testing myself that if I add the Buildpack first for puppeteer and then push
up to local, I simply don't get any output in my console, so it doesn't seem like it's running at
all.
So it's important we first do this push where we should be getting an error when we run it.
So let's go inside a new terminal here and let's say Heroku logs.
Tail.
And then a and we call it puppeteer.
Udemy, which is our Heroku app name.
Okay, so now it's saying deploy to Heroku and.
Let's see inside the console now, we see lots of errors here.
So these errors come up because we don't have the build pack added.
But remember that you need to do this push first before we add the build pack or else you're not going
to get any console output at all.
It's weird.
It doesn't make sense, but I've tested this multiple times and this is what seems to happen every time.
If I add the build pack right away before pushing first.
So with that now let's go and add the Buildpack and then let's do a little commit and push again.
So let's add the puppeteer Buildpack here.
It's called this one.
Join the whatever.
And then we say Dash A and have our Heroku app name Udemy.
Just like that.
Okay, so now I've added the buildpack and then we can run git push heroku master.
I think I need to make a little commit before I can push.
Yeah, I need to make a little commit.
So let's do that.
I just add a.
Space like that.
And let's go ahead.
And commit that.
With Buildpack.
And then we can say git push, Heroku master.
Okay, so now it is building the project again.
This time with with the build pack added.
So it's going to take a little more time because it is quite a big pack that we're adding.
But now it should be running and giving us the designs inside of the console.
So we know for sure that now the puppeteer scraper inside Heroku is running.
Okay, so now our building has finished and our deploy to Heroku.
So if we go ahead and check inside the other terminal where we had the Heroku logs running, we can
now see the whole of the site that we decided to scrape.
So just like that, that's how we deploy a puppeteer web scraper to Heroku.