A brief recap inside of the chops with headers.
That map we're returning a promise and await promise.
Dot All is waiting for all of these promises to resolve.
And again, this returns a complete array of all the chops with their description.
We use return await.
And this is going to return the job description.
All the jobs with their description.
Let's try and console log all of the data, see what we're getting.
And it's going to take more time.
We can see that there's a undefined on this array.
So I'm clearly doing something wrong here.
And I think I need to return also the object every time, like so.
And I will try and run it again and.
This time we can see an array full of objects containing the description and all of the other properties
that we scraped before.
Now let's try and get the compensation for each job also.
So we go again and select the element and we can see we have this just a span element without any class
or ID, so but it's inside this attribute group class.
And so let's try and select the element with the class attribute group instead.
And so if we can get the compensation out from that.
So I could use, for example, the select the class attribute group and then select the first child
of this element.
Let's try and do that.
So attribute group and text.
We can see there's compensation and the employment type, but we just want the compensation.
So let's select the first child.
So and then we get the compensation $23 per hour.
That's exactly what we want.
I think I want to clean this up a bit and maybe remove the compensation in front and just have the 23
slash hours instead.
Let's make a variable first.
Let's try it out.
Compensation and replace compensation.
With just an empty string.
And this way our data is a bit more clean, I think.
So let's do that inside of a NodeJS project as well.
First, let's get the raw compensation data.
I think I will make this into a variable first.
Paste that in.
And then let's get the more clean version of the text without the compensation in front where you use
the replace to replace the compensation with a empty string like so.
And then we have the data nicely formatted on chopped compensation.
Let's try and see how the code looks.
And we can see we have the compensation here, hourly rate.
Some of the some of the jobs don't have a compensation listed.
But I think it's an interesting property to have.
At least then let's put it inside a try catch clause to catch any errors we might get.
I will just output the error to console log for now or console dot error.
And that's it.
Now we got a good base of building a scraper, at least for Craigslist.
And there's probably a lot of other sites you can use these techniques techniques on.
And let's try and see how many jobs we got.
We should have 120 like there should be on the page.
So you can see 120 jobs.
So that's exactly the amount we need.
Now you can keep building on this project.
You can make it pay.
Scrape the sites or pages on this as well.
We are only scraping one site or one page right now.
There is multiple pages, but you can also build it up so it scrapes all of the pages there is of the
jobs.
You can also build a rest API based on this scraper.
So if you go for to a rest endpoint, then bam, you get all of the data like we get in the console
here and you could present it inside of a client or just a basic HTML view.
It's up to you.
You could also just save all of the data to a file if that's what your client or what you want, like
a CSV file, and that's up to you.