In the introduction of this section, I told that we are trying to get job listings from Craigslist,
but what data are we trying to scrape exactly?
Well, here I am on a Craigslist site.
It is the San Francisco Bay Craigslist, to be exact.
So I'm going to go into the job section and then go to software QA, DBA.
And in here we can see a listing of jobs.
So you can see they have a date that the job was posted.
They also have a title of the job.
And then they have a neighborhood in the parenthesis.
Then we can also click on the job listing and then we get to a job description.
In here.
We have all the job description here, and then we also have a compensation.
So it's not a lot.
You're getting paid for this job, but they obviously haven't filled it out.
Some job listings do not have a neighborhood filled out and some do not have compensation filled out.
But we're going to scrape it from wherever we can.
So if we're logging, we get that data with us.
If not, then at least we have the job description, the job title, and the date it was posted.
So this is the data we're trying to scrape from Craigslist.
We're going to get the data was posted, the job title, the neighborhood, and then we get the URL
of the job description and we get the content of all the job description and a compensation.
So that's a brief overview of the data we're trying to scrape in the next section.
I'm going to show you how our data structure would look like for these scraping results.