Thu, Feb 20, 2020 06:37 PM - Updated

Wanted: Web Scraping Help @stanford.edu

Reply to: Use the form at the right to send messages to this user.
Date: Wed, Feb 15, 2017 12:48 PM

I'm a graduate student looking to analyze message board posts from two websites and I need help scraping the content from these message boards.

These sites contain thousands of message threads and millions of messages so I need to 1) get all the urls for each thread so I can pick a random sample of threads to analyze and 2) scrape the contents of the random sample of threads (i.e. thread title, username, time stamp, body text for every message posted on the thread) and store them in a csv file.

I'm hoping to get this done as quickly as possible but I'm not sure how long a job like this might take. If you think you could do this can you send me a message with a brief description of your experience with web scraping, how long you think this would take you to write, and how much you would like to be paid. I can then give you more information about the websites and we can discuss the project further in person or over the phone.

Note: Your help with this project would not include authorship on any papers.

please do not message this poster about other commercial services