This is the sample code we used in our webinar to illustrate:
- Basic scraping with the API (basic.js);
- Extended basic scraping to illustrate large volume, muli-server scraping (basicWithRedis.js);
- Scraping with Structured Data Endpoints, to eliminate the need to extract and transform data from HTML (sde.js);
- Scraping using our async service, to elimite the need to manage bulk submission and concurrency (async.js).
To install, simply clone this repository, open a command prompt at the install directory and run npm install.
Each sample can be run individually from the command prompt with, for example:
node basic.js