So many developers believe that scraping a URL’s content is easy — just a simple
curl and you’re done, right? Nope. CAPTCHAs, rate limitations, origin-specific limitations — they make getting content off the web a nightmare. Luckily we now have services like zenscrape which do the heavy lifting for you — the perfect middle man between you and the desired content!
- You can sign up for free
- Request a webpage from any location around the world
- Bypass CAPTCHAs and other listing limits
- Satisfies important use cases: downloading reviews, feedback, pricing, and other data
- Generate code for Node.js, PHP, cURL, and other utilities to use zenscrape’s API
- Use the visual Web Scraper tool to visually select the data you want
After signing up for free, you’re ready to take advantage of zenscrape’s easy to use API. Armed with an API key, the easiest use of zenscrape is:
Aside from providing the obvious URL, there are other parameters you can pass with your request:
location: The country location code from which the request should be made
render: Use a headless browser to render the page
premium: Allows you to use residential addresses to make scraper detection more difficult
These options can be appended to the API request as follows:
curl "https://app.zenscrape.com/api/v1/get ?apikey=MY_API_KEY &url=https://davidwalsh.name &location=au render=1"
Another really nice feature that zenscrape provides is a code snippet builder that allows you to save time:
Web Scraper Tool
APIs are great for using zenscrape but visual tools are sometimes more helpful. zenscrape provides a visual point and click tool to select, store, and export data from URLs of your choosing:
You can schedule recurring scrapes so you always have the most up to date information.
Imagine always knowing competitor pricing and being able to beat them at all times!
zenscrape is a really cool service for those that want to scrape content reliably. Building tools to try to deal with the hassle of CAPTCHAs and other limitations isn’t worth the time. Have a look at zenscrape and th eir other tool, zenserp!