scrapy

Package — ScrapingPython 3.8+Intermediate

Full-featured web crawling and scraping framework

Quick Info

Documentation
Official Docs
Python Version
3.8+
Dependencies
Twisted, lxml, cssselect, parsel, w3lib, itemadapter, protego
Install
pip install scrapy

Learn by Difficulty

Easy2 examples
Examples|Playground
Intermediate1 example
Examples|Playground
Advanced1 example
Examples|Playground
Expert1 example
Examples|Playground

Quick Example

python
# Install: pip install scrapy
import scrapy

# Basic scrapy usage
print(f"Using scrapy")
# See documentation for detailed examples

scrapy is a third-party package. Full-featured web crawling and scraping framework. Install with: pip install scrapy

Try in Playground

Tags

packageweb-scrapingcrawlerframeworkpipeline

Related Items

beautifulsoup4
Package — Scraping
HTML/XML parser for extracting data from web pages
selenium
Package — Scraping
Browser automation for testing and scraping dynamic JS pages
requests
Package — HTTP
Simple HTTP library for GET, POST, PUT, DELETE requests
parsel
Package — Scraping
Scrapy's selector library standalone; CSS and XPath selectors on HTML/XML