C1. Crawling the Web with Python and Scrapy (Eduardo Freitas, 2019)

1. Course Overview: 1. Course Overview 00:00:00 2. Extracting Data from the Web - Core Concepts: 1. Version Check 00:01:55 2. Introduction Overview and Prerequisites 00:02:10 3. Concepts 00:04:04 4. Legal or Illegal 00:06:49 5. Legal Consequences 00:09:30 6. General Advice 00:12:53 7. Why Scrapy 00:15:41 8. Demo - Extracting Data without Scrapy 00:19:52 9. Summary 00:27:41 3. Scaffolding and Running Your First Scrapy Web Crawler Project: 1. Introduction and Overview 00:29:01 2. Introduction to Scrapy 00:30:05 3. Scrapy Architecture 00:33:44 4. Beautiful Soup 00:36:28 5. Demo - Creating and Scaffolding a New Scrapy Project 00:39:17 6. Summary 00:44:15 4. Achieving Common Spider Behaviors Using Built-in Classes: 01. Introduction and Overview 00:45:35 02. Spiders Overview 00:46:39 03. Types of Scrapy Spiders 00:48:18 04. 00:50:03 05. CrawlSpider 00:51:29 06. XMLFeedSpider 00:53:01 07. CSVFeedSpider 00:54:27 08. SitemapSpider 00:55:36 09. Demo - Implementing a 00:57:27 10. Demo - Implementing a CrawlSpider 01:05:14 11. Summary 01:08:57 5. Influencing Scrapy Crawling: 1. Introduction and Overview 01:09:48 2. Allow and Deny Rules 01:10:48 3. Processors 01:11:30 4. Item Loaders 01:14:29 5. Item Pipelines 01:15:31 6. Demo - Implementing a Scraping Pipeline 01:17:40 7. Summary 01:24:18 6. Scrapy Outcome and Data Export: 1. Introduction and Overview 01:25:10 2. Feed Exporter 01:26:00 3. Demo - Using an Exporter to Save Data 01:27:19 4. Summary 01:29:05