KnowledgeHub
Questions
Tags
Users
Search
Alex Rivera
|
Logout
Edit Question
Title
Body
In the Scrapy docs , there is the following example to illustrate how to use an authenticated session in Scrapy: class LoginSpider(BaseSpider): name = 'example.com' start_urls = ['http://www.example.com/users/login.php'] def parse(self, response): return [FormRequest.from_response(response, formdata={'username': 'john', 'password': 'secret'}, callback=self.after_login)] def after_login(self, response): # check login succeed before going on if "authentication failed" in response.body: self.log("Login failed", level=log.ERROR) return # continue scraping with authenticated session... I've got that working, and it's fine. But my question is : What do you have to do to continue scraping with authenticated session , as they say in the last line's comment?
Tags (comma-separated)
Save Edits
Cancel