KnowledgeHub
Questions
Tags
Users
Search
Alex Rivera
|
Logout
Edit Question
Title
Body
I'm trying to scrape a JavaScript enables page using BS and Selenium. I have the following code so far. It still doesn't somehow detect the JavaScript (and returns a null value). In this case I'm trying to scrape the Facebook comments in the bottom. (Inspect element shows the class as postText) Thanks for the help! from selenium import webdriver from selenium.common.exceptions import NoSuchElementException from selenium.webdriver.common.keys import Keys import BeautifulSoup browser = webdriver.Firefox() browser.get('http://techcrunch.com/2012/05/15/facebook-lightbox/') html_source = browser.page_source browser.quit() soup = BeautifulSoup.BeautifulSoup(html_source) comments = soup("div", {"class":"postText"}) print comments
Tags (comma-separated)
Save Edits
Cancel