Edit Question - KnowledgeHub

Title Body

I need to do some web scraping. After playing around with different web testing framework, of which most where either too slow (Selenium) or too buggy for my needs (env.js), I decided that zombie.js looks most promising, as it uses a solid set of libraries for HTML parsing and DOM manipulation. However, it seems to me like it doesn't even support basic event-based Javascript code like in the following web page: <html> <head> <title>test</title> <script type="text/javascript"> console.log("test script executing..."); console.log("registering callback for event DOMContentLoaded on " + document); document.addEventListener('DOMContentLoaded', function(){ console.log("DOMContentLoaded triggered"); }, false); function loaded() { console.log("onload triggered"); } </script> </head> <body onload="loaded();"> <h1>Test</h1> </body> </html> I then decided to trigger those events manually like this: zombie = require("zombie"); zombie.visit("http://localhost:4567/", { debug: true }, function (err, browser, status) { doc = browser.document; console.log("firing DOMContentLoaded on " + doc); browser.fire("DOMContentLoaded", doc, function (err, browser, status) { body = browser.querySelector("body"); console.log("firing load on " + body); browser.fire("load", body, function (err, browser, status) { console.log(browser.html()); }); }); }); Which works for this particular test page. My problem is a more general one, though: I want to be able to scrape more complex, AJAX-based sites like a friends list on Facebook (something like http://www.facebook.com/profile.ph

Tags (comma-separated) Cancel