Alex Rivera | Logout

Simple Screen Scraping using jQuery

Asked 2011-04-14T18:31:44.037
47

I have been playing with the idea of using a simple screen-scraper using jQuery and I am wondering if the following is possible.

I have simple HTML page and am making an attempt (if this is possible) to grab the contents of all of the list items from another page, like so:

Main Page:

<!-- jQuery -->
<script type='text/javascript'>
$(document).ready(function(){
$.getJSON("[URL to other page]",
  function(data){

    //Iterate through the <li> inside of the URL's data
    $.each(data.items, function(item){
      $("<li/>").value().appendTo("#data");
    });

  });
});
</script>

<!-- HTML -->
<html>
    <body>
       <div id='data'></div>
    </body>
</html>

Other Page:

//Html
<body>
    <p><b>Items to Scrape</b></p>   
    <ul>
        <li>I want to scrape what is here</li>
        <li>and what is here</li>
        <li>and here as well</li>
        <li>and append it in the main page</li>
    </ul>
</body>

So, is it possible using jQuery to pull all of the list item contents from an external page and append them inside of a div?

Edit
Report

1 Answer

6

If this is for the same domain then no problem - the jQuery solution is good.

But otherwise you can't access content from an arbitrary website because this is considered a security risk. See same origin policy.

There are of course server side workarounds such as a web proxy or CORS headers. Of if you're lucky they will support jsonp.

But if you want a client side solution to work with an arbitrary website and web browser then you are out of luck. There is a proposal to relax this policy, but this won't effect current web browsers.

answered 2011-04-15T02:24:14.930

Your Answer