Edit Question - KnowledgeHub

Title Body

I get a SocketTimeoutException when I try to parse a lot of HTML documents using Jsoup. For example, I got a list of links : <a href="www.domain.com/url1.html">link1</a> <a href="www.domain.com/url2.html">link2</a> <a href="www.domain.com/url3.html">link3</a> <a href="www.domain.com/url4.html">link4</a> For each link, I parse the document linked to the URL (from the href attribute) to get other pieces of information in those pages. So I can imagine that it takes lot of time, but how to shut off this exception Here is the whole stack trace: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(Unknown Source) at java.io.BufferedInputStream.fill(Unknown Source) at java.io.BufferedInputStream.read1(Unknown Source) at java.io.BufferedInputStream.read(Unknown Source) at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source) at sun.net.www.http.HttpClient.parseHTTP(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) at java.net.HttpURLConnection.getResponseCode(Unknown Source) at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:381) at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:364) at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:143) at org.jsoup.helper.HttpConnection.get(HttpConnection.java:132) at app.ForumCrawler.crawl(ForumCrawler.java:50) at Main.main(Main.java:15)

Tags (comma-separated) Cancel