Recommendations for a spidering tool to use with Lucene or Solr?
Asked 2008-11-12T00:50:36.407
16
What is a good crawler (spider) to use against HTML and XML documents (local or web-based) and that works well in the Lucene / Solr solution space? Could be Java-based but does not have to be.