arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

Download found Microsoft office files?

rated by 0 users
Answered (Verified) This post has 2 verified answers | 2 Replies | 2 Followers

Top 150 Contributor
2 Posts
masanek posted on Fri, Jul 20 2012 12:24 PM

Hi,

As a few other post mention I am working on a crawling project and AN looks awesome. My project requires crawling for certain file types (most MS office) and downloading them to disk. Can AN do this right out of the box? If so can you estimate how much work it might take? It seems AN is a complete solution.

 

Thank you,

Michael

Answered (Verified) Verified Answer

Top 150 Contributor
2 Posts
Verified by masanek

Never mind. I went through the demo plugin and did a little tweak and it shows promise.

Top 10 Contributor
1,905 Posts
Verified by arachnode.net

AN can, yes.  Thanks for the good word.

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Page 1 of 1 (3 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC