arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

MongoDB and AN crawler

rated by 0 users
Answered (Verified) This post has 1 verified answer | 1 Reply | 2 Followers

Top 25 Contributor
25 Posts
Dinesh posted on Fri, Jun 7 2013 7:55 AM

Mike,

Is AN crawler compatible with MongoDB or any other BigData. Can you just through some idea on this, if it is compatible how to work with MongoDB/BigData?

--

Dinesh.

Answered (Verified) Verified Answer

Top 10 Contributor
1,905 Posts
Verified by arachnode.net

AN is written for MS-SQL Server.

The ArachnodeDAO is the central point of data access, so, in theory, you could replace the ArachnodeDAO DB calls with whatever you wanted.

AN doesn't natively communicate with MondoDB.

[EDIT: I refactored the entire AN solution to be able to work abstractly with any class and any DAO.  There is the start of a MongoDB DAO.  MySQL is also in process.] 

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Page 1 of 1 (2 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC