An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does scale? | Download the latest release

Building a distributable application

rated by 0 users
Answered (Verified) This post has 1 verified answer | 1 Reply | 1 Follower

posted on Thu, Sep 2 2010 9:53 AM


A couple of questions...

I'm looking to build a simple desktop-based web research program with Arachnode.

It will distributed to Windows based every-day users.

1.  Can Arachnode be installed with the Express versions of MSSql?

2. What type of licencing requirements would we need if we were to sell the commercial version of our software that uses your spider technology as it's  main engine?

3. If we were to offer a web based solution instead (1 single instance), what  licencing requirements would we need then?

Answered (Verified) Verified Answer

Top 10 Contributor
1,905 Posts
Verified by


1.) Functionally, it may work, if you were to remove the CLR functions.  Practically, you won't be able to store a lot of data due to the 4GB limit, and there are processor limitations to Express.  (Not recommended, and I haven't even tried because the index padding used to faciliate rapid data ingestion for the initial crawling experience already consumes a large chunk of what would be your working base in Express).  I am working on a no-SQL version ATM.  Likely won't take more than a week or two to remove the SQL crawling dependency.  If you are serious, and are able to move volumes of your product, I will work with you to make the no-sql changes you need.

2.) A fee per license.  The amount depends on what you want for your efforts.  Open to discussion.

3.) AN is licenced per machine, not per user, and requires you to communicate your server names to me.  So, one machine can serve as many users as that server will support.

EDIT: - the license terms here supersede this forum post.  1/18/2017


For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Page 1 of 1 (2 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, LLC