Hi mike
Does AN.net respect following metatag and skip indexing the pages containing this tag?
Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4
Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 <meta name="robots" content="noindex, nofollow">
Or is there such metatag which would help skipping pages from being indexed?
Thanks,
You would modify the parsing logic in DiscoveryManager.cs.
For best service when you require assistance:
Skype: arachnodedotnet
oops!
Don't know what happened there. I was trying to write
<meta name="robots" content="noindex, nofollow">
You know... I never implemented this.
OK
We have tried implementing noindex logic in ManageLuceneDotNetIndexes.cs and it works.
For nofollow, where we don't need to follow the links on that page, we are still investigating. Can you direct us with some solution?
Thanks
Great, thanks mike!
We will try this and let you know how it goes.