Tried crawling Google Docs using my credentials. Get following error:
System.Net.WebException was caught Message=The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. Source=System StackTrace: at System.Net.HttpWebRequest.GetResponse() at Arachnode.Next.Crawler`1.CrawlThread(Object o) in E:\dev331c\ad\ediscovery\Connectors\WebCrawler\arachnode\trunk\Next\Crawler.cs:line 466 InnerException: System.Security.Authentication.AuthenticationException Message=The remote certificate is invalid according to the validation procedure. Source=System StackTrace: at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result) at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size) at System.Net.ConnectStream.WriteHeaders(Boolean async) InnerException:
I am setting the credentials at around line 437 of Crawler.cs with the following:
httpWebRequest.Credentials = new NetworkCredential("username", "password", ""); crawlRequest.HttpWebRequest = httpWebRequest;
http://stackoverflow.com/questions/536352/webclient-https-issues
Investigating this.
For best service when you require assistance:
Skype: arachnodedotnet
Which AbsoluteUri?
If you have a gmail account, just go to your own google docs uri and supply your own credentials. It will be something along the lines of
https://docs.google.com/?tab=mo&authuser=0#home
Yes, this fixes it.
I am not at a place where I can check in, however.
Quick update: I did check this in...