Eddie Peloke's Blog

Wednesday, August 03, 2005

Beanboy hard at work....


Last Thursday night (July 28) I sent Beanboy out to crawl 30 Java sites. As of today, and 8.9 gig of data collected, he' still crawling...hasn't even stopped for a water break. I can definitely see that if this is something I want to keep updated, I am going to need to get a few machines each crawling a subset of the list and then merging the results. Hopefully beanboy will finish soon as I am running out of space on my old linux machine.

0 Comments:

Post a Comment

<< Home