virus beware...

[quickhollywood]

well it's one of the 3 DMOZ message boards I singed up with, or occurred from DMOZ it self when I submitted my url for addition. basically it's a stupid web hack virus, but the only ones that knows of my site is from these boards until my urls are submitted to the search engine of course. But as of recently I started getting some hammering from a couple of IPs, which I'm sure were affected by a virus to start with. Good thing I don't keep any of the Windows IIS virtual pathes available through IIS, and have latest patches. Just a warning to all those before posting, if you're using IIS have all the latest patches and get rid of all the admin virtual directories before letting people know your url.

[Advisor]

I don't understand what you're trying to tell us. Could you please elaborate?

Jill

[ihelpyou]

hey quick, your new engine there, is it just the dmoz directory? You mention your spider will crawl the net looking for sites, so is it also a spidered search engine?

[quickhollywood]

Jill - sorry for the jiberish :) but basically after I became a member of the msg boards here and had my url as my signature, I got some traffic from some hacked computers with a virus... no big deal for me, my system is ok... just trying to warn others.

ihelpyou, yes it is a spidered engine too... I had to stop it last night to test out some data backups and such just in case :) the search should work really really late tonight or tomorrow morning, unless I run into coding problems (which happens way too often)

[ihelpyou]

Well shoot then, how do we add url's to be spidered? I hope you have plans to be an algo based search engine with a spider that crawls the web. We need more of them to compete with the big boys, namely Google right now.

[quickhollywood]

Well initially I was building the whole site from scratch, meaning the directory structures and everything... then got fed up after 2 weeks, so just adopted the ODP. At current present time, I'm set up to just use ODP's data, so url submission would be through them, and I just update from that. I'm working on the numbers right now to see which is viable. Currently 4-8 hours to parse the ODP data of structures and catalog is a bit much, but do-able. I'm trying to make a faster parser so I can try doing at least a bi-monthly update. Spidering is currently pretty slow, but I'm reworking my spider code too... I'll finish the parser code in 2 weeks for dealing with new additions, so I can just respider a site every few weeks rather than every update.

Not too sure of the absolute final concepts as of yet... it's basically a work in progress... it wasn't even going to be a search engine =)