Hm. Google very active? My maiden voyage into server logs.

[dragonlady7]

Forgive my ignorance, but I'm somewhat new to this. I'm in the process of redesigning a site, and so have been paying little attention to the site my predecessor created which is still online and active until I can finish the replacement. Today I got my first peek at the access logs from the webserver, and while it was interesting, it was also 2000 pages long and only 4 days of traffic. So I poked through it, scanning for anything that looked interesting (I don't have an analysis tool of any kind, so my scrolling finger got plenty tired).
I spotted Googlebot a whole bunch of times between the 8th and the 13th. It seems to have crawled my predecessor's entire site twice in four days. Let me mention that the site hasn't been updated in about eight months, and our webserver goes on the fritz frequently because the admin is a bit of an incompetent, but the site retains a PageRank of 5 somehow. (Lots of content, several directory listings, I guess.)
I also spotted Inktomi's Slurp, and a Wisenut Bot. Also, Grub, and something called AmphibiBot, which I traced to Amphibi.com... And Voila indexed me too, which seems to be some kind of a French search engine with terrible English translation.
So. I just wanted to ask...
Is Googlebot usually that active? How about the others? Or have I just missed the last time Google's going to come by for the next three months or something, and now my update (with much better keywords, more up-to-date text, fewer broken links, etc .) will go unnoticed and unindexed for ages?
I'm also trying to figure out if there are many incoming links to various internal pages, as I'm planning on replacing the old site with the new one, and the structure is different now. Does anyone have a suggestion for how I should go about checking for incoming links? :nerd: I combed through the logs until my eyes hurt and found only one; anyone else who came directly to an internal page came through Google, Yahoo, MSN, or in one case each Altavista or AOL. So for the most part I should be OK, I think...

So is this one of those there-are-no-answers things, because Google's changing so much and is of course always in flux, or am I right in assuming this is a period of heightened activity for our spidery friend and I should not assume he'll be by this often in the future (once I'm ready for him)? (It's funny, in my head I've anthropomorphized google so much that I always felt happy when I recognized Googlebot in my logs, like I was seeing a friend. Obviously, I need a vacation.)

I suppose I should also ask if it's good or bad that I'm starting off with a PR of 5. Will my total site update help or hurt my PR? The links will be there still, if I'm smart enough to find where they all go and retain those pages... and there's nearly twice as much content now and no broken links...

OK, that's enough questions for now. I'm getting sleepy and incoherent. Happy Friday the 13th, by the way, everybody. :scattered

[ihelpyou]

Oh no, Google should be back through again as long as you have links coming into the site. It sounds like you do so I would not worry about it. Just get the new site ready as soon as possible. If you changed file names on internal pages just make sure you have a custom 404 error page made so googlebot AND visitors can find other real pages of your site if they come across an old file extension from the old site.

[dragonlady7]

Thanks. :-)
Well, the new site is done but my senior co-worker has insisted she has to approve every page, but she doesn't have time to, so for 2 weeks now I've been sitting on the new website... Waiting for her to have time to look at it... which she won't. In that time, our sales representatives have attended a major conference and given out our URL to hundreds of people, and our server logs indicate a lot of them have come to see.... the old site.
I could've had it launched in time for this, but nooo....
So I'll try to put the heat on this woman to make her approve the damn thing by telling her Google's going to go away at the end of the month. Think I could tell her that? Maybe she'll believe it...
Sigh.
Ah well. I had figured out how to do the custom 404 thing, but I've forgotten it now that I've spent 2 weeks waiting for this woman, so I'm going to have to learn it again. I was thinking of putting up a server-level redirect page on those internal pages that there are links to from outside, as well... But that sounds tricky. :/
I just want this thing uploaded so I can figure out whether my SEO has been useful at all!
Bollocks.

[ihelpyou]

Sure, you can put up a server-side permanent 301 redirect as well.

It's funny companies have zero clue. That woman is a bozo and I don't even know her. She should let you do your job and upload the damn thing. SHE is hurting the company she works for.

[AskMeNoQuestions]

(assuming your old site is adldata.com)

Apache has a lot of useful features for redirects - if your new site has a similar organisation to the current one, as far as folder/filenames goes, it's possible to write URL rewriting rules to redirect to the relevant new page. The details vary a lot, although a good starting point for info is at http://httpd.apache.org/docs/misc/rewriteguide.html.

Also, your server has PHP (albeit an older version), and Perl, both of which could allow you to create a database to map Old Page --> New Page, so that search engines (and users who have bookmarked your pages) can find the new content. It's not always the easiest stuff to get the hang of, but it's well worth the effort!

[dragonlady7]

Yep, we've got an Apache server and are running PHP. There is another employee who knows a bit more about PHP so I'm going to lean on him to help with the redirects.
My co-worker is a bit of a twit; we all have to live with these things. My boyfriend and I are in the process of designing a fabulous website so that we can use all the cool stuff we learned at our respective jobs that our employers were too dumb to let us use (he's a programmer and is involved in some cool single-source publishing stuff). So our personal site will be optimized to the teeth and will have everything done right (I'll link to it and will actually be able to be proud of it)-- just so I can see what it's like to actually do the right thing in any given situation. :rolleyes:
So apparently I'm learning a whole lot about the way the world works, in this job. It's nice to have a steady job and no higher worries, but having no say, even over the things you're the most qualified to speak about, is a real drag.