SEO eBook Question #1

[Sharon & Roy]

Scenario: You have a new Site with no inbound links at all. You submit it to Google via their Add URL submission Page. What happens to this page. Does it get "crawled/spidered" and "indexed" on the next update (which is monthly right?) or does it depend?

Next Scenario: You have a new Site with just one inbound link. That link's page gets "crawled/spidered" so that means your page did too, right?

Next Scenario: You have a new Site with just one inbound link. That link's page doesn't get "updated" for say 3 months, so that doesn't necessarily mean that the new page had to wait that long does it? In other words, can the new Site be updated before the old linking Site?

(We are assuming that Google currently crawls/spiders about once per month, right? And that some Sites, even though they get crawled/spidered, they may not actually get updated until months later, right? And do all Sites get crawled/spidered at once or is there some sort of rotation that takes place?)

[ihelpyou]

I believe I answered most of those questions here:

http://www.ihelpyouservices.com/forums/t423/sc672e65e64888d9161b0cbf495d33456.html

My opinions only.

Next Scenario: You have a new Site with just one inbound link. That link's page doesn't get "updated" for say 3 months, so that doesn't necessarily mean that the new page had to wait that long does it? In other words, can the new Site be updated before the old linking Site?

I do not understand that question?

[Sharon & Roy]

Originally posted by ihelpyou

I do not understand that question?

Oh, sorry, Doug (and Forum-Mates), but we are of the understanding that Google ...

1) spider's a page and then ...

2) updates that page ...

But they are two separate processes and that they rarely occur at the same time or on the same day. Is that correct?

---

In other words, is it true that just because a page has been spidered by Google, does not mean that the page has been (or if it even will be) updated?

Definitions:

spidered = Means that a page has been found by Google' spider, and that the page's "content" is then placed into a "special database" (separate from the actual database they offer the public at www.google.com)

updated = Means that the "content" from the "special database" is now entered into the public database at www.google.com

---


1) Are the above definitions of what we understand true? (If not can you please explain what is true?)

2) If the above definitions are true, do ALL pages in the "special database" get updated (or do some NOT get updated, between now and the "next" time Google spiders, or at all)?

3) If all (or all that will be updated this time) the pages in the "special database" get updated, does it happen all at ONCE, or over a period of days or maybe even weeks or months?



---

In our research we found that this page (http://www.google.com/search?as_q=beer&num=10&btnG=Google+Search&as_epq=&as_oq=&as_eq=&lr=&as_qdr=m2&as_occt=any&as_dt=i&as_sitesearch=ehomebrew.com&safe=off) hasn't been updated within the last 2 months, but it has been updated within the last 3 months (http://www.google.com/search?as_q=beer&num=10&btnG=Google+Search&as_epq=&as_oq=&as_eq=&lr=&as_qdr=m3&as_occt=any&as_dt=i&as_sitesearch=ehomebrew.com&safe=off)

Research Method Used:

We simply compared that page which has a low PageRank rating (actually the page is spamdexing, is a doorway, and is one of 321 such pages from that domain alone) with ...

The Home Page (http://www.google.com/search?hl=en&imgsafe=off&safe=off&q=women+site%3Aivillage.com&as_qdr=m1&btnG=Google+Search) of one of the most popular Sites on the Internet, which shows that it was just updated within the last 30 days (1 month).

(For Your Information Doug, your page was last updated about 3 months ago, since we know that it hasn't been updated within the last 2 months (http://www.google.com/search?hl=en&imgsafe=off&safe=off&q=search+site%3Afreemoneyservices.com&as_qdr=m2&btnG=Google+Search) but has been in the last 3 months (http://www.google.com/search?hl=en&imgsafe=off&safe=off&q=search+site%3Afreemoneyservices.com&as_qdr=m3&btnG=Google+Search))

---

[ihelpyou]

Which page was updated 3 months ago? My site is spidered and updated once a month.

Okay. You are correct with all you wrote. A page can be spidered and NOT indexed at the end of the month. Once a page is indexed, it is spidered every month and then reindexed. If at the time a page might have lost PageRank, it could fall out of the index.

I am not sure how you can know that another site was updated by Google unless you had access to the spider logs of that site?

[Sharon & Roy]

Originally posted by ihelpyou

Which page was updated 3 months ago?

freemoneyservices.com

Originally posted by ihelpyou

I am not sure how you can know that another site was updated by Google unless you had access to the spider logs of that site?


Doug, as you can see from the examples we listed above, just change the month variant which will be the m3 variable of this URL (http://www.google.com/search?as_q=search&num=10&btnG=Google+Search&as_epq=&as_oq=&as_eq=&lr=&as_qdr=m3&as_occt=any&as_dt=i&as_sitesearch=freemoneyservices.com&safe=off) to either m2 (http://www.google.com/search?as_q=search&num=10&btnG=Google+Search&as_epq=&as_oq=&as_eq=&lr=&as_qdr=m2&as_occt=any&as_dt=i&as_sitesearch=freemoneyservices.com&safe=off) for 2 months or m1 (http://www.google.com/search?as_q=search&num=10&btnG=Google+Search&as_epq=&as_oq=&as_eq=&lr=&as_qdr=m1&as_occt=any&as_dt=i&as_sitesearch=freemoneyservices.com&safe=off) for 1 month.

Since the search for your Site with the settings ...

Return web pages updated in the past 2 months does not show up and ...

Since the search for your Site with the settings ...

Return web pages updated in the past 1 month does not show up and either, but ...

When the search for your Site with the settings ...

Return web pages updated in the past 3 months DOES show up, we can then deductively reason that your Site was last updated over 2 months ago but less than 3 months ago.

---

The above syntax is easily obtained using Google's Advanced Search (http://www.google.com/advanced_search?q=search+site:freemoneyservices.com&hl=en&safe=off&as_qdr=m3) page and then changing the variable as shown above for ...

Date Return web pages updated in the [past 3 months]

---


Then just to be sure that the m1 and m2 variables are valid ones, we checked This URL (http://www.google.com/search?hl=en&imgsafe=off&safe=off&q=women+site%3Aivillage.com&as_qdr=m1&btnG=Google+Search) and verified that they are valid.


We can check the validity of our deductive reasoning here from time to time to be sure, but from this, can we not say that the most popular Sites get updated monthly and the rest of us every 3 or 6 months or maybe even longer, since Google displays a choice for a yearly listing as well?



---

Along the same lines, here is the useful Yahoo syntax to know the DATE when Yahoo listed your Site ...

http://search.yahoo.com/search?p=u:freemoneyservices.com&w=dir&fr=op&o=i&h=c&g=2000&n=100

(Doug, do you remember the date? If you do, was it June 18, 2000)

---

[JuniorHarris]

Are you suggesting the page itself was updated or the Google index?

[Sharon & Roy]

"the page itself"

[ihelpyou]

S and R, I have no idea what you are trying to say.

Googlebot is around once a month, and the site is reindexed once a month.... what am I missing here?

[Sharon & Roy]

Hi Doug,

If you mean by reindexed that it also means updated, then this may help you understand what we are trying to say.

---

Google has introduced the capability to restrict your search to pages that have been updated with a certain time frame. Google date range search uses a drop-down menu to restrict searches for web pages that have been updated in the past three months, six months, or year. Once a date range has been specified for a search, another drop-down menu will appear at the top of the results page, enabling users to manipulate the date range without back-clicking to the advanced search page. Google's date range search feature is available at the advanced search form for both Google and Google Groups.

Source: http://www.searchenginewatch.com/sereport/01/08-google.html

---

Alta Vista has had a date range search for some time now.

If you look here (http://www.altavista.com/sites/search/web?q=search+engine+optimization&kl=XX&r=&dt=tmperiod&d2=7&d0=&d1=&nbq=50&pg=aq&search=Search) you'll see that these 36 pages have just been updated/reindexed by Alta Vista within the last week.

Alta Vista has the following settings to check when your page was lasted updated/reindexed ... They are ...

---
Today
Week
2 Weeks
Month
3 Months
6 Months
Year
Anytime
---

Compared to Google who currently only shows settings for ...

---
Past 3 Months
Past 6 Months
Past Year
Anytime

* But you can just adjust the variable as explained above to reflect ...

Past 1 Month
Past 2 Months
etc., etc., etc.
---

Source: http://www.altavista.com/sites/search/adv/

[ihelpyou]

Okay. But what exactly does that mean?

[Sharon & Roy]

Well, doesn't it mean that not all pages are update every month? And some have been over 3 and 6 months ago since their last update?

[ihelpyou]

lol..... but that is what I do not understand... what you are saying an update is. What is an update exactly?

[JuniorHarris]

Is it possible the update date used by Google advanced search is reviewing the page's last modification date and not the index update date itself?

[ihelpyou]

You may be right JH. My last update was June 5th, I believe. That would make sense then.

[Sharon & Roy]

Is it possible the update date used by Google advanced search is reviewing the page's last modification date and not the index update date itself?

Good point to make, JuniorHarris, so you had us thinking, and we went to test it.

The results do not prove that to be true, so we have to conclude that it is when "THE SEARCH ENGINE" itself last updated the page in their index, No?

Just using deductive reasoning, here, what else could it mean?


--- THE TEST ---

This Page was last updated on The Page's Server on 07/13/01

http://www dressmaker com/orderfor.htm

(Just copy & paste the code below into your browser's URL window and press the "Enter" key on your keyboard to verify this date.)

javascript:alert(document.lastModified)

NOTE: This forum (code) is placing a space between java AND script:alert(document.lastModified)

Please make sure you close that space up before executing the code in your browser or it will not work.

Now 07/13/01 was less that 3 months ago, right? Yet, both ...


Alta Vista
http://www.altavista.com/sites/search/web?q=url%3Awww.dressmaker.com%2Forderfor.htm&kl=XX&r=&dt=tmperiod&d2=3&d0=&d1=&nbq=10&pg=aq&search=Search
Look Here: http://www.altavista.com/sites/search/adv - then enter this ... url: www dressmaker com/orderfor.htm - set to 3 months

... and ...

Google
http://www.google.com/search?hl=en&imgsafe=off&safe=off&q=order+site%3Adressmaker.com&as_qdr=m3&btnG=Google+Search
Look Here: http://www.google.com/advanced_search?q=order+site:dressmaker.com&hl=en&safe=off&as_qdr=m3


Do not display this page as having been updated within the past 3 months.

But,

Google will display results as having been updated within the past year.
Look Here: http://www.google.com/advanced_search?q=order+site:dressmaker.com&hl=en&safe=off&as_qdr=y1

... and ...

So does Alta Vista
http://www.altavista.com/sites/search/web?q=url%3Awww.dressmaker.com%2Forderfor.htm&kl=XX&r=&dt=tmperiod&d2=5&d0=&d1=&nbq=10&pg=aq&search=Search


So with that test it rules out that it is the date of the last update on The Page's Server right?

[ihelpyou]

hmmm, then I have no idea what the Google "update" could mean. Cause if that was the case, my site would be no where to be found if Google has not updated. Also, the cache would not be very current, and it is.

[JuniorHarris]

Well I have to ask the same date question about altavista as well, before I would/could consider it as supporting evidence. It is also important to remember that the Google index is typically a month behind, so an additional question, is this a relative three months from today or the actual date of the index itself?!?

Sorry I don't have the time for extensive research, but I did run a couple of searches on Google. Granted these examples in themselves are not meant to be conclusive, as they are merely a single example in a very complex equation. I searched Google using search engine optimization for only the past three months, and indeed all the top 10 pages reflected a recent date (using the given JavaScript). I searched again using search engine optimization but selected the past year, and in pops a doorway page at number one with a mod date of 10/29/2000. But also remember, since the index is behind, any recent modification dates could reflect updates which occurred after the indexing.

So without extensive research in a more controlled manner, I only have my opinion. But my opinion is that the date is indeed relative from today and reflects those sites modification dates and not that of the index. It just does not make sense from the surfer perspective that there would be any value in when Google found the page(s), but it would be nice to know (and make sense) those sites with recent [updated] content. After all, relevant results should/could include recent as well.

It would be interesting to locate pages in the index which display the date, as the cache page can then be viewed to determine when Googlebot last indexed/cached the page (and usually if it indexes, it also updates). Then find a suitable range of modification dates from these pages for testing and THEN compare the date restricted searches. Using both dates for comparison should add additional insight, as both the [current] modification date AND the index modification date are known.

[ihelpyou]

It would be interesting to locate pages in the index which display the date, as the cache page can then be viewed to determine when Googlebot last indexed/cached the page (and usually if it indexes, it also updates).
That would be a great feature and not too hard to implement. Google should do just that.

[Sharon & Roy]

Return web pages updated in the past 3 months (http://www.google.com/search?as_q=&num=10&btnG=Google+Search&as_epq=How+does+Google+rank+pages&as_oq=&as_eq=&lr=&as_qdr=m3&as_occt=any&as_dt=i&as_sitesearch=google.com&safe=off) <-- This returns 2 pages on the Google.com domain ...

This Page (http://www.google.com/webmasters/) was last updated 07/25/01 - which is EXACTLY 1 month ago

This Page (http://www.google.com/webmasters/4.html) was last updated 06/15/01 - which is NOT in the past 1 month or in the past 2 months but is in the past 3 months

So that is why BOTH pages are returned in the search for pages updated in the past 3 months (http://www.google.com/search?as_q=&num=10&btnG=Google+Search&as_epq=How+does+Google+rank+pages&as_oq=&as_eq=&lr=&as_qdr=m3&as_occt=any&as_dt=i&as_sitesearch=google.com&safe=off) and only 1 page is returned for the search for pages updated in the past 1 month (http://www.google.com/search?as_q=&num=10&btnG=Google+Search&as_epq=How+does+Google+rank+pages&as_oq=&as_eq=&lr=&as_qdr=m1&as_occt=any&as_dt=i&as_sitesearch=google.com&safe=off) AND pages updated in the past 2 months (http://www.google.com/search?as_q=&num=10&btnG=Google+Search&as_epq=How+does+Google+rank+pages&as_oq=&as_eq=&lr=&as_qdr=m2&as_occt=any&as_dt=i&as_sitesearch=google.com&safe=off)

---

Okay, we believe that since tomorrow's date will exceed the in the past 1 month criterion, and that since This Page (http://www.google.com/webmasters/) was last updated 07/25/01 and is still listed today, will no longer be listed in the search results for the 1 month criterion as shown here (http://www.google.com/search?as_q=&num=10&btnG=Google+Search&as_epq=How+does+Google+rank+pages&as_oq=&as_eq=&lr=&as_qdr=m1&as_occt=any&as_dt=i&as_sitesearch=google.com&safe=off) tomorrow.

So let's check this tomorrow to see if this is so. If it is, we can deduct that a month in this criterion is like from 01-01 to 02-01 and that 02-02 starts the 2nd month.

This will still not conclusively prove if the update date is the date that "Google" updated the page into its index or if the update date is the date that the "Web Master of that page" updated the page to the page's Web server.

One step at a time may determine conclusively which it is.

Anyone know who we can ask for sure, like is there an eMail address for Google to find this out?

[ihelpyou]

Yes. A nice to the point email to Google might get a response:

search-quality@google.com

[Sharon & Roy]

Just wanted to add the following information to further TEST the update date.


(Note: Look at the very bottom of the page for the DATE)

As of today, Google's cache page doesn't yet relect the update Doug made on August 11, 2001

It would be interesting to see when Google update's Doug's page?

Doug, does Google "normally" update your page ONCE per month? And if so, what part of the month, do you know?


Google's Cache Page (http://www.google.com/search?num=30&hl=en&imgsafe=off&newwindow=1&safe=off&q=cache%3Afreemoneyservices.com&btnG=Google+Search) shows ... Page last updated on Tuesday, June 05, 2001

Doug's Actual Page (http://www.freemoneyservices.com/) shows ... Page last updated on Saturday, August 11, 2001

[ihelpyou]

The August 11 update will show up in the Google cache the last week of September I would assume.

[Sharon & Roy]

eMail Message Originally Sent to Google by Sharon & Roy

Greetings,

Would you please be so kind as to explain what the ...

"Date section" of the Advanced Web Search Page means?

http://www.google.com/advanced_search

It says ...
Date Return web pages updated in the past 3 months

What does the word "updated" refer to?

1) The date that "Google" last updated that page in the Index Database

.... or ...

2) The date that the "Webmaster" last updated that page to "their" own Web Server.

------

Also, could you please let us know if EVERY Page already in your Index Database, that returns results to the public, is updated on a monthly basis? Or are some pages updated by you on a less frequently basis, like maybe, every 2 months or 3 months, etc.

Thank you for you help, as it is most appreciated.

(Reply From Google)

Date sent: Mon, 27 Aug 2001 10:32:01
From: "GoogleTech" <googletech@google.com>
Subject: RE:{Google#187-208}Question Regarding Your Advanced Web Search

Thank you for writing Google.

These are pages that are updated by their webmasters within the given time period. Google refreshes its entire index every few weeks.

Regards,
The Google Team

[JuniorHarris]

S&R thanks for posting the follow-up....

So it looks as if the date criteria is indeed the modification of the page itself, and according to Google they refresh the [entire] index every few weeks. Great stuff and good to know!~ :)

[Sharon & Roy]

We're just posting that Google has not yet reindexed/updated Doug's Page that he updated on August 11, 2001.

It still shows the June 05, 2001 updated page as the cache page from the last check above on 08-26-01, almost 4 weeks ago and 6 weeks since the page was actually updated.

Google's Cache Page (http://www.google.com/search?num=30&hl=en&imgsafe=off&newwindow=1&safe=off&q=cache%3Afreemoneyservices.com&btnG=Google+Search) shows ... Page last updated on Tuesday, June 05, 2001

[Sharon & Roy]

Google still has not updated Doug's Page yet. It's now been 7 weeks.

Google's Cache Page shows ... Page last updated on Tuesday, June 05, 2001 still.

[JuniorHarris]

Google appears to be behind in updating any site this month!~

[ihelpyou]

The last update Google shows for the site was the crawl the first part of August. At that time, my page had not been changed since June 5. Because of this, the updated index in Google at the end of August still showed my June 5 page. Now, Google has not updated the end of September yet, so the cache still shows the june 5 date. This is all very correct and very logical.