ihelpyou, Inc.
small business center
Toll Free:  (888) 795-3154
  or (740) 244-9313
   Talk to Doug Heil!

What Is Spider Friendly and What Is Not

Submitted on:  04/18/2007

If your website doesn't comply with the basic rules of search engine friendly web-design, then all your attempts at achieving any success in the engines will be a waste of time: no copywriting will ever help you, and nor will any number of inbound links.

Unfortunately, the importance of building websites so they're search-engine friendly is still widely underestimated; too many webmasters still think that a few meta-tags and a search engine submission campaign is all they need to be successful. A lot of sites are created using content management systems that give the impression that they're specially programmed to make sure the site will never be visible to the engines' bots! Forum scripts (even those released by the most well known vendors) need a lot of hacking work to comply with the simplest SEO standards; shopping carts aren't much better.

And yet those standards and rules, once understood, are really simple.

Internal navigation links

All the internal navigation links should be coded as plain HTML links (examples: <a href="page1.htm"> text</a> or <a href="http://www.sitedomainname.com/page2.php">text</a>). Links coded using JavaScript in any way, shape or form, are not followed by the search engine spiders. The same is true for links embedded in Flash objects. If the site has JavaScript-based links or Flash links, it should have an alternative navigation bar in plain HTML (or a complete alternative HTML version of the whole site, for pure Flash sites). Linked images can be followed by search engine spiders, but again, the code should have that plain HTML "href" part in it.

All internal links pointing to the home page of the site should link to "/" or "http://www.sitedomainname.com/". Any sort of "index.html", "default.asp" or "main.jsp" in the URLs of those internal links introduce unnecessary problems like the duplicate content problem and split page's authority. It's been reiterated and discussed thousands of times all over the SEO forum circuit. And yet, all widely known vendors of forum scripts make this same mistake over and over again, from version to version. I have to wonder why.

301 and 302 redirects

We all know that the 301 ("Moved Permanently") redirect is much more search engine friendly than 302 ("Found"). Actually, using 302 improperly is the best way to ruin your success in search engines; cases when it should actually be used are exceptionally rare. In spite of this, most web servers are set up so that 302 is the default redirect; if you need a 301, you have to specially state it, and web developers often forget about it.

Besides, 301 itself can be your good friend if used properly, but produce a lot of harm if misused. Example: some CMSs are built so that the home page ("/") gets 301-redirected to a page like "main.htm" every time you try accessing it. It won't kill your site in the engines but it will add a lot of unnecessary complications and should be avoided. If your CMS is built this way, my advice would be to phone your vendor and yell (politely) at their customer support person until she contacts the developers and they fix the issue. If the CMS is developed by your in-house programmers, then yell at them (been there, done that, and from my personal experience I can tell you that they can fix it easily; if they tell you they can't, they are just being lazy).

The worst example I can think of regarding the improper use of 301 is 301-redirecting your main domain to a subdomain on a web designer's test domain that has been temporarily used to rebuild your website. If your designer has done this to you, you need to demand that the temporary subdomain is dropped ASAP and the content is transferred back to your main domain - or that you want your money back! Subdomains are never treated as serious entries by the search engines, and all the authority you have gained over the years will be quickly lost if you allow the situation to remain like this.

Believe it or not, but I have really seen it done - and people who did it couldn't understand why I was so exasperated with them after noticing it. They thought they had found the best solution.

Dynamic URLS, cookies and sessions

Dynamic URLs are not necessarily bad, though static URLs are, of course, better. The "no more than three parameters in the query" rule is widely known and in most cases strictly followed by web developers. But there are two other rules, no less important, but often neglected, which would be "no "id" letters in parameter names" and "no session ID parameters for guest visitors".

The naming convention of dynamic parameters is the easiest thing to fix; the only thing required here is to know and remember about the rule. Unfortunately, a lot of people have never heard about it; those who have usually forget about it or just don't take it seriously. I would like to emphasise this: the dynamic parameters shouldn't have the "id" letters in them to avoid looking to the search engines like session identifiers, which they hate.

To actually turn off the random session ID parameters off for guests and the search engine spiders while still having them turned on for logged-in users is a much more complicated programming task. But it has to be done, because the engines hate session IDs. The reason behind it is simple: since the session IDs are assigned every time the user agent comes to the website, they create lots of different pseudo-URLs for the same page. The engines would see those pages as different entries with the duplicate content that would fill their indices with tons of unnecessary information and make them sort all the duplicates out. To avoid this, they often ignore websites that show them session IDs, or don't go further than the home page.

Another way to scare a spider away is to require that the user-agent supports cookies - and refuse to serve any content unless cookies are enabled. Since the engines' bots don't support cookies, they will just go away.

A few other things to remember

There are a few other things you need to keep in mind when building a search engine friendly site. These are:

  • frames are both SE-unfriendly and out of fashion;
  • if you can reduce the size of your HTML code but keep the look and feel of your page, do so;
  • ideally, each page should have unique title, meta description and meta keywords tags;
  • always keep only one version of your URL, either www or non-www; the other one should be 301-redirected to it;
  • valid markup may not bring you higher rankings, but it will improve your credibility and at the same time ensure that the spiders won't have problems crawling your pages;
  • using client-side meta-refresh redirects is a bad idea;
  • if a site is going to be big, consider using folders as a structure element from the start;
  • spiders can't read text written in graphics.

That's all I wanted to mention in this article. The quality website building theme is a large one, and I'm planning on returning to it over and over again in the future. But the information above should be enough for you to build a basically search engine friendly site.



About the Author:

Name:  Irina Ponomareva

Irina Ponomareva has been a practising SEO consultant since 2004. She joined Viscomp, a Russian web design agency, in February 2008.

Find out more at http://www.viscomp.ru/


Comments

Stumbled over your great article - our brand new site www.ourinnersource.com (has an Oscommerce engine) is a mess of dead end non-SEO friendly links. Since we are now doing SEO - can you tell me if this plugin would work http://addons.oscommerce.com/info/5080 and what your recommendations are? best, Mark

By :  Mark McGinnis



Post a Comment:

You can post a comment of maximum 1024 characters
Name:
Email:
Comments:
Enter Verificartion Code:
 

Other Articles by the Author:


Other Articles in this Category: