Shopping Cart Optimization

[pageoneresults]

Good evening everyone! Well, we've crossed another barrier in our quest for the ultimate cart for small and medium businesses. I got involved with this a little over two years ago and it has slowly progressed to a respectable cart interface.

A few years ago, Junior Harris and I participated heavily in another SEO forum. Junior was clearly the guru when it came to the optimization of dynamically generated URL's.

I was intrigued by the concept of making a dynamic web site spider friendly. So much so, that I started putting a little pressure on the genius behind the cart.

It wasn't until he launched his own store when he realized the benefit of what I was telling him. This kid, and I do say kid because he is only 21 or 22, is awesome. He actually figured it all out and when you look at the store below, you'll see what I mean.

We've spidered the links that appear in your address window when you browse the cart. Those used to be full of ?'s and &'s and who knows what else! The spider now has a clear path to the content of the departments and the products within. This is an .asp cart powered by a Sequel database. It's got some really cool features. I'm going to show you a few stores that utilize this cart now. The main one, is the only one that has the fully optimized cart.

Main Site
Goped Super Store (http://www.gopedsuperstore.com)

Beta Site
Eagle Malls - Patriotic Promotional Products (http://www.eaglemalls.com)

The beta site will show you some of the custom pricing features that have been completed in the past few months. You can specify different price points based on sizing, volume discounts, etc. You can add any number of custom attributes for each product.

The beta site has no graphic element yet. It is mainly there to test the functionality of the cart.

P.S. The Goped site has not been spidered yet by Google with the newly parsed URL's. There is another store that uses the cart and had a couple of the features in place for a while, and that one holds the number one position for a main phrase.

Take a close look at the page titles and meta descriptions as you browse the cart. They are being dynamically generated using the product names and descriptions from the cart. I had him set it up so it would capture the first 160 characters of the product description. Not bad huh?

[Advisor]

So, P1, those plus signs and percent signs aren't a problem for the spiders? I see them on pages at the Goped site.

Jill

[Advisor]

I'm confused! I'm still seeing &'s and = signs in the URLS at the Eagle site also...what am I missing here?

Jill

[pageoneresults]

He was supposed to fix the spacing issue (%). Sorry. They will be fixed by Monday. Not all have them. My understanding is that the + symbols are not an issue. In fact, when you do deep searches, you use the same sequence...

goped+hp+parts

Oh, you should see the results I'm getting from Brett's SIM Spider. Its even picking up the reviews!

[pageoneresults]

You're not reading...The main one, is the only one that has the fully optimized cart.We just finished with Gopeds this morning. We will now slowly incorporate the URL parsing into the other sites using the cart.

[Advisor]

Ahh...okay...you're right, I was just skimming!

Jill

[f00sion]

the + signs are only there when the product or department title have a space, if they are linked without first encoding the spaces netscape will spaz out.. the same thing goes for . or any other non alphanumeric characters, they have to be encoded with the hex value (ie %2E for .) or it will cause other, sometimes unforseen problems.

[pageoneresults]

Ah-ha, it appears our fearful programmer has joined us, welcome to the forums f00sion, great username!they have to be encoded with the hex value (ie %2E for .)What exactly are you telling us here. That the %2E has to stay?

[f00sion]

i think it has to stay... i dont know any thing about the search engines but i cant understand why it would be a problem, they both mean exactly the same thing as far as the browser is concerned.... i remember the first web page i ever made was hosted on earthlink and the address was members.earthlink.net/~somethingorother but by replacing the ~ in the url with the equivalent hex like this: members.earthlink.net/%7Esomethingorother you would to the same place... but i could be wrong...

interesting to note when you hover over a link with a special character in it such as H.P. Parts and look at the browser status bar at the end it will say H.P.+Parts, the periods wont be encoded yet, they only turn into %2E's when you are on that page... i think it may be purely a function of the browser. I only have IE right now but if you test it with different browsers i wouldnt be surprised if the links show slight differences.

[pageoneresults]

f00sion, its 0104 in the morning, does your mom know you are up this late? :)

I spidered the links with the %2E and they spider fine. I was always under the impression that the % would cause problems with some browsers.

I'm hoping that a long time friend, Junior Harris, will drop in and post a few comments. f00sion, you two would have a blast. Junior is the one who got me started on this and I'm sure he could provide some insight.

If you want to test spider friendliness, try Brett's SIM Spider (http://www.searchengineworld.com/cgi-bin/sim_spider.cgi).

[pageoneresults]

Can you control the sequence of the URL? I mean, can you position where the department number is, the product name, etc... So instead of this...

http //www.gopedsuperstore.com/d/65//
X9JCTNKZT31BTCWBHNB6QKU5H/479/GoPeds

You get this...

http //www.gopedsuperstore.com/GoPeds/65/
X9JCTNKZT31BTCWBHNB6QKU5H/479/d

What concerns me is the number of /'s before you get to real content. If I count the number of /'s in the link, there are 5. That means 5 levels deep. Could be too deep for some spiders if they see it that way. Get my point?

[f00sion]

yes, i can change the order of the parameters, wouldnt be too hard, there was no logic behind my original ordering.

go to sleep edward!

[ihelpyou]

Welcome to the forums f00sion! :hi:

Glad to have you aboard. :)

[highman]

Pageone is the system using the 404 method?

your urls are like this:
http://www.gopedsuperstore.com/p/65//3138/EKPRTSM8UPAJGKP8GC97X5LAU/0/Bigfoot

what we tend to do is more like this;
http://www.gopedsuperstore.com/gopeds-Bigfoot-65.htm

where 65 is the product ID from the db

your custom 404 page strips the url of its .htm and only uses the 65 as the db ID call to load the page, the - are the field delimiters.

To the SE the URL now looks like its in the root and is full of keywords, we have great success with this technique, the best results are achieved if when the product page is called it writes the file to disk as a .htm file, next time its called it actually exists so there is no db call to make, no 404 to use, little processor power to use and the cache is fully used :)

When your products are updated via your admin system get the admin system to delete all .htm files from the disk, then the process begins again of creating the htm files........

Nice thing is even if the sql db falls over the site is still up :) and it makes reading stats and tracking user paths through the site dead easy ;)

now combine that with the .css stuff we are working on and you have one awesome SE friendly website

[Blue]

Welcome to the f00rums f00sion! :hi:

[f00sion]

Originally posted by highman
Pageone is the system using the 404 method?

your urls are like this:
http://www.gopedsuperstore.com/p/65//3138/EKPRTSM8UPAJGKP8GC97X5LAU/0/Bigfoot

what we tend to do is more like this;
http://www.gopedsuperstore.com/gopeds-Bigfoot-65.htm

yes, its using the 404 method, i could have used -'s or anything else for the delimeter but chose just a /. pageone originally suggested using - but was concerned about the name of the file looking too long.

[pageoneresults]

hey f00sion, is this a must...

/d/65//X9JCTNKZT31BTCWBHNB6QKU5H/479/

Can we eliminate that entire string and replace with something less obtrusive?

P.S. Man, I'm glad I got f00sion to join. I get more answers from him here than I do over the phone! ;)

[pageoneresults]

Now combine that with the .css stuff we are working on and you have one awesome SE friendly website.This would be the next step. I think I've sparked an interest in f00sion with the CSS. Once he updates my Eagle Malls testing site, I will convert that to CSS. Ooh, that's going to be fun, all that .asp, what am I going to do? :eek:

[f00sion]

Originally posted by pageoneresults
hey f00sion, is this a must...

/d/65//X9JCTNKZT31BTCWBHNB6QKU5H/479/

Can we eliminate that entire string and replace with something less obtrusive?

P.S. Man, I'm glad I got f00sion to join. I get more answers from him here than I do over the phone! ;)
what would be less obtrusive?.. we have to have that long string to keep track of the shopper..i could use a session but then it runs the risk of timing out or the users cookies being turned off and it wouldnt keep track of whats in their cart.

[JuniorHarris]

Welcome f00sion!~ :hi:

The cart is looking very good!~ You guys have really done your homework!~:read:

I might suggest adding a line of code to ensure the status is always returned with a 200-OK. Using WebBug (http://www.cyberspyder.com/webbug.html), the pages return the 200 using HTTP 1.1, however HTTP 1.0 returns a 302 Object moved. This may not cause a problem, but I like to set the status just to be safe!~ In ASP, this can be done using Response.Status = "200 OK".

Another suggestion, I think was mentioned above, would be to move all relevant keywords as close as possible to the front of the url:gopedsuperstore.com/p/65//3138/AN70UFIC8A7HDLEY38EMGBUBG/0/Bigfoot
would become
gopedsuperstore.com/BigFoot/p/65//3138/AN70UFIC8A7HDLEY38EMGBUBG/0/In addition, removing some "sub-directories" might also help. I will add that I have not noticed any problems with spiders indexing 5-6 levels deep. Especially if the site is interlinked well. However Google does appear to dilute the PageRank from the home page for the subdirectory pages, so that may be something to think about.

I do have a few concerns using encoding in the url may cause problems with some spiders. I have found a number of URLs in the serps with %20 for space in the URL, but this is typically a side effect and not necessarily hard-coded in the URL. I didn't spend much time, but I could not find any SERPS on Google (using allinurl:) with %2E or %7E encoded in the URL.

They may display fine or equal in the browser, but that does not guarantee that a spider will index the page(s). Since many sites sometimes use encoding to track sessions, these could be ignored for that reason. It may also be possible that a spider is smart enough to translate these, but I don't know for sure. Though it would be an interesting experiment, and I love to hear the results!~;)

Personally I would limit my URLs to simple alphabet, even avoiding punctuation marks. And "fill" any spaces with either a hyphen or underscore. For example the Riva+Racing+Deck would become Riva-Racing-Deck, and if moved to the front would read gopedsuperstore.com/Riva-Racing-Deck/[...]. The character "fills" or "substitutions" could be performed on the data when it is returned from the database (and again for processing requests), or even possibly resolved by creating an "engine friendly" field in the database to use.

I hope I helped more then I hindered!~;)

[f00sion]

Originally posted by JuniorHarris
Welcome f00sion!~ :hi:

The cart is looking very good!~ You guys have really done your homework!~:read:

I might suggest adding a line of code to ensure the status is always returned with a 200-OK. Using WebBug (http://www.cyberspyder.com/webbug.html), the pages return the 200 using HTTP 1.1, however HTTP 1.0 returns a 302 Object moved. This may not cause a problem, but I like to set the status just to be safe!~ In ASP, this can be done using Response.Status = "200 OK".

is the 302 due to the redirecting from the 404 page?.. should i set the response.status at the top of every single page?

I can easily change spaces to _ or - and strip out periods in the links if it will make a difference... at the time it was just easier to do a server.urlencode() instead of a bunch of different replace()'s checking for every special character that may be in a product name.

I hope I helped more then I hindered!~
definitely a help! thanks!

[pageoneresults]

I can easily change spaces to _ or - and strip out periods in the links if it will make a difference...Good morning f00sion. Hyphens are the preferred the method. I would also take Junior's advice and move the core content to the beginning of the url string. Put all that other junk at the end. How many levels deep does gopedsuperstore go?

I know Junior has been very busy and hopefully he'll reply to your question above so we can further optimize the cart. Hey Junior, thank you very much for your input. You are the Parsing Guru!

[f00sion]

but what happens if i change the /'s to -'s and somewhere in the product or department name it has a -... such as Go-ped.. the parsing would get completely messed up.... i could use an underscore instead of a slash, that is less likely to appear in a product name.

[f00sion]

Originally posted by JuniorHarris

Personally I would limit my URLs to simple alphabet, even avoiding punctuation marks. And "fill" any spaces with either a hyphen or underscore. For example the Riva+Racing+Deck would become Riva-Racing-Deck, and if moved to the front would read gopedsuperstore.com/Riva-Racing-Deck/[...]. The character "fills" or "substitutions" could be performed on the data when it is returned from the database (and again for processing requests), or even possibly resolved by creating an "engine friendly" field in the database to use.

I have been thinking about this some more and now I have a new question... Using a + or %20 in place of a space in a url is basically standard for all web scripting languages like asp, perl, php, etc.... so wouldnt it make more sense for the search engines to ignore that or replace it with a space as opposed to a dash, which looks good for a human to read, but doesnt mean close to the same thing as a space as far as a computer is concerned? Perhaps if I had riva%20racing%20deck the spider would see it as riva racing deck, while riva-racing-deck would remain riva-racing-deck, and not show up if somebody searched for "riva racing deck"?... or do the spiders generally strip out all of the special characters?

[pageoneresults]

while riva-racing-deck would remain riva-racing-deck, and not show up if somebody searched for "riva racing deck"?... or do the spiders generally strip out all of the special characters?I don't think spiders strip out anything when it comes to the URL's. I'd be more concerned with the spider stopping at special characters and not indexing any further.

The hyphens represent a space to the spider. You stand a better chance of ranking for riva racing deck by using this...

riva-racing-deck

Instead of this...

riva racing deck (riva%20racing%20deck)

This conversation is moving beyond my area of expertise but you can be assured that by the end of this topic, we will have accomplished our goal of making the cart 100% search engine friendly. Am I correct f00sion? You'll do whatever it takes, right?

[f00sion]

Originally posted by pageoneresults

This conversation is moving beyond my area of expertise but you can be assured that by the end of this topic, we will have accomplished our goal of making the cart 100% search engine friendly. Am I correct f00sion? You'll do whatever it takes, right?
yes, i would give my first born to have that... but if i replace all the spaces with hyphens then i cant use the hyphens as a delimeter in the path, so should i stick with /'s for that?

[JuniorHarris]

Sorry I've been very busy...but I do believe the 302 is a result of the 404 redirect for http 1.0, this can be resolved by simply placing the status 200 at the top of every page. I think I have it as first statement...but of course be sure to change to status = 404 for any "real" 404 pages the code may parse [and handle appropriately] .

I'm sure some engines may interpret/ignore/substitute special characters, but I think using "regular" characters would stand a better chance of being interpreted by the algorithms correctly. As I posted earlier, I did search and find pages listed with %20 in the url, however I was specifically searching using '%20'. The real question is, will the algorithm use these special characters as word demarcation points or will it be interpreted as one unique string?

Sticking with hyphens for [key] word demarcation and using '/' for directory levels is pretty much guaranteed to work!~;)