white paper about spam

[french dread]

check http://www.ebrandmanagement.com/whitepapers/spam-classification/
it is a white paper trying to define what is spam and not spam. what you think?

[Mel]

HI French Dread:
I'm glad to see that there are people out there with enough interest and ethics to publish such a white paper, but I will have to disagree with some of this papers basic definitions and thus with some of its conclusions.

Search engines exist only to serve viewers or surfers; therefor to define relevancy from the search engines point of view is very limiting. I would prefer to define relevant search results as those which contain the maximum number of pages out of all those published which contain information which meets the viewers needs and are then ranked accordingly.

This concept implies that the search engine must to some degree be able to interpret the viewers search term and provide related information (i.e. search engine optimization might well contain information which would normally be returned if the term web site promotion were used as the search term) and also that it must have available in its index information on most if not all of the pages published. You can't search that which you don't have. Note from the beginning that this is not how most search engines work, but how I would like to see them work.

Taking this philosiphy a bit further; to define spam as something which would not be done if it were not for the existance of search engines is tantamount to saying that almost all commercial content published is spam, since if the search engines did not exist we would likely not publish any commercial pages since no one could find them. Certainly we would not have any page titles or meta tags, since these exist soley for the search engines use.

All in all I think we need a better set of definitions, but this paper is good in that it gets one thinking.

[ihelpyou]

It was quite a long page so will have to read it later. I did not recognize the domain.

[ihelpyou]

I know they said this is a "basic" thing but it is waaay too general with many things.

To say that just because invisible text may look like spam but may not be classified as such does not seem right to me. What else could invisible text possibly be?

To build a "what is spam" page would require great detail and would require one page on each individual aspect.

[markymark]

Re: invisible text. Dunno about this one. After all -using CSS and DHTML - you can define an invisible layer that becomes visible when an action is performed. IE: when the visitor to the site moves his mouse over an image, this 'invisible' text appears to explain what that section is about.

This could be viewed as invisible text. But is it spam ? I don't think so.

[ihelpyou]

oh okay. I assumed they were talking about regular html. yea, in that case, not spam.

Gots to be much more detail on the page. Too general.

[french dread]

Yes i don't agree with their definition of spam.
For me spam is to fool the search engines and the visitors, or to use techniques that can lead to see the website banned by SE.
If one use a technique to be found on a keyword but the "spamming" website is fully relevant according to the visitor's query, I don't see any problem, and don't consider it spam.
If SEs were able to index all documents there won't be no need to use such "tricks".

[ihelpyou]

well french dread, the case can also be made that all sites could also simply optimize what they have and not have to use tricks to do it. Sure it sometimes entails the site to make some layout changes but if that is what it takes to make it engine friendly, then that is what should be done. Tricks should never be used.

[french dread]

i do agree...i get better results optimzing directly websites. But the big majority of our customers don't want to make any change to their websites, except sometime the home page title and meta datas...so we are forced to create optimized pages (not spammy)reflecting the website content...and i consider it is not spam, only a visible page for SEs.

[ihelpyou]

What you are doing is fine. But what the owner does not understand is that the front page is the Most important to the search engines. IT should be optimized. If an owner refuses do change the front page, I simply refuse to have them as a client. :)

[MakeMeTop]

Alan from Ebrandmanagement has attended a number of conferences and discussion forums putting forward this point of view. Indeed, I believe that at the last SE conference in San Francisco the discussion became somewhat heated with Alan taking a pretty hard-line on the issue. I'll try and dig up the threads that discussed this issue previously - but ebrandmanagement are quite well known to conference attendees and are technically very adept. Alan is a founder member of WAIM.

I have to say that although I respect his opinion (which is pretty similar to yours, Doug) - I (personally) don't agree with all his points. It leads to good discussion material though.

[ihelpyou]

Alan who? I did not recognize the domain,... but might his name.

[ihelpyou]

Perkins. Did not see the name all the way at the bottom. Oh yes, I recognize him. :)

[Advisor]

Yes, I've met Alan at a few of the conferences. Very nice, English gentleman!

I do have to say though, that I think he is taking the definition of spam just a weeeee bit too far (and told him as much at the last conference!). I didn't see his panel in SF, but they put him on the cloakers panel as a point-counterpoint kinda thing. I thought that was funny of them to do that. For him to be on the cloakers panel would be like putting Doug at the how to spam Yahoo panel!

Jill

[Alan Perkins]

Hi there

It's my White Paper, so I'll attempt to address some of the issues you've raised so far.

Before anyone can define spam, they need to define relevancy. I chose to define relevancy from the SE's perspective, because I think that's the only meaningful way to do it. Why? Because there is no absolute, objective relevancy factor.

E.g. one search engine ranks coke.com number 1 for a search on "cola", another search engine ranks pepsi.com number 1. Which of them is "right"? The only answer *you* can give to that question is a subjective one, because the answer *is* subjective. So the answers the search engine gives on its SERPs are its subjective opinions. If the answers were objective, every search engine would give the same set of results for every query. How useful would that be?

So, given that there is no absolute, objective relevancy factor, I chose to define relevancy as "The search engine's measure of how well a particular resource matches the input search criteria". The SERPs are ordered by relevancy. Does anybody disagree with that?

Now we know what relevancy is, we can define spam. I defined spam as "Any attempt to artificially influence a search engine's ability to calculate relevancy".

What does that mean? It means that if you can find ANY reasonable excuse for doing something other than trying to obtain an increased search engine ranking, then it's not spam. But if the ONLY reason you're doing something is to get an increased ranking, then it is spam.

This makes sense to me. If the only reason you do something is to get an increased ranking, then you are taking advantage of some knowledge of a search engine's algorithm, not to deliver more relevant results according to that algorithm, but simply to get an increased ranking. That is spam. If you were a really good at it, you would obtain top positions for everybody on every search engine. Every search engine would give the same set of results for every query. How useful would that be? (Recognize that phrase? Second time this post)

Another reason I defined spam as I did was this: because otherwise you will get into endless heated debates over what is and isn't relevant, and therefore what is and isn't spam. Such debates are pointless. Relevancy is subjective, so spam is subjective. The only way you can define spam objectively is to consider it on a search engine by search engine basis, as I have.

If you agree with how I've defined spam and relevancy, but disagree with something later in the White Paper, find the point where your opinion starts to differ and let me know. The document is (supposed to be) a logical progression from beginning to end.

I'm interested in your opinions but if you do disagree with something in the paper, please say why with enough detail to enable a discussion. It took a long time to write - so please don't just write it off with a casual phrase.

Alan

[ihelpyou]

Welcome to the forums Alan! :cheers:

Thanks for explaining that very simply. Now I see that we are on the same page. :)

I will have to read it carefully when have some time. As far as what you just wrote, I agree completely as far as I can tell.

Thanks for becoming a member as you can be a big help in here right along with the many other quality members we have. :thumb:

[Mel]

Hi Alan and Welcome to the forums!

I am apprecative of the fact that you took the time to write the paper which must have been a lot of work, and is a great place for discussion to start. I do not propose to pick your paper apart, its your opinion, it just so happens I don't quite agree. I prefer to look at the bigger view and if the basis for the conclusions is flawed, then why debate the details?

I guess my objections or differences may come from viewing things from a different perspective, or from trying to view from a larger or smaller perspective.

Relevancy from the point of view of the search engines is interesting, but I believe the search engines should exist to serve the viewers, not vice versa, and so to define relevancy as whatever the search engine decides it is, makes me feel that the millions of web users are being short changed by this definition.

Rather it is the search engines who should seek to satisfy the users not the users who should conform to the search engines dictates. Just because Google does a good job of indexing and ranking a good part of the web does not mean there cannot be improvements, or that perhaps someone, say, Wisenut, might not just do better and with different rules.

Now I know when viewed from the perspective of SEO, your definition of relevance is perhaps the way it works now, but thats because thats all we have to work with. We should try to encourage the SEs to provide more relevancy and more pages, and thus our definition of relevancy shoud be what they should or could do, not just what can do now.

The definition of spam is a different kettle of fish however, since spam exists only according to the search engines definitions. If Google says tomorrow that they will rank blue colored text higher, everyone will rush to change their color to blue in order to get a better ranking. So according to your definition those who had blue text before Google decided to rank it higher would not be spamming but those who used it afterwards would be? This does not seem to make any sense to me.

Conversly to my argument about relevancy, if there were no search engines there would be no spam. Therefor it seems to me that spam is whatever the particular search engine decides it is. Granted they don't always tell us fully what their definitions are,and that is perhaps we are tyring to define what spam is.

So without going into a lot of detail about relevancy and spam, I would beg to differ with the basis of your conclusions.

[ihelpyou]

Yes, I took the paper to mean what it is like right now. So, relevancy is according to what the engines deem it is.

I do agree that the engines need to improve based on the searchers, but I think that is a given as they always try to improve.

I really do need to take some time to read the paper carefullly, however. :)

[Alan Perkins]

Hi Mel

"Search engines should exist to serve the viewers". Of course. A search engine that delivers the most relevant results to a particular market (i.e. set of viewers) deserves to be (and often is) the most popular search engine in that market. But "relevancy" will still be the search engine's measure, which its market happens to agree with. Web users aren't being short changed by this. They will naturally gravitate to the search engine(s) they agree most with - market forces. It's a bit of a chicken and egg argument, but since a search engine and its market should have similar ideas about relevancy, it makes no difference which comes first. In other words, the search engine's idea of relevancy *is* its market's idea of relevancy - otherwise it wouldn't have that market. In simple terms, Coke drinkers will use the SE that ranks coke.com first for "cola", Pepsi drinkers will use the SE that ranks pepsi.com first.

Still disagree? Then *you* define relevancy in an objective way.

No search engine defines spam adequately. One of the aims of the white paper is to do this. And not by giving out a list that says "invisible text is spam", "noframes text is spam", etc. etc. I have a general principle that if somebody has puublished a site without ever thinking about search engines while designing the site, then they CANNOT be spamming. I haved used the corollary of that principle as a the definition of spam.

I'm working with search engines to get their agreement on this White Paper (can't tell you more about this at present). The publishing of lists just gives ideas to spammers and gives them refuge when they find a new technique that isn't on the list. I'm looking for principles. Concentrate on publishing good, well-structured accessible documents and sites, and let the best search engines find and index those sites. That way, the best sites and the best search engines survive.

Alan

[JuniorHarris]

Welcome Alan!~ :hi:

First let me say I despise spam, be it in the engines or email!~ I was very interested by your distinction between IP cloaking and IP delivery. It really seemed to draw a reasonable line between the two techniques.

But how do you address those which claim IP cloaking protects their code from being plagiarized by others? Webmasters and SEOs put many hours into the content and design of their site, all which can be quickly copied and presented by pirates as their own. If I were a SEO by profession, I would be greatly concerned over rouge duplication, as this could provide competition with an exact copy of the techniques and tactics employed. Especially when they can copy in minutes, what may have taken years of practice to develop.

[Advisor]

Hey, Alan! Welcome to our little home away from home!Concentrate on publishing good, well-structured accessible documents and sites, and let the best search engines find and index those sites. That way, the best sites and the best search engines survive. This is exactly what I've been saying for years, but unfortunately, it's truly a naive view. In a perfect world, the best most relevant sites for the search query would be the ones to rise to the top of the SERPs. But it's not a perfect world. If the Seo specialists all adhered to the absolutely no spam rules that you discuss, things would work perfectly. However, there will always be others out there who want to get that extra edge and will do whatever it takes to get it. Which leaves those who try to operate within the rules left out of the top results.

What you are proposing (it seems to me) is basically the abolishment of search engine optimizers. After all, if one can't use search engine optimization techniques because they are really just spam, then we SEOs will become extinct. Anyone should be able to publish a great page and it will rank high naturally.

It's a nice fairy tale, but I don't see it happening. We SEOs have a duty to our clients to do what it takes to get their sites good rankings (within the rules of any given search engine). It sounds like according to your definition, the very fact that they are hiring us to do something to their site that they didn't do naturally, would be considered spam.

Jill

[ihelpyou]

Personally, I believe that excuse is just that......... an excuse.

I think there are too many other things involved with good ranks than someone simply copying code. First of all, if someone copied my code and then eventually got ranks up there with me, I would discover this and report them immediately and say they copied and show the proof. I do not think that would happen however, as many other factors are involved.

oops. BTW, my post was referring to the JH post. :)

[Alan Perkins]

Hi Junior

"How do you address those which claim IP cloaking protects their code from being plagiarized by others?"

I could write a very long answer to that one, but I'll just give you the short, brutal version. The "code" is spam. How can anybody say that content is relevant to humans when it can't be seen by humans?

As for plagiarizing, theft is theft. Work with the search engines to stamp it out. Presumably the thieves will use cloaking to hide their crime. Leave cloaking as a technology for criminals, and it makes it easier to spot the criminals.

Alan

[Alan Perkins]

Hi Jill

"However, there will always be others out there who want to get that extra edge and will do whatever it takes to get it. Which leaves those who try to operate within the rules left out of the top results. "

What are you saying? You have joined the others and you don't operate within the rules?

"We SEOs have a duty to our clients to do what it takes to get their sites good rankings (within the rules of any given search engine)."

What if those rules didn't allow you to do what it takes to get good rankings for a competitive keyword?

"It sounds like according to your definition, the very fact that they are hiring us to do something to their site that they didn't do naturally, would be considered spam. "

Absolutely not. If you can find ANY reasonable excuse for doing something other than purely to increase rankings, then it's not spam. What you'll probably find yourself doing is improving the accessibility and/or information architecture of the site - both very worthwhile activities.

Alan

[ihelpyou]

What you'll probably find yourself doing is improving the accessibility and/or information architecture of the site - both very worthwhile activities.
That is good. This is what I do and knowing Jill, exactly what she does as well.

We do things to acheive ranks for many more reasons than simply to "manipulate" ranks. For this reason, I do agree.

[Mel]

HI Alan:

First off I see no way to justify the statement that just because a search engine is popular it delivers relevant results. This is more true the more exeprienced a user is, but look at the millions of AOL users as just one example. Do you consider they are getting relevant results? There must other factors besides market influence at work here, and one of them is convenience.

I will not try to define relevancy because it is not an objective term in my view. From my point of view relevance is subjective - some engines do it better than others and some users need different information presented in a different ways, and thus thier definition of relevancy and yours or mine will necessarily differ.

I personally would like to see search engines clever enough to know that when I search for the term search engine optimization, for instance, there might also be relevent results from other nearly synonomous terms like web site optimization, etc. Wordtracker is doing this for keywords now, but its up to the user to make the decisions. That is the sort of relevancy that IMO an all encompassing definition of relevancy should encompass.

I agree that the search engines do not adequately define spam - nor are they ever likely to, given human nature. But to try to isolate the creation of a web page which is solely built to attract potential customers (that is the purpose of commercial sites is it not?) and say that if you consider search engines when you write your copy or build your page titles or put in your headers then you are spamming, I simply cannot agree with that in any way shape or form, nor do the search engines agree with this view. The SEs give us some guide lines and we infer others, but the idea that if I include an extra header to get better rankings I am spamming is not something the SEs would agree with nor act on.

Please do not think I am just trying to tear down all your hard work, but if you are striving for a set of universal defnintions and trying to get the SEs to agree with them, they should be forward thinking.

[Mel]

Yes Jill

What you are proposing (it seems to me) is basically the abolishment of search engine optimizers. After all, if one can't use search engine optimization techniques because they are really just spam, then we SEOs will become ex
Thats the way I see it too, this defintion of spam does not take into account the real world fact that what clients pay SEO firms for is to increase their rankings.

[Advisor]

Originally posted by Alan Perkins
Absolutely not. If you can find ANY reasonable excuse for doing something other than purely to increase rankings, then it's not spam. What you'll probably find yourself doing is improving the accessibility and/or information architecture of the site - both very worthwhile activities.Well, this is exactly what I do currently do, but I'm merely playing devil's advocate here.

The problem that I still see is that people can always come up with some lame "reasonable" excuse for doing what they're doing. Whether or not we agree with it, doesn't mean it's necessarily wrong.

In a way, it's much the same with the whole problem at Yahoo that Doug talks about a lot, and that I discussed in this week's Rank Write. What do you think about that? Is it spamming Yahoo to create a new company and Web site that uses keywords in it's name and url? I can think of lots of excuses that can justify it.

Ironically, I had a potential client call me yesterday because of that discussion in RW. He was in the process of forming a new company for his product anyway, so when he read how you can "spam" Yahoo and get good rankings, he changed the name he was thinking of using. Nice to know I'm helping people learn new spam techniques...Yahoooo!

Jill

[Alan Perkins]

Hi Mel

I think you're wandering off-topic in talking about how search engines might work. Let's stick to defining spam. I'm disappointed you won't define relevancy, because that's required in order to define spam. But let's look at your issue with meta data.

The White Paper further classifies spam as either Content Spam or Meta Spam. Meta Spam incorporates tites and other meta data. Since meta data was not designed to be seen "on the page", the "in the absence of search engines" test cannot be used to test meta data for spam. The White Paper makes this quite clear. It defines meta spam as follows:

"Data within a Web resource that describes that resource or another Web resource inaccurately or (when the data should be readable by humans) incoherently"

In other words, you need to describe a resource accurately and coherently using a title or meta description, and just accurately using meta keywords. BTW, Jill, this is another area where SEO skills will always be required.

Alan

[ihelpyou]

Great thread!

If you take a step back and try hard to look at this from all points of view, you can quickly see why this topic is ALWAYS a hot and good debate no matter where it takes place.

This highlights the strong opinions of all views on the subject. Because of this, the whole subject seems to be based on what point of view you are looking at it. I can agree with Alan's point of view and yet at the same time, agree with Mel's point of view.

Truly interesting and one that will rage on and on.

[Alan Perkins]

Hi Jill

The White Paper wasn't designed to look at directory spam, although the meta spam section comes close. I'm not too worried about directories as they have human editors to look after them. We're talking about tricking robots in this thread.

You joined this thread by saying I was going a wee bit too far in my definition of spam. Do you still think that?

Here's the kind of "Reasonable" test I'd apply. Is the technique being used for the purpose it was designed? If not, does the creative use to which it is being put have any purpose other than attempting to achieve a rankings boost?

Here's a question for you (I wonder if you'll spot where I got this idea). If wrapping a large chunk of html content with a H1 tag (modified with CSS or a font tag so it wasn't so big and ugly) gave you a rankings boost, would you do it?

Alan

[ihelpyou]

Here's a question for you (I wonder if you'll spot where I got this idea). If wrapping a large chunk of html content with a H1 tag (modified with CSS or a font tag so it wasn't so big and ugly) gave you a rankings boost, would you do it?
Many do this and it is talked about in this forum. I would not do this simply because it is Not necessary and does look spammy to me. Another reason is that you can simply use a numbered h tag that is the right size without any CSS. IMO, there is really no difference or boost given with an H1 tag compared to a h5 tag.

[Mel]

Alan:
Well....... If you are going to restrict your definition to include only the way that things work right now, this minute and not tomorow, I would perhaps take a different view, but I am interested in tomorrow, not yesterday.

As I said, I won't attempt to define relevancy because IMO it has very different meanings to different individuals, and in fact it should. What is relevant results for a professional researcher looking for data on Molecular metallurgy, and what relevant results are for Joe Blow looking for a good second hand car are and should be vastly different. They have different needs.

You are of course free to put forth your definition of relevancy, but I feel eqaually free to say that I do not think my definition would necessarily apply to someone else, nor would yours apply to me.

Now honestly how can you apply one standard to one thing and then another to something else? If you do it on the visible page with search engines in mind its spam, but its ok to do something off the visible page with SEs in mind??? That seems to smack of double standards.

But how about a reply to commercial pages which are built expressly for the purpose of attracting customers through search engines? Are they all automatically spam, since the were built with search engines and ranking in Mind?

The brick and mortar corollary to me would be advertising. It seems akin to saying anyone who advertises is unethical, let them all build good products and let the best product win. How are you going to find out about who has the best product, or even what products are available? Let word of mouth spread through the 6 billion souls on this earth? I'm not abig fan of advertising, but it is a cornerstone of modern business and does provide a useful service in that the public is at least informed as to what's available.

[ihelpyou]

I'm not too worried about directories as they have human editors to look after them.
Too bad the editors of Yahoo do not do a very good job of this. :rolleyes:

[Alan Perkins]

Hi Mel

"This defintion of spam does not take into account the real world fact that what clients pay SEO firms for is to increase their rankings."

In what way does it not take that into account? In any case, I would not build a business on increasing rankings. There is no long term future in it. SEs are already rotating results. Imagine the day when each individual user gets their own personal set of results. What do rankings mean then?

Build a business on increasing relevant traffic or improving ROI. Maybe thats not what clients pay some SEO firms for right now, but its what they really want.

Alan

[Mel]

Hi again Alan:

Perhaps I should rephrase that to say "this Definition of spam assumes that all SEO is automatically spam" which would be more precise.



Imagine the day when each individual user gets their own personal set of results. What do rankings mean then

Indeed. Imagine that day, what does spam mean then and how does your defintion of relevancy apply?

[Advisor]

Here's a question for you (I wonder if you'll spot where I got this idea). If wrapping a large chunk of html content with a H1 tag (modified with CSS or a font tag so it wasn't so big and ugly) gave you a rankings boost, would you do it? Yep. And I put keyphrases in alt tags too.

Jill the spammer

[ihelpyou]

Yes Jill, but you are assuming that the h1 tag equates to a boost over a smaller number h tag. ..... hence, looks spammy to me. :) I am simply saying there is no justifiable reason to do this, even for a spammer as there is no boost noticable. :)

Alt tags are used to allow those surfers with graphics turned off to see the content.

[ihelpyou]

This is fun! :cheers:

[Alan Perkins]

Jill:
Keyphrases in ALT text is not spam per se. Keyphrases (and particularly streams of keywords) applied to images they have no relation to is spam. And yes, using H[1-6] tags for things that aren't headings is spam. Why else do it?

What about visitors who don't have CSS support? What will they see? What about visitors who can't see, who have the page read out loud by a text-to-speech converter? All Heather's lovely copy, SHOUTED at them because its in a heading tag. Is this professional work?

Next you'll be saying "It's relevant because I say it's relevant". After that, it's one small step to protecting your highly optimized code by cloaking.

Mel:
"What does spam mean [when all results are personalised] and how does your defintion of relevancy apply"

Exactly the same as now.

[ihelpyou]

And yes, using H[1-6] tags for things that aren't headings is spam. Why else do it?
It is a known fact that no matter if you use a <b> tag or a <h> tag for this, they both give a slight boost.

So..... if I want to bold some text to emphasize it to my visitor, I prefer the h tag for this purpose. So yes, some tags are not necessarily true headers.

BTW, there is No documented absolute proof of which tag gives a bigger boost, so I do not consider this as "spam" per se.

Spam is pretty much how we look at it and is the reason why it is sooooo hard to discuss. :)

[Advisor]

Well, actually, I lied. I don't put whole paragraphs in Header tags. Just the actual headers! But I don't leave them at the base (BIG) size.

You can see what we did at my new highrankings.com site for the headings. The red text is just for headers and it is in H1 I think, but not huge (thanks to CSS).

And you're right, Doug, this may or may not boost rankings. I've never tried to prove it either way. In my opinion though, it's one of those little things that may work, so why not use it where possible (and reasonable).

Jill

PS. On my old site, in the olden days, I did have the whole thing in <H3> tags. But I didn't change the font size or anything. I actually liked it that big and bold! (Old eyes I guess!)

[MakeMeTop]

Hmm, I think all decent SEOs are very interested in giving clients an ever better ROI by increasing relevant traffic - but this is hard to do without having visibility - which means enhancing their rankings. All SEOs (as I've said before) 'manipulate' search engine results. The addition of a <title> tag or changing it from Home Page is a simple manipulation. Buying several directory listings in order to boost rankings for competitive terms such as LS on Inktomi is manipulation. Is this spam?

I'm paid to get clients results and believe I do it in an 'ethical' manner - but that too is a subjective term. Fact is - as a professional SEO who is in this for the long haul and have clients stay with me consistently - I don't believe in stupidity and won't use short cut, fly-by-night methods. But I will and do use methods which I believe will get my clients the results they are paying me for - without (hopefully) pushing the SE umbrella too far. To me a non-spam result is the user clicking on a listing which is completely self-explanatory as to the purpose of the site and then arriving at a web site that satisfies their query without having to go through 'click here' type pages. A spam result is the opposite.

The methods used to achieve this can only be deemed spam by the search engines themselves - and that gets cloudier as paid inclusion grows in popularity! Look at the Ink policy on gateway pages - a previous no-no. Or AltaVista's Trusted Feed - whereby you just send them a title, meta data and description and up you'll pop - regardless of the page content. Or how about LS listings where they will happily stuff the SERPs with listings from one company (in conjunction with MSN) as long as you pay per click.

Things are changing - and SEOs who are starting to contribute to the coffers of the search engines are starting to be treated with a little more consideration by the SEs we pay (I get e-mails from SEs every few days). Is this good? Not really - everyone likes a level playing field. Is it progress - yes. It is a commercial balance. I think that professional SEOs WILL be viewed by search engines as people who will be rewarded (provided we pay the piper) as long as we serve up relevant results and give the user of the search engine, results which are useful - and not really care about the methods too much - and penalized heavily if we abuse this trust.

Companies pay me a significant amount of money to get them results and they do appreciate that it is not all about being #1 for some generic term - but good focussed traffic. But for better or worse that means I use my knowledge of search engines to my clients best advantage. I believe, however, doing it the way I do is also to the benefit of search facilities and (as importantly) the user who selects one of my results after having typed in the subject they were looking for.

[Advisor]

Barry,

Yes, indeed! I agree completely and couldn't have said it better myself!

Jill

[Alan Perkins]

Doug:
"Spam is pretty much how we look at it"

That doesn't get us anywhere. The whole point of the White Paper was to be objective about it. You call someone a spammer. They say they are not. Who's right?

The White Paper gives you a way to find out.

Jill:
If you lied, then you didn't answer the question:

If wrapping a large chunk of html content with a H1 tag (modified with CSS or a font tag so it wasn't so big and ugly) gave you a rankings boost, would you do it?

Everyone:
We're sort of going round and round. Does anybody need anything clarifying in the White Paper, or does it all make perfect sense? And, if it makes sense, do you agree with its definition of spam? Reminder - it is here:

http://www.ebrandmanagement.com/whitepapers/spam-classification/

Alan

[Advisor]

I can't answer your question because it all depends on what the definition of "a large chunk is." Which is why I said yes, to begin with. I thought that a Heading on the page might constitute a large chunk, and I have no qualms about using an H tag in that case.

Jill

[Alan Perkins]

Jill

Stop wriggling! What if it wasn't a heading?

Alan

[nicebloke]

Hey Alan why not check out this thread.

http://www.webmasterworld.com/forum5/713.htm

I'd be interested to know what WebmasterWorld make of your white paper. Will you be posting it there?

[Advisor]

Stop wriggling! LOL...you sound like my husband :D

What if it wasn't a heading? I don't generally use Header tags (especially H1) if it's not a heading. But I wouldn't report someone for spam if they did that! And I'm not saying I never would do it. However, there's more important things that I would prefer to worry about in my SEO campaigns.

Even if those H tags do give a boost, the actual body copy is so much more important that it's not really worth the trouble, in my mind, to resort to playing those sorts of games. However, I'm stopping short of calling it spam.

Jill

[ihelpyou]

Welcome to the forums nicebloke! :hi:

Glad you stopped by. BTW, we have many who post in both forums. I encourage it and see Brett's Place as a very good resource. As well as here. :)

Just the forums I mentioned, however. ;)

[Advisor]

Want me to post it over there? Alan may not feel comfortable posting his own white paper (so as not to look like self promotion).

Jill

[ihelpyou]

Very true,... no self Url's are allowed over there.

Anyone can post it.. either Jill or nicebloke or even me. Alan may want to approve it or not though.

[Alan Perkins]

nicebloke:

I'm sure it will get round WmW and I'll join in then. It really is a White Paper - i.e. a discussion document, not a set of rules - but there are only so many hours in the day. Feel free to post it yourself, but I'd put it in a newer thread!

I noticed the discussion on this forum from our server logs (I honed in on the White Paper because it's new), checked it out and thought it looked a good place to start the discussion (no hardcore spammers here). This site has a board called "Spam Reports". WmW has a board called "Cloaking - Stealth". I think that neatly summarizes the difference between the two sites. Don't get me wrong, I think there are some great people at WmW, but it wasn't the place to start this discussion. I've been there before:

http://www.webmasterworld.com/forum24/35.htm

That was a year ago. Since then, I have been getting the ideas documented and consistent. They're pretty clear now. I also spoke at the August Search Engine Strategies in San Francisco on this topic. That was good fun!

I'm not trying to make trouble for the SEO industry. I just want to work towards a mature, professional industry. All professionals have a body of ethics they can call upon and evaluate each other against. This is my attempt at it.

I think I can get search engines to agree to the definitions of spam in the White Paper. I already have off-the-record agreement from one leading search engine and some leading Web marketers. If the "industry" includes search engines, it is a much more worthwhile industry. And why shouldn't it include search engines?

Alan

[ihelpyou]

If the "industry" includes search engines, it is a much more worthwhile industry. And why shouldn't it include search engines?
Couldn't agree more with that.

[JuniorHarris]

>First of all, if someone copied my code and then eventually got ranks up there with me, I would discover this and report them immediately and say they copied and show the proof.

Yes, but the damage has already been done. Duplicate content is never appreciated by the engines. And not all engines are as prompt to remove pages. In addition stubborn duplicators would have to be addressed by costly legal actions. Unfortunately I know someone who had this experience...with plenty of proof, the process still took months.

Regardless of the end acceptable format for delivery, there will always be those which push and exploit the boundaries. Take away frames, redirects, layers, alt and meta tags, and there still exists an "optimal" format for delivery. So there will still be those who strive for optimization, and those which exploit and exceed those boundaries.

Search engines alone will not be able to stop spamming effectively. It will require interaction with registrars and host providers as well. With domains a dime-a-dozen, most heavy-duty spammers can produce a whole army of domain/page fodder to throw at the engines. So simply removing a single page or domain may have minimal affect.

Instead we must go after the General and eradicate it at the source. Using registrar and host information, as well as any paid subscription services to track down each and every domain/page which would be associated.

[ihelpyou]

I feel that no matter what someone gives reasons for cloaking, the fact is that the code is hid from all surfers. As long as cloaking is allowed in any form, there will always be the temptation for honest SEO's to spam by cloaking and spammers to use cloaking to continue spamming.

Search engines should not allow it for this reason. Of course, this is IMO.

[Alan Perkins]

It's Friday night here in the UK and I've got kids to see to. I think you all know where I stand now. I guess I'd still like to know where you all stand. Is the White Paper something you could agree to? The consensus appears to be "no" at present, but I'm not sure why...

I'll check back in later to see how you're getting on.

Alan

[ihelpyou]

I have not read it carefully yet Alan. Will do so tomorrow. :thumb:

[Alan Perkins]

Here's my goodnight sign-off. I'll check back Monday, probably:

Doug:
I look forward to your response. Thanks for taking the time to read it carefully. I know you were looking for a fully itemised list of spam. Here's why I did it like I did ...

JH:
Take away frames, redirects, layers, alt and meta tags, and there still exists an "optimal" format for delivery
No DON'T take those away. Don't take any piece of HTML away, or make it illegal. It was all invented for a reason. Just use it for the reason it was invented.

I get so annoyed at phrases like "invisible text is spam" when "invisible text" could have been placed on a perfectly good page by somebody who had never even thought about search engines (perhaps because it was going to be made visible later). By contrast, cloakers make sure that invisible text is never placed on their perfectly bad pages, yet the whole page is invisible and they maintain it's not spam!

Everyone:
Blindly following specific rules is not the answer - understanding principles is. The industry, both SEOs and search engines, needs to grow up a little and realise that the whole world does not revolve around *it*. Far from it in fact, most "outsiders" think it's a lot of shoddy trickery and black art.

Take a look at W3.org and you'll see a fantastic site designed by real pros. See how search engine friendly it is. It would be ridiculous to say that any part of that site was spam, whether or not it broke some spam "rule". It certainly doesn't break any of my principles. While you're there, take a look at the accessibility section. I get so frustrated by people who make a site rank better on search engines at the EXPENSE of making it accessible to less capable platforms and users, when it should be BY making it accessible to less capable platforms and users. Read the quote by Tim Berners Lee while you're there, in context. Here it is out of context:

"The power of the Web is in its universality. Access by everyone regardless of disability is an essential aspect."

I know that the way the White Paper defines "spam", probably every SEO reading this is spamming now or has at some time in the past. I take no pleasure in that, believe me. I wish it wasn't that way - life would be so much easier, for a start.

Would you believe that the way I define spam, I was once a spammer? It's how you react when you discover you are spamming that defines *you* (professionally, at least). The three ways of reacting are Denial (can't or won't believe it, carry on doing it), Disdain (don't care, carry on doing it) and Development (learn, mature, do better). I get on fine with people who react with disdain, since at least they agree they are spammers. They can usually justify to themselves if at least they think their results are relevant. It's those in Denial that are the most difficult.

On the WmW issue, I've been thinking and I would prefer to fight my battles on one front at a time. So please hold off until we're done here. If I can't win over you lot, I'll be wasting my time there anyway.

See you Monday.

Alan

[Mel]

HI Alan:

You said you were disappointed that I did not define relevancy. I hate to see disappointed people so......

OK, please reread my thoughts on why I feel relevancy may have different meanings to different people, and with that to build on let me propose a definition of relevancy as regards search results:

Relevancy - the ability of a search engine (or directory) to present a set of results in response to a users query which contain a significant proportion of all published (web) pages containing information on that subject and which the user finds useful.


You also mentioned that one had to define relevancy before you could define spam.

Note that my proposed definition does not discuss rankings, (although the user may well like to have his information prioritised in that manner), and therefor is not tied to spam in any way. IMO spam only exists in relation to rankings rather than relevancy. Rankings on the other hand have to do with convenience rather than relevancy and so have no place in a definition of relevancy. The inescapable conclusion is that relevancy and spam have no direct relationship to each other. In fact a very spammy page can be very relevant and there is no reason it could not even qualify as an authority on a particular subject.


With regards to rankings vs traffic - the two are obviously related, unless you are going to suggest that poorer rankings will drive more traffic? And yes there are other ways besides search engines to get traffic, but I assume here that we are discussing search engines only as this is a search engine forum. Better rankings will generate better traffic, but you also have to write better pages if you want to convert those visitors, but this has to do with salemanship rather than relevancy.

Its true that every business wants a better ROI, but that has to do with much more than SEO. It has to do with product design and quality, Pricing, after sales service, guarantees etc. All of these are outside the scope of normal SEO activities.

I therefor base my business on getting better rankings for my customers.

True this will drive more traffic (but you cannot say that a particular ranking on a particular engine will drive a particular amount of traffic) and it may well also increase his ROI, but this will be as a result of better rankings and the resultant traffic.

[Mel]

Hi All;

On reflecting further I think the problem we are having here is related to considering SEO as a Science rather than an Art.

IF (a very big IF) SEO were a science, then we could develop rules and formulas to define it.

On the other hand if you define, as I do, SEO as an art, then you can no more define it with rules and formulas than you could define Rembrandts art by assigning values to each color used and developing formulas as to the ratio of each color used, their proximity to each other and whether broad brush strokes were better used with red than blue.

You may be able to imitate Rembrandt, you might even create original works which are comparable, but you cannot define that which makes his art great with formulas.

As a meassure of the truth of the assumption that SEO is an Art, consider that while the new and inexperienced are continually looking for "how to" lists and are anxious about keyword density, meta tags, etc, the professional SEO more often than not does something because "it just feels right"

[ihelpyou]

As a meassure of the truth of the assumption that SEO is an Art, consider that while the new and inexperienced are continually looking for "how to" lists and are anxious about keyword density, meta tags, etc, the professional SEO more often than not does something because "it just feels right"

Very good.

This comes with experience and LOTS of it. My opinion is, the more clients you have had, the better you are at "feels right". I believe this could be one of the most important parts of SEO.

Do I spend lots of time optimizing a client site? Many say they do. I do not. Not comparatively, that is. Do I think SEO is all that hard? Many say it is. I do not. Do I think SEO is a numbers, figures, and density game? Many say yes. I do not.

I simply do what feels right at the time depending on the site's situation at that time. If I do not want to submit to a particular engine, I won't. Even if the site is NOT indexed in it. I only submit if it tickles my fancy at that time.

I strongly believe in this feeling right thing. This is how I work, and maybe how I can keep catering to the small to small-medium business with a low price. I don't make things complicated. Nice and simple.

Does any of my ways compromise ranks? Obviously, absolutely not.

[Advisor]

On the other hand if you define, as I do, SEO as an art, then you can no more define it with rules and formulas... Yes! You don't know how many times I've said that exact thing. Mel, you've just gone and summed up the whole thing as to why I'm so uncomfortable with the whole what is spam and what isn't issue. I didn't even know it myself. I felt the same way a few weeks ago when Bruce C. was in here with his own ethics rules. There is no way to place any one set of rules on something like SEO, and *that* is why it when I read those kinds of rules it makes me squirm!

On the other hand, if an engine themselves said, this, this and this are spam, then that would be another thing. However, I would be surprised if even the engines would feel comfortable agreeing that Alan's white paper was the definitive answer to what is and isn't SEO spam.

I wholeheartedly agree with most of what Alan says in principle, but as I said before, I feel that it's somewhat of a naive view.

Jill

[ihelpyou]

On the other hand, if an engine themselves said, this, this and this are spam, then that would be another thing.
This would be the only true way. Obviously, this would never happen as the search engines are using this to keep us guessing. If they spelled things out, then who would be at the top and who would not? We would all know exactly what to do it seems. Along with all the different parts of an algo, is all the different parts of what spam is and what spam is not. Only the engine knows these parts and would not have a great interest in relaying the exact definitions to us.

I also believe that some of the search engines have a so-called "black book" of who is good and who is being bad. We know that Inktomi does this and I am doubly sure Google has this as well. Of course, the good would be in a "blue book". :)

[ihelpyou]

The big problem with cloaking is that there are waaaay to many reasons and whys and ifs and buts to the equation. The only way to rid of this is to outlaw cloaking. Period. Sorry guys who use it. :eek: It is so fuzzy and has many gray areas that the engines probably struggle with it daily.

The engines are constantly striving to be more "relevant" in the eyes of the user. Their optimal quest would be to be able to get inside the searchers head and figure out what they are truly looking for and then present the results based on this.

Since they cannot do that, they are constantly trying to improve the robots to do this for them. I have seen a move to try and figure out the whole page or site as an entity and produce results based on that. It would be a good thing if they could take all the content of a page, decipher it and analyze it, then based on the content as a whole, produce the result. Google is getting much better at this, and the results do show it.

[Advisor]

But, Doug, I should think it would be in their best interests to spell out exactly what spam is and what it's not (if it were even possible). That way if they catch you with anything they've defined as spam, they can boot your butt, no questions asked! I suppose they can boot your butt anyway, but at least then they'll have a good reason in black and white.

Still, I don't think it would work, because you'd always get people who could think of a "legitimate reason why they were doing something, and then a whole trial would have to be held, and someone would have to judge whether or not it was spam. So in that case, it's probably easier for the engines just to say, we can boot the butt of any site we feel like it, so nyah, nyah, nyah to you!

Jill

[MakeMeTop]

>As a meassure of the truth of the assumption that SEO is an Art, consider that while the new and inexperienced are continually looking for "how to" lists and are anxious about keyword density, meta tags, etc, the professional SEO more often than not does something because "it just feels right"

Shhh, Mel! Don't give the game away ;)

That statement is so true. It also explains why nearly all the leading SEOs I've met - truly love their work. The analogy can continue - we create the work of art and it is up to the gallery (search engines) as to where (or if) they display it.

Anyway, back to the thrust of this thread - do I believe in Alan's definition of 'spam'. My answer has to be a no, I'm afraid. However I applaud him in opening up this debate.

[Mel]

Hi Alan:

I guess its time to summarize my feelings on this issue. No surprises here.

1. I regret that I cannot agree with your white papers definition of relevancy as to me it seems to be more a definition of rankings than relevancy, and I would like to be able to see those two terms each stand on their own with different meanings.

2. Nor can I agree with the papers definition of spam as anything you do soley to get better rankings or visibility. Using this defintion strictly would mean to me that any site I put up with the goal of getting good rankings and traffic would be spam, that would include almost anything I have ever done, and I am not ready to accept the spammer tag just yet.

I also have a problem with the definition of what is not spam since the litmus test here is if you can think of any legitimate reason to say you used something then its not spam. This seems to me to be just a loophole to allow you to wriggle out of the more strict definition.

An example might be the routine use of section headers when composing your pages. Most people do not do this unless they are involved in technical writing, where it is quite common. However, I can use section headers legally under this definition (when I would not otherwise) by saying "This is to make my meaning clearer" when actually it is to garner some ranking points from the SEs. How on earth can search engines determine what is and what is not spam?

What I really do like however is your attempt to define relevancy and Spam. If only we could all agree on that how much simpler life would be (but perhaps not nearly so much fun?)

[Alan Perkins]

back to the thrust of this thread

Good idea Barry. Let me try to address some of the issues raised today:

Mel:
I think your last post summed up your position, so I'll just consider that one:
In response to point 1) I don't understand you. SERPs (search engine results pages i.e. rankings) are ordered by relevancy. Relevancy = Rankings. What do you think SERPS are ordered by???
In response to point 2) see the points addressed to all below...

Jill:
I wholeheartedly agree with most of what Alan says in principle

It's principles we are trying to establish. So, you agree then? But will you stick to your principles in practise? If the answer to that is "No", then answer this: Are you a professional? In what sense of the word?

Sorry, those are nasty questions. But they need asking, not just of Jill, but of everyone. I asked them of myself, once, and got a shock. Now I check my answers every day.

it's somewhat of a naive view

Naive would be the wrong word, since I have at least as much experience in this field as anybody else. And, surprisingly, I earn a living, even with this view. :D Can we agree that it's a principled view?

All:

(I do not) believe in Alan's definition of 'spam'...
spell out exactly what spam is and what it's not ...
this would be the only true way...
I am not ready to accept the spammer tag just yet


What are you all saying exactly? Do you want an enormous book or Web site itemising every possible current instance of spam. Do you want this book or site to be added to or modified every time something new crops up? If that's what you're asking for, then

a) you're not going to get it, and, more seriously
b) you haven't persuaded me you even know what spam is

You appear to think spam is just something you shouldn't do because the SEs say you shouldn't do it, without knowing why they're saying that. Can any of you answer this simple question:

Why do search engines consider invisible text to be spam?

Where's the art in applying yourself parrot fashion (invisible text - that's spam) to a list of rules? It appears to be in finding things that aren't written on the list at the instant that you look at the list. So, if there was such a large and contantly changing list (as new techniques were discovered or became available), what would you do if a technique was declared to be spam the day after you'd implemented it for a client?

Presumably, you think search engines should write this list. What factors do you think they should take into account when they decide what spam is? If you want to be part of the same industry (as Doug says), don't you think you should agree on those factors? Do you think the White Paper would be a good starting point for that agreement?

Are you starting to see the pointlessness of the "big book" approach? Search engines have a simple reason when they label something as spam. SEOs need to know the simple reason, not an infinite set of example applications of the reasoning. The White Paper gives you the reason. It also tries to bring a bit of balance to the way search engines (reportedly) interpret spam. A hypothetical example...

A robot requests the same URL with its user agent, and with a Mozilla user agent. If it gets two different results, that must be spam, right? Wrong! The whole reason the user agent field was invented was to allow variable content to be delivered to platforms and users with different capabilities. Is it any wonder that variable content is delivered?

Hence I still maintain this: Rules aren't want you need. You need principles, in every sense of the word. As yet, no-one has proposed an objective set of principles that counter those in the White Paper. I don't expect that because it requires a lot of work and thought, and I have a few years' march on you. But, if you go back through this post and answer every question in it for yourself, can you finally answer this question to the forum with a reasoned answer:

Are the principles in the White Paper principles that search engines and professional SEOs could agree upon evaluating any particular technique as spam?

Alan

[ihelpyou]

The search engine's measure of how well a particular resource matches the input search criteria

I would change that definition of "relevancy" to:
The continuing struggle between a search engine which wants to produce results based on what the searcher is actually looking for, and the searcher who wants real good results.
A search engine robot seeing a HTTP series response or short META refresh should follow the redirect to the target, without indexing the source.
This is true, but Google says it does not follow the redirect at all right now. They are looking into it.
* It isn't my spam if somebody else did it outside my control - so penalise them, if anyone, not me.
I disagree with that statement. If this is a client saying this about his SEO, then both the client and the SEO should be penalized. The client should take careful screening measures before hiring any SEO.

Okay Alan,... Overall, I certainly agree with what you are trying to accomplish here and applaud you. I do believe that it is waay too sweeping.

I also disagree with the IP and agent based cloaking. Since no one wants to take the time to view every page indexed by the engines, there will be no way to discover who is actually spamming and who is not spamming based on your definition. As a consequence, neither one of them should be allowed. Unless, of course, the Googlet Team wants to hire a department to do nothing but this.

This is such a broad-based topic full of different means, ways, and excuses. It would seem that based on this paper, any spammer could wiggle his way out using a number of excuses.

I believe the only way for a paper to fly would be if Google wrote the paper and consulted with who they know as Ethical, honest SEO's in delivering this information on spam and relevancy.

This kind of seems like a paper produced by someone who is Not a SEO? I do not know what you do or even what your web site offers... have never looked at it, but what is your main thing on the net?

I agree with what you are trying to do and with your general principals, but not the actual delivery.

Very good idea Alan, and one that should be pursued in some way with much more input from Google at the start. Actually, they should be the ones who write it with someone like me to consult. :)

[Advisor]

It's principles we are trying to establish. So, you agree then? But will you stick to your principles in practise? Alan,

I would have to say that for the most part, I DO stick to those principles in practice. My problem with wholeheartedly agreeing with your white paper is that it is not up to me to judge other people's techniques of SEO. If it works for them, then so be it. Eventually, if what they're doing is truly spam, it won't work any more becauase the engines won't like it.

When people ask me about how to do certain techniques that I don't agree with, I will tell them that I don't believe they should use those techniques because the search engines may consider it spam.

As to ethics and principles, well, everyone has to live with themselves are are responsible for their own ethics and principles. What is *right* for one person, may not be right for another. That doesn't make them an unethical person necessarily. Again, it's not for us to judge others. Well, at least it's not for me to judge them. People are different, people have different morals. Different doesn't mean bad or good. Plus, we will never be able to make principled people out of those who simply aren't principled.

Not sure if I went a bit off track here or not, but since you brought up principles, I felt I needed to give my feelings on this.

Jill

[Alan Perkins]

Jill


Not sure if I went a bit off track here or not, but since you brought up principles, I felt I needed to give my feelings on this.


You are right on track. Establishing your own ethics and principles is an essential part of the process. The question is, do your ethics and principles apply to search engines? I think they have to. SEO work has no direct contact with searchers. The contact is indirect, through a search engine. Therefore, you need to treat a search engine with respect. Let it operate how it sees fit in order to deliver what it thinks is relevant. If it is wrong, it will not survive. But give it a chance.

The first time I met you, in Dallas about a year ago, there was a guy you called a spammer because he cloaked. I would bet when you pitch for work that you sell yourself as a non-spammer, whereas some of your competitors are spammers. We can't all go round calling each other names unless there is a common set of principles. And there will be. The question is, common to whom?

BTW, I worded my last post a little harshly. Sorry about that. I've toned it down a bit since. Hopefully you only read the toned down version. Basically, where it now says "If the answer to that is No, " it did just say "No?" Same meaning, but it looked a bit harsh as a single word question.

Doug:
Sorry but the definition of relevancy is not at issue. Search results are ordered by relevancy. Relevancy is by definition then the search engine's measure of how well a particular resource matches the input search criteria. Whatever else it might be doesn't matter in the context of defining spam.

In fact some major search engines used to put relevancy as a percentage after the ranking position, e.g.

1. [95%] Al's Diner
Come to Al's Diner for a great dining experience.

They stopped doing that when they figured out that SEOs were using the relevancy factor to find out what techniques worked best. Sort of self-defeating.

Both the client and the SEO should be penalized. The client should take careful screening measures before hiring any SEO.
I agree that with evidence of a relationship between the client and the SEO, both should be penalized. Without evidence, you need to be very careful. Think guerilla tactics.

Alan

[MazY]

Originally posted by Alan Perkins
What are you all saying exactly? Do you want an enormous book or Web site itemising every possible current instance of spam. Do you want this book or site to be added to or modified every time something new crops up? If that's what you're asking for, then

a) you're not going to get it, and, more seriously
b) you haven't persuaded me you even know what spam is

OK I tried not to get drawn in but...

I'm not entirely sure any individual has to convince you that they "know what spam is", for any reasons whatsoever. The notion of you expecting everyone to agree 100% regarding what spam is is almost as ludicrous as anyone expecting a a book or a site to be added or modified anytime a new method pops up.

There are things that Doug considers spam, for example, that I do not. Who is right? Neither party. Unless, of course, all search engines agree with either one of us. Then by definition that the main objective is to create relevance as defined by them, within their guidelines, we have a conclusion.

Yes, I do believe that spam becomes more clearly defined when a search engine tells me it is, according to their guidelines.

That is not to say that I agree with them all the time but it does mean that they have put their cards on the table and I can choose to play with them or against them using my own personal definitions. The latter will do neither me, nor my clients any favours. So who else, if not the search engines, should I let define spam for me? You? Hell no. You can be as professional as you like. That doesn't mean that I trust you enough to make those definitions for me.

I choke enough at your statement that your white paper tells me the reasons why search engines consider some things to be spam. Did they help you write it then? If not, then it is not the reasons that search engines consider it to be spam at all. It is your definition and assumption of their possible reasoning.

As for nobody proposing an objective set of principles that counter those in the "white paper" - are we reading different threads? I see people doing exactly that. Just because they haven't termed their views under the hood of a white paper doesn't mean their own personal principles, borne from their experiences, knowledge, etc. are less valid than your own.

You actually had me with you for the most part of the thread until you started to turn it into a "I am considerably more experienced, gifted and professional than you are so I must be right." scenario.

Sadly, at that point, you lost my support in full. If you really are that insular then no matter what people say in here, your views will not adjust. So what exactly are you looking for? A challenge of views or converts to the Alan Perkins school of thinking? If it's the latter I'm afraid to say that my freedom of view and expression is not for sale, no matter how experienced, professional and so forth the buyer may or may not be, or, dare I say it, how many conferences they have attended and spoken at. I once spoke at a Microsoft conference, doesn't make me a Windows expert!

Hate to be so brutal in my honesty but I really do react quite violently at having people thrust their own beliefs on to me and then assume me to be less a mortal or less professional if I don't agree with them. Not true, not logical and not at all professional! It's starting to read less like a "can I have your views" and more like a "why aren't your views the same as mine - an industry leader with the most professional view......"

I dare to add that you are guilty of breaching relevance yourself. Is the topic about your article or about you? If it's the latter I'm afraid it holds little interest to me. I'm sure yer a nice bloke and all that but hardly feature in my daily life. Whereas the subject matter does.

Personally, I would like to see a white paper on why the field of SEO is so rife with, excuse me a moment, egomaniacs who are all clambering like children to be seen as the number one top-dog (sponsored product placement :D ) for the whole industry.

Out of this entire forum, I can think of only two people I know well enough through reading their work, exchanging views with, seeing their credentials (note seeing, not having them thrust down my throat) etc, that they could make me alter my work and views by them suggesting that something may be wrong. In fact, one of them has done just that in the past

The bizarre thing is that all these voices shouting "I'm the best" are all very nice, but it isn't the likes of us you have to coinvince, it's the clients. Talk about wasted energy and efforts.

[Mel]

Hi Alan:

The nub of my argument and yours is the same in some ways - i.e. that what we should be discussing is the principle of relevancy and spam, not the details. That was my objection in my first post in this thread; that I found the basis of your thesis unsound and therefor saw no point in discussing the details. I see now that I should have stuck to that rather than trying to answer your detailed questions, but it is a principle in this forum that we try to answer all questions.

By YOUR definition SERPs are ordered by relevancy, but not by mine. By mine SERPs are ordered by ranking.

"Doug: Sorry but the definition of relevancy is not at issue. Search results are ordered by relevancy. Relevancy is by definition then the search engine's measure of how well a particular resource matches the input search criteria. Whatever else it might be doesn't matter in the context of defining spam

Sorry Alan but I thought the whole point of this discussion was to see if the members of this forum agreed to your definition of Relevancy and Spam and now here you are saying that the definition of relevancy is not open to discussion????

I am not willing to give up the perception that relevancy is a truth, not a set of rules that differs from search engine to search engine. Look up the definition of relevancy in any English dictionary and tell me if you can find a definition that includes anything about search engines. If, however the only definition of relevancy you are willing to discuss is your own idea, then discussion seems pointless. I have rather reluctantly provided my definion of relevancy, but it appears that you either did not read it or decided that is was beneath your notice, as you did not comment.

Now it may be that SERPs can be in some limited ways be characterised as a particualar SEs idea of relevancy but even that is fraught with danger since many of the SERPs we see are not relevant and/or are not ranked accurately.

Since you state that your definition of spam depends on your defintion of relevancy, how can we discuss spam without first agreeing on what relevancy means? And we most certainly have not agreed on that definition.

I most especially do not agree with the idea that search engine rules define relevancy. GoTos (Overture) implementation of relevancy is then "He who pays the most is the most relevant" I don't think you will find GoTo saying that they will give you better relevancy if you pay more, they only promise better ranking - there is a difference.

Regarding Spam, I have no idea where you decided that anyone here wants a thick book of Spam rules; the only one suggesting that seems to be you.

I take offense at the suggestion that anyone not agreeing with you is either unprincipled or at least not as well principled as you. You are welcome to your own set of principles, and I am willing to listen to any reasonable discussion on ideas that might change my views, but I refuse to accept that you or anyone else has the right to determine if I am principled or professional or honest. That is a determination I have to make daily and live with.

Are the principles in the White Paper principles that search engines and professional SEOs could agree upon evaluating any particular technique as spam

I have answered that in my previous post, but Again "NO I do not agree with the principles of either your definition of relevancy or spam."

[Mel]

Hi all:

After rereading my last post I see that perhaps I may create some confusion since the term relevancy is commonly used in the way Alan proposes. I would like to expose my ideas on this subject a bit more clearly.

As an example. Suppose that I am a famous sports star and I decide to write a book on my life, accomplishments, and my thoughts and feelings about a variety of subjects - much of this informtion has never been published before. I then decide to publish that book on the web, but I know nothing about search engines so I just dump the text of my book into the pages chapter by chapter and walk away - no submissions to SEs or directories, no optimization, not title tag, pages 15000 words long etc etc.

I dare say that this book might rank very poorly (if indeed it were ever indexed by SEs) but that does not make it any the less relevant to the subject of me. In fact by the english definiton of the word relevant, this is the most relevant site on the web regarding me. But it will not have high SERPs.(rankings)

The point here is that relevancy and rankings are very different fish, and we should be careful to differentiate between them. If we have to know, understand and use fully the ins and outs of search engines before a unique site written by an expert is considered relevant then we are in trouble regarding relevancy.

Search engines strive to reach the Nirvana of relevancy in their results, but that does not equate to their results defining relevancy

[MazY]

You know, whenever I read a post from Mel, it is like reading a post from my long-lost twin Brother....

Nice post, Mel.

[Alan Perkins]

Mazy:

Thanks for your comments. To answer the nub of your question, yes, I am working with search engines and SEOs towards building a consensus. The views in the White Paper are not just my own. They are built on years of dialogue with SEs and SEOs. I have designed and programmed search engines (not Internet-scale, but the issues are the same) and I have been an SEO.

Maybe my comments did sound a bit pompous, but when you are called naive you have to offer your credentials somehow. Otherwise all the experts on the list will refuse to debate.

Why am I doing this? I don't know, is the honest answer. Hopefully some good will come of it. I'm just passionate about it. Sometimes that passion inspires a heated debate. I have no problem with that, if we can stick to the issues. Everything I say is, of course, In My Opinion. That's the whole point of a discussion forum.

Mel:
"SERPs are ordered by ranking." Rankings are just numbers (e.g. 1 to 10). The reason the search engine results are in the order they are in is because they are in order of the search engine's perceived relevance of each resource to the search criteria. This is too fundamental to debate.

All:
Why do search engines consider invisible text to be spam?

[Mel]

Hi Alan;

Well......... I must certainly disagree with you that the definition of relevancy is whatever the search engines say it is.

This is not too fundamental to debate unless you can provide some clear and recognized authority backing that statement. In fact, it is the heart of your thesis and if you are unable to defend your definition, then your entire argument is shattered.

Your statement that it is too fundamental to debate is not only arrogant it is extremely self-serving. I challenge you to provide a link where a major search engine says this, failing which I think all must agree that it is just your opinion and no more.

It seems that perhaps you are not familiar with the definition of the term relevancy here's how Merriam Webster defines the root word (relevant):


1 a : having significant and demonstrable bearing on the matter at hand b : affording evidence tending to prove or disprove the matter at issue or under discussion <relevant testimony> c : having social relevance

A further definition of relevancy from the same source;

something relevant

Here are the synonyms from the same source:

Entry Word: relevance Text: Synonyms USE 3, account, advantage, applicability, appropriateness, fitness, service, serviceability, usefulness, utility

Sorry my tone is so harsh, but I do not take glady to condescension, or those who practice it when they have no answer to the question.

[MazY]

Originally posted by Alan Perkins
All:
Why do search engines consider invisible text to be spam?

Alan.

Firstly thank you for taking my comments on the chin, as it were. I know many who get upset at my somewhat direct and brutally honest approach. Respect to you for that.

On to your question. My feeling is that you are barking your question at the wrong forum. No individual can possibly know why search engines consider invisible text to be spam, only that a great many of them do.

Further, it is somewhat immaterial to the SEO, why they in fact consider it as spam. The point is that they do. Whether you, I, Mel or any other SEO like it, that is the case. We then, as individuals, choose whether to abide by that decision or go against it and try our luck.

If you were to bully me for an answer as to why they consider it as spam I would say that it is because it offers the site reader absolutely no benefit or experience whatsoever. Search engines are, at the end of the day, there to provide the most relevant results. Results that the user can gain benefit from. They do not gain benefit from invisible text, only the webmaster gains from that. I feel quite sure that the webmasters are very low down on the "must please" list of the major search engines. Visitors and users must maintain top priority.

There should be more symbiosis but alas there is not. It is, in actual fact, very parasitic on both parts. We provide the content that the search engines have their relevancy judged upon. They, in turn, dictate to a large degree, how our content must look and feel in terms of layout. Having said that, those that are willing to abide by their definitions of spam get rewarded with an extra avenue of traffic.

[Mel]

Hi all Sorry for the last harsh post, but I gave my reasons for it.

In the clear light of reason I have tried to explain the difference between relevance and rankings (which as Alan expresed may be considered as a particular search engines perception of how relevant those results are for the search term in question)

Ok Maybe I am off track somehow, if so someone please look at the following and put me straight.

Search engines try very hard to provide relevant results; on that point I agree and it is an important one.

Each search engine has its own (varying) definition so that each search engine returns different results.

Each engine returns those results in the form of rankings which are (usually) numbered from 1 to however many results it returns.

The search engine does not assign a relevancy number to each result but instead assigns a ranking to each result. This ranking is not an absolute measure of the sites relevancy in that a site with a relevancy of only 5% might be able to get a #1 ranking on one search term, while for another term it may require a 95% relevancy for that exalted ranking. Thus the order that the sites are ranked in is based on the perceived relative relevancy of that site for that search term based on the pages in that index, but which in no way establishes the absolute relevancy of that site.

This is further complicated by the fact that the SE measures the relative relevancy based on how well you have complied to their set of rules. This results in many very relevant (meaning sites which contain a lot of information relevant to that search term) sites not receiving high rankings because they have not ordered their site according to that SEs (somewhat unknown) rules. Other sites use less than ethical methods and receive higher rankings even though they do not contain as much relevant infromation.

This results in the rankings being in an approximate order of relevancy according to the SEs rules, but not in an order of absolute relevancy, and will contain many anomolies errors and downright omissions, some departing quite a ways from the factual. Further you can have virtually the same information on your page, make a few tweaks, and you will get higher rankings, but the relevancy of your page will not increase at all.

What is then my defniniton of the relevancy of a page? Well, I normally don't run about making definitions, but I am sure it should bear some relationship to how much useful information the user finds on that page.

In summary I hold that search engines deliver rankings to users, who should be the sole judge of how relevant that page is for their use. Thus I very much like to see a more precise use of the terms ranking and relevancy.

Please feel free to critique the above and shoot me down anywhere you feel I am wrong.

[MazY]

Sorry, my ol' mucker. I tried to find something that I disagree with but I cannot.

[MakeMeTop]

Why do all search engine view invisible text as spam?

Frankly, I think that is because it is pretty easy to detect by even the newest SEO and webmaster and it encourages them to 'rat' on each other's sites while the SEs laugh like drains. If Google really had a hidden text ban, why are so many sites in their SERPs with hidden text, hidden links etc., etc. No, they just wait for people to turn in the sites with hidden text - as they know will happen. No need for them to bother with it - they turn their technology to and spend their money on what they truly believe is spamming Google - which is manipulation of PageRank (much tougher to detect if you haven't used link-farms). Pretty much anything else is (to Google) irrelevant or self-policing. A invisible text ridden site will sink or rise due to PageRank - not any tricks. Other SEs are far better at detecting hidden text than Google and react to it far quicker than Google.

I've rarely seen a site which has used hidden text on Google to be a site which is not relevant to the search term - but people will turn them in regardless for what I regard spurious reasons ie. they are hoping to manipulate the results in their favour by removing a competitor!

Why don't SEOs use invisible text? Because it is likely to get a site banned by being reported by their competitors - not through some ethical consideration.

SEOs 'manipulate' results. We can call it anything else we like, 'make sites search engine friendly', 'sites ready for submission', 'tidying HTML', but it is all designed to get our clients higher rankings than their compititors - I still maintain the only common ground we have between ourselves and SEs is the area of relevancy which I defined earlier - and that is the basis on which we can build the foundations of principles for this industry. Anything else is never going get agreement - SEs will always say that they want 'clean' results based on their formula which sorts relevancy - SEOs are paid to ensure their clients appear in those listings for terms which are relevant to their client.

Of course any SE is going to agree with Alan's definition - are they really going to say 'yes, manipulating results is fine'? No.

[ihelpyou]

Very true. It the paper is handed to Google for their approval and to "sign", but of course Google would approve it. BUT, if Google actually wrote it, it would be written in a much different way. This is why any paper on this should be written by the engine while consulting with others. After all, it will be their rules to which we all have to follow.

[MakeMeTop]

I agree with you, Doug. But it does need a prolonged dialogue between the major SEs and SEO representatives - if such a thing is possible. Hopefully that is what organistions like WAIM are trying to set up - but it is going to be tough. Not only are SEs trying to fight gross result manipulation (and no-one should forget that huge spam factories churning out thousands of spam pages per day and spending big bucks are their real problem - not really us 'small fry') - but also each other for market share. End of the day - user satisfaction is what will help pay the SEs bills and they are always going to try and keep how they sort results secret from SEOs and other SEs.

[ihelpyou]

Yes. Dialogue would be very important between the SE's and the SEO's. I am not so sure about WAIM however because I was under the impression that WAIM was made up of a lot of internet marketing people who really do not know or practice SEO? Maybe I am wrong, but there is a big difference between marketing companies who know a lot about email advertising, ebook, viral, etc, etc, but do not go into the SEO aspect of things. Some say they do, but truly do not.

I know that IM encompasses ALL and EVERYthing to do with the internet. It is every correspondence and dialogue that we have everyday with anything we do. But a paper written by the search engines that has to do strictly with ranking and relevance on the search engines, should be a dialogue between true SEO's and the search engines. Not by companies who do SEO as an after-thought and on the side.

[Mel]

Hey Doug:

I looked into WAIM and there are a lot of well known SEOs in the group of founding members including Danny Sullivan, Detlev Johnson. etc etc, but was taken aback to see who runs it -

The WAIM organization from the 1st of January 2001 is currently governed by one President, Lennart Svanberg, and a board consisting of Jim Wilson and Jonathan C. Cohen.

[MakeMeTop]

WAIM is about two thirds SEOs and one third other forms of internet marketing currently, but each area of expertise is trying to draft the standards for their area and open communication. Certainly, everyone who is involved on trying to put down the methods of opening dialogue with the SEs is an SEO with decent experience of our industry with some of them being reasonably well known.

[ihelpyou]

Yes Mel. I knew that at the very start. I am also a member of WAIM but have not done anything because of One who runs it.. Danny and Detlev are great!

Actually, I would find it hard to be a participant just because of that fact. He is strictly an internet marketeer BTW, not a SEO. Never has been.

[Mel]

Hi Barry and Doug:

I very much agree that the SEs and the SEOs will always be at opposite sides of the stream, and will occasionally throw a rock or two at each other.

If this ever changes, SEO will cease to exist as we know it. As you said Barry, it is our job to 'manipulate' the results in our clients favor, but this is not all bad for the SEs as they get better sites with more relevant information, better navigation and with overall fewer surprises for the viewer and more useful information more closely matched to the SEs ranking.

The users also benefit, so I view it as a very useful and productive occupation. Too often the poor user is left out of the equation when he is really the reason for SEO.

[Advisor]

The first time I met you, in Dallas about a year ago, there was a guy you called a spammer because he cloaked. Yes, I remember this incident. But, no, I don't believe I called him a spammer. What everyone was calling him was a "cloaker." Not spammer. In your mind, you probably equated cloaker with spammer. I, in fact, would have referred business to the cloaker had I received a potential client who absolutely wanted a company that would do cloaking for their site. I would first, of course, try and talk them out of it, and explain what I consider the "right" way to optimize, and also explain that they could get banned for life, however, if they had their hearts set on cloaking then, that's up to them.

As to me calling you naive...I don't believe I actually called you personally naive, as I know you are not. What I was calling naive was the *notion* that everyone will agree on what is and isn't spam.

As an aside, I simply can't agree with your statement doing something strictly for high rankings being spam. For example, I built my Rank Write site without much thought to the search engines, originally. We just wanted to get it up and running so we could start the newsletter. Months later (close to a year really), I had some time and decided to spend a little bit of it on optimizing the site for the search engines. Was I happy with the site the way it was originally? Sure, it was fine for it's purposes and for the people who visited it. However, it wasn't fine for the search engines. In your definition, the work I did on it subsequently would be considered spam, because I did it solely for the purposes of achieving high rankings. Oh sure, I could probably come up with some *legitimate* reasons for doing some of the stuff I did that was not related to getting high rankings, but they would be lame excuses as everyone knows the real reason why I made the changes.

The bottom line is that my site is extremely relevant to the phrase "search engine optimization" and I don't think anyone would deny that. Therefore, my making certain changes soley for the purposes of getting it found for those terms is absolutely NOT spam in my opinion. But if one were to strictly follow your definition, it seems to me it could be put in that category, even though I didn't do anything that any engine would currently consider spam. Nor would any other person in the SEO field that I know of consider it spam. I just used regular SEO techniques that I've always used. But it seems that your paper is saying that because I did them for high rankings (my sole purpose) then it's spam. Sorry, can't agree with that and never will!

Jill

[Advisor]

I very much agree that the SEs and the SEOs will always be at opposite sides of the stream, and will occasionally throw a rock or two at each other. I really don't think that the SEs are at opposite sides when it comes to most of the SEOs at this forum. Since the bulk of the people who post here are of the highest ethical standards, the SEs probably welcome us with open arms. It is those companies that Mel mentioned earlier in this thread that have had to keep them on their toes. Those that churn out hundreds and thousands of pages in order to manipulate the results. We are not the ones causing problems; we are actually producing the types of sites the engines want.

I think perhaps, this is the reason why we at this forum are having so much trouble with the white paper to begin with. It's really not intended for us who are currently already practicing "good" honest methods of SEO, but for those others.

Jill

[ihelpyou]

Good post! And there is the whole problem with the paper. Waay too sweeping and all encompassing. I would have to say that these forums have some of the most upfront, honest, and ethical SEO's that one would ever see. Even with that, we simply have a few problems with the paper the way it is spelled out.

I think anyone would agree that many members here are NOT spammers. Just the fact that we are SEO's would deem all of us as spammers according to the paper.

[JuniorHarris]

Great discussion by all!~ And some very good point/counter-point posts by Alan, MazY, and Mel! You guys really got the noggin thinking this morning. Basically I don't define spam, as my definition makes not one hoot of difference. As it is what each engine has defined as spam that governs what is acceptable or not.

[The Ney]

Wow!

This was so clear, so to the point and so not beating around the bush. Thumbs up! :)

I so agree with Mel's definition of, not only relevancy and ranking, but also with the description of the corelation of these on SE.

But if we turn to the starting point of what is spamming (if i am correct, i havent read the whole thread), and if i connect it to one of the threads that concern the new Yahoo algo, called by some "the spamming algo", there are several question marks over here. I think that it was Doug that said that Yahoo's algo is a spamming one, because now everybody knows how to optimize for Yahoo and the rankings have nothing to do with relevancy (or something like that, Doug i apologize already if i am misquoting you, that is what i understood then).

Well, first of all, in this thread there seems to be a general agreement about the fact that high ranking does not always mean high relevancy and i agree with that. As Mel pointed out so clearly, ranking is a matter of complying to SE rules (and according to my understanding, one and only thing SEO company has to do for its clients) and merely achieving those is certainly not spamming. Even if you bring to #1 place the site that is not the most relevant site for that field. However optimization becomes spamming if you use decieving methods, such as serving the SE and it's spiders one thing, while all the others something else AND COMPLETELY IRRELEVANT (used the caps to emphasise the fact that cloacking is not spamming by rule).

Unfortunately (for the searchers) relevance is another thing that is not measureable by machines (read: spiders). There are SEs that do it better than the others, but none of them are doing it 100% like humans. Not even close. If they measure popularity (clicks) then those that came first are usually those with most clicks (not the most relevant ones). If they measure links, than those with most friends in the field concerned are ranked high (not necessarily the most relevant ones). If they measure title- keywords-description-keyword on page saturation, they rank higher only those that are optimized in a best way.

SO.... to make the short story long...;). We, the SEO gang, are not spammers if we optimize. We are just playing according to the rules that SE set for us. We are spammers if we lie, decieve, steal, and do all the things that deep deep inside of ourselves i believe we know are WRONG, in order to achieve higher rankings for us or our customers. And it is our responsibility to stay out of these wrong paths.

I apologize if i said things that are common knowledge, or have insulted someone or if i have wasted everyone's time.
Its just that i am still pondering about that Yahoo thread and since both here and there spamming was discussed, i thought that... but.... ok, i'll go now....

[Alan Perkins]

I've been away all day so I've tried to catch up with the discussion. It appears you think its for SEOs to be told what spam is, but not why something is spam. I guess I'm surprised at that, with all the talk of cooperation (particularly from Doug). However, MazY took a giant leap towards my position:


MazY
If you were to bully me for an answer as to why they consider it as spam I would say that it is because it offers the site reader absolutely no benefit or experience whatsoever.

Thanks for bullying yourself. Take your response, and extend it to every other technique, whether or not search engines currently label that technique as spam. Things like using heading tags for things that aren't headings...

I chose invisible text because everybody "knows" it is spam. The idea of the White Paper was to let people know why.


Barry
Why do all search engine view invisible text as spam?
Frankly, I think that is because it is pretty easy to detect...

Yes, but why detect it? Why is it wrong? MazY has answered that. Do you agree?


Jill
you probably equated cloaker with spammer

I do indeed. MazY's answer points to why. The White Paper spells it out in detail. If you don't equate the two terms, then what is your problem with cloakers? Why did you make a crucifix with your fingers, hold it up to the poor guy and hiss "Cloakerrrr"????

Mel:
Search engines try very hard to provide relevant results.Agreed.

Each search engine has its own (varying) definition so that each search engine returns different results. Agreed

Each engine returns those results in the form of rankings which are (usually) numbered from 1 to however many results it returns. Agreed (close enough to my opinion)

The search engine does not assign a relevancy number to each result but instead assigns a ranking to each result. The search engine does assign a relevancy, but these days does not usually publish it.

This ranking is not an absolute measure of the sites relevancy in that a site with a relevancy of only 5% might be able to get a #1 ranking on one search term, while for another term it may require a 95% relevancy for that exalted ranking. Thus the order that the sites are ranked in is based on the perceived relative relevancy of that site for that search term based on the pages in that index Agreed

but which in no way establishes the absolute relevancy of that site. There is no such thing as absolute relevancy. Relevancy is subjective. When we define relevancy, we have to say in whose opinion. In order to rank highly, you need to be relevant in the search engine's opinion.

This is further complicated by the fact that the SE measures the relative relevancy based on how well you have complied to their set of rules. Agreed (although I would say "algorithm" instead of "set of rules")

This results in many very relevant (meaning sites which contain a lot of information relevant to that search term) sites not receiving high rankings because they have not ordered their site according to that SEs (somewhat unknown) rules. The SE "rules" are not really unknown. They are based on a whole other topic, called information architecture. If you write good, well structured, accessible Web pages, you should rank quite well.

Other sites use less than ethical methods and receive higher rankings even though they do not contain as much relevant information. Agreed!

This results in the rankings being in an approximate order of relevancy according to the SEs rules, Agreed if you delete "an approximate".

but not in an order of absolute relevancy, There is no such thing as absolute relevancy.

and will contain many anomolies errors and downright omissions, some departing quite a ways from the factual. Agreed. This is because search engines are only a very short way down the infinitely long path to perfection.

Further you can have virtually the same information on your page, make a few tweaks, and you will get higher rankings, but the relevancy of your page will not increase at all. This is very important. If we are talking about the relevancy to the end user, then agreed if the tweaks are spam, not agreed if the tweaks improve the information architecture or accessibility of the page or site

Relevancy...should bear some relationship to how much useful information the user finds on that page. Agreed.

In summary I hold that search engines deliver rankings to users, who should be the sole judge of how relevant that page is for their use. You call it rankings, I call it a list of results. Agreed. The search engine has calculated the relevancy of each result in the list, and ordered the list on that relevancy. The user can then judge the search engine. If the user consistently agrees with the search engine, then great, they have found a search engine they can use. If there are many such users, the search engine should survive and continue moving along the path to perfection ... or, at least, perfection for those users.

Thus I very much like to see a more precise use of the terms ranking and relevancy. Agreed.

Please feel free to critique the above Done.

[Advisor]

If you don't equate the two terms, then what is your problem with cloakers? Why did you make a crucifix with your fingers, hold it up to the poor guy and hiss "Cloakerrrr"???? It was a joke. And I'm not saying that I don't also equate cloakers with spam. It's just not up to me to make that determination, in my opinion.

J

[ihelpyou]

Cloaking:

Too much temptation to stick in irrelevant material. All cloaking should be banned unless the engines can actually check each and every one. Otherwise, how does anyone know if it is meant to deceive or not?

[Advisor]

Originally posted by ihelpyou
Cloaking:

Too much temptation to stick in irrelevant material. All cloaking should be banned unless the engines can actually check each and every one. Otherwise, how does anyone know if it is meant to deceive or not? Yes, but apparently Inktomi allows cloaked pages through their paid inclusion program (or so I've heard). Therefore, how can it be spam where Inktomi is concerned? That's what I mean by it's up to the engines to deterimine what is and isn't spam, by what they allow.

If you always use relatively *normal* and innocuous SEO techniques like I always have, you won't have to worry about what the engines might or might not call spam. Obviously there are some companies who don't care if they spam as they will do things that the engines have publically stated is spam.

Jill

[MazY]

Thanks for bullying yourself. Take your response, and extend it to every other technique, whether or not search engines currently label that technique as spam. Things like using heading tags for things that aren't headings...

No problem. Always time for a bit if self flaggelation, in the textual sense of course.

Using heading tags for things that aren't headings:

Strictly speaking, my personal definition (not the search engine's) is that if it doesn't add to the "visitor experience" (TM 2001 MazY Enterprises) and the page would be just as legible or appear the same without it then it qualifies for my definition of spam.

What is the point in using an H1 tag for text that looks exactly the same as the normal text? Pure attempt at deception to suit the search engine rankings, not the visitor. Visitors should always, always and always again, come first. Ergo, why I try to push the barriers of web design a little, in respect of what the popular opinon is on what the search engines will rank highly and what they will not. I won't expatiate my views on the "lean and mean" theory any more as they are already well documented.

Somebody (MakeMeTop?) cited the w3.org as an excellent example of a site that the search engines will love. Maybe it is, but from a user perspective it is worse than ugly, it is boring and uninviting. (Though fine for its own particular audience.) It would not suit any company that I have ever worked with and I would not even consider anything like it for a company site. What is the point of high rankings if the user doesn't like what they find? A totally ineffective and illogical pursuit. Especially when it isn't strictly required to be that way.

Another issue that was raised within the thread was reporting spam. I for one one have done it. I have no problem with it at all. Is it because I want to remove the competition, as was suggested. Not even crosses my mind. (Though a bonus all the same.) The reason why is that I happen to share with the search engines, their quest to make things relevant but within their guidelines.

I should stress that I only report SEO companies that are cheating. Why? Because it looks bad on our industry as a whole. The less there is of it, the better for all.

I do agree with MakeMeTop that Google really don't seem to care about spam at all. Eventually, I trust that their software will filter it out but I am sure they have more pressing issues. However, whilst they leave an avenue for me to report it then I will use that avenue. Whether they do anything about it is beyond my control. Though invariably they don't. In fact, I think it is one of the worst engines there is for spam.

Anyway, I went totally off the subject matter. Sorry and all that, chaps and chapettes.

[ihelpyou]

hey Maz, just looked at your "daily-news" and the missing D's. :green:

[MazY]

LOL Keep up, Doug.

It's been there almost a week.

[MakeMeTop]

>Yes, but why detect it? Why is it wrong? MazY has answered that. Do you agree?

I am probably being obtuse here (and I have never used hidden text) but I see no real difference between hidden text and the meta keywords tag - except one is 'acceptable' and one isn't. However the tag is 'correct' HTML and can't be banned (there would be too much of an outcry) but has now been downgraded in value so much that I barely use it. In fact I advise people 'if in doubt - leave it out' if they are unsure as to what words to put in. The reason it has been downgraded/ignored or (with hidden text) banned is because it has been used by many, many people to insert words/phrases which are totally irrelevant and are designed to capture users looking for something entirely different to the site they wind up on. Not because it enhanced relevancy. They banned it because THEIR USERS COMPLAINED about this. If the users had been getting totally relevant results and there were no complaints, do you think they would have banned it?

I get an average of 30-40 sites per day which we are asked to check. 85% of them spam keywords, have hidden text and put in terms that are completely unrelated to their sites - particularly competitive brand-names and trademarks. The people who have these sites are not spammers, their web designers have done this in the mistaken belief that it works - and because (in the short term) it often does.

If I take on a site and correct this so the site appears for terms that are relevant to the site - yes I have hopefully enhanced the user experience when they go from the search engine to the site - but I have used my 'skills' to boost the sites rankings on relevant terms and demote them on irrelavant terms. In my definition I have prevented spam and rewarded the user plus enhanced the quality of data in the search engine. In your definition I have become a spammer as my main task is to get higher rankings (that is what I'm paid for) even though I will only do it on terms and with methods I feel are acceptable. Search engines (of course) may totally disagree!

Search engines are using the relevance test when they are checking for spam (if you have paid inclusion). They are not interested in how you get the rankings (within reason) but are interested in you serving up worthwile, non-misleading results which are useful to their users - ie. is the term the surfer has looked for clearly visible on the page they land on and in keeping with the theme of the site. (I do know this - as I've had a site tested) I always make sure that this is the case - but I'm sure you would say my methods of achieving rankings are probably spam.

[Alan Perkins]

MazY:
I agree with all your "on topic" statements. I think we are in agreement! Does this mean that the White Paper is a fair representation of your opinion?

Barry:
"I am probably being obtuse here " - You are, because you have avoided the "why" question. Stop wriggling! Why do search engines consider hidden text to be spam?

"I have never used hidden text". I think you admitted to cloaking, didn't you? If you cloak, you don't need to make text invisible, because the whole page is invisible. How can the cloaked page be relevant to end users if it can't be seen by end users?

"I see no real difference between hidden text and the meta keywords tag". Hidden text is within the <BODY> tag of a HTML page, and is therefore in "an area" designed for uers to see. The keywords tag is within the <HEAD> tag of a HTML page, and is therefore not designed for users to see on-the-page. The White Paper differentiates Content Spam from Meta Spam. It also reaches the conclusion that it is impossible to spam using the keywords tag. I know that isn't the current opinion of many search engines, but the White Paper follows it through and it's the only logical conclusion I could reach.

Jill:
"Cloaking" is a loosely defined word with lots of history behind it. In the White Paper, I have attempted to pin it down very precisely. Inktomi does not allow cloaked pages using my definition of cloaking. How do you define cloaking when you say that Inktomi allows it? A statement from an expert that Inktomi allows cloaking gives refuge to spaammers who can apply their own definition of cloaking.

Point out to me somewhere on the Web where Inktomi (not someone else on their behalf) says that they allow cloaking...

Alan

[MakeMeTop]

I'm not going to get drawn into the cloaking/anti-cloaking debate ;) Suffice it to say that it is completely unneccessary to cloak a site where it is possible to work with the designers/site owner to change the HTML appropriately - and in those circumstances I don't do it, certainly would never do it on a client's own site and never do it without the express written instruction from the client. I also prefer not to cloak as I actually quite like people to see my wonderful copy writing :)

I also seem to remember you and me sitting in Stockholm when Joe Frost from Inktomi was asked (with other SE representatives) if they object to cloaking where it was used to index an otherwise unindexable page and was relevant to the users search. He said that this could be a valid use - but if abused would result in severe penalties.

Hidden text is banned because it has caused searchers to get irrelevant results. If SEOs and SEs can get the relevancy issue out of the way - then is the time to lay down rules of 'best practice' and start talking about the fine points of what is spam. Until this first issue is put to bed - all we are doing is trying to get 'ethical' SEOs to accept rules which are consistently breached by most web designers on a routine basis and spam farms constantly. We are not the bad guys (IMHO). We attempt to create relevant results for users and good listings for SEs - many others just plain want to get their sites to come up for anything and stuff the SERPS with their results.

[Mel]

Hi All:
I would like to continue the saga of my sports stars (Lets call him Joe) journey of discovery into the the world of web pages and search engines.

Our faithul readers will remember that he had published his great book on the web, but the search engines and users could not seem to find it.

Joe also had his book published in paperback by Hack & Kouph and was pleasantly amazed that the initial printing sold out all 58 copies in only ten days - the problem was that now others wanted to read this hot new book, but were having trouble finding it.

Joe turned to his faithful agent Carl and said "You get 10% of everything I make, so get out there and find a way for readers to find the web version of my book, so that when the new printing comes out they'll buy" Well, Carl had no idea how to go about this but he did have a bright young nephew, Jack, majoring in computer science at the University of Virginia.

In short order Jack reported back that he had found this place called Ihelpyouservices right there in Charlottesville where you could ask all sorts of questions and learn how to please the search engines for free, so Carl logged on and started to learn. He was mindboggled at the amount and complexity of information available there, and soon decided that he would turn the task of spiffing up the boss website to one of the regulars there who seemed sensible.

And so it happened that Madam J. was entrusted with the task of making Joes website visible to the world. J. immediately sensed the worldwide potential of this masterpiece and so called on that renowned continental master of lexicon, Prof M.Y. to consult.

In less than a week (Oh yes, Joe was paying top dollar, since he was a rich footballer with an equally rich and beautiful wife) this dynamic duo reported back to Joe that his site was a mess, but they had repaginated it and reduced all the pages to less than 450 words length, had added some funny things called metatags and titles to each and every page, had converted the numerous footnotes, endnotes and the bibliography references to something called hyperlinks, and were recommending that there should be some modifications made to the wordings, in particular that the word soccer should be replaced with football every now and then as Prof M.Y. had discovered that in Europe that was what Joes sport was called. While they would give him no guarantees they said that they thought his site would now become very visible and easy to find if Joe would agree to paying a few hundred dollars for submission to directories, and they would include the submission to the search engines in their already considerable fees. They also promised that the completed online book would contain the same information and wording as the printed one (with only minor wording changes).

And so it happened, in a fortnight Joes site was the talk of the sports web and soon started to garner high rankings and awards. Joe was a convert to the web way of doing things and if you visit his site today you'll find an Ebook version of his bestseller (yes, the next printing of 1834 copies sold out too, and the book is now in its 5th printing) as well as an Amazon link so that you can buy right there online and have this great book delivered right to your door.

BUT, then Carl reported that all those things that Madam J. and Prof M.Y. had done to his web were now being called spam on the I helpyouservices site! Joe is now in a quandry and so I call upon you, gentle reader, to make the determination if this is so or not.

[Mel]

Hi Alan:

I'm still stuck on the differences between ranking and relevancy, and in particular your insistance that the SERPs are ordered by relevancy.

The SERPs are produced by what I believe the search engines themselves call a "ranking algorithm" . Now I wonder why they don't call it a relevancy algorithm? Could it be because the search engines themselves make no promises regarding the relevancy of the SERPs returned and so use the term which describes what it really is - a "ranking" algorithm?

[Alan Perkins]

Paraphrasing Barry, "Joe Frost said that the use of cloaking to index an otherwise unindexable page could be valid."

In the very precise way I have defined cloaking, a page is always indexable without using cloaking. There is no valid use of cloaking.

I have tried to be precise in order to offer the opportunity to do work that is useful to some human, somewhere, on some platform. To do otherwise is spam. But let's not digress into cloaking. Let's stay purely on the topic of spam.

Mel
I'm still stuck on the differences between ranking and relevancy, and in particular your insistance that the SERPs are ordered by relevancy
Imagine there are only two cola sites in the world, coke.com and pepsi.com, and two search engines, SE #1 and SE #2. A user searches for "cola". SE #1's algorithm determines that coke.com is 95% relevant to the search term, and pepsi.com is 76% relevant. SE #1 ranks coke.com at number 1 and pepsi.com at number 2. SE #2's algorithm determines that coke.com is 89% relevant to the search term, and pepsi.com is 92% relevant. SE #2 ranks pepsi.com at number 1 and coke.com at number 2. Which of the two SE's is right? Who can say. There is no such thing as absolute relevancy. Which of them will be the more popular search engine? Depends how many people think coke.com was the "right" answer to the question, and how many think pepsi.com was "right". Expand that example up to Web scale and I hope it shows the issues.

This thread is now getting very long and I think we've lost the focus as we've gained insight into each other. Can I suggest we wrap it up by answering these questions:

1) Do you think SEs and SEOs should agree on a definition of spam, or should SEs just tell SEOs what spam is?
2) Should the definition be a long, ever changing set of rules or just a few fixed principles?
3) If it should just be a set of principles, do the principles in the White Paper suffice?

Alan

[Mel]

Hi Alan:


1) Do you think SEs and SEOs should agree on a definition of spam, or should SEs just tell SEOs what spam is?
2) Should the definition be a long, ever changing set of rules or just a few fixed principles?
3) If it should just be a set of principles, do the principles in the White Paper suffice?


1. I think it very hard to agree on a common definition of spam, since each search engine has its own peculiarities, have a habit of changing their ideas regularly, and are continually having to develop new responses to new threats to stuffing their algorithms. Even if we did agree on a common definition, there would be a need from time to time for interpretations, and that would necessarily have to come from the SEs. Thus I feel that SEs do define what spam is by definition.

[MazY]

[i]Can I suggest we wrap it up by answering these questions:

1) Do you think SEs and SEOs should agree on a definition of spam, or should SEs just tell SEOs what spam is?
2) Should the definition be a long, ever changing set of rules or just a few fixed principles?
3) If it should just be a set of principles, do the principles in the White Paper suffice?

Alan [/B]

1. I think it would be nice if SEs and SEOs could agree on a definition but in practise it is never likely to happen.

2. A silly question. It should be in the format that it needs to be. If you mean which is more preferable to me then I would say the latter - a few fixed principles. But again, that is never really going to be the case. The problem with that method is, as has been in the past, it gets abused. This, in turn, cheeses the SEs off and ergo they become more rigid and suddenly decide that the method which was previously fine is now spam.

3. I'll nip off and read it again. I warn you in advance however, I am one hard cookie to satisfy! :)

[MazY]

OK The first hurdle to cross with me:

"Search Engine Spam
Any attempt to artificially influence a search engine's ability to calculate relevancy."

Would like to see it as:

"Search Engine Spam
Any attempt to artificially influence a search engine's ability to calculate relevancy, using techniques that it has stated to be innapropriate."

[Alan Perkins]

using techniques that it has stated to be innapropriate
That's the long list of rules approach. I've been reading back through your posts. You and I are very much in agreement on the principles, I think. So in what way does my wording need to change in order to remain a principle, rather than require expansion into a list of rules?

Alan

[MazY]

Originally posted by Alan Perkins

That's the long list of rules approach. I've been reading back through your posts. You and I are very much in agreement on the principles, I think. So in what way does my wording need to change in order to remain a principle, rather than require expansion into a list of rules?

Alan

It isn't really the long list of rules approach. It is taking what is current and applying your document to that fact.

I'm not suggesting that you then list each set of rules, one by one. That would be nonsense. But your statement as it is, just totally alienates the entire SEO industry, spammers or not spammers.

If you can resolve that then I think I can put my stamp on it. Though I'm still only about 75% of the way through. <Goes to read the rest.>

[highman]

Sorry to jump in here....I have posted this in another thread just before reading this page

How do you all feel about this:



Think of a user without flash installed..... what would they see?

From the macromedia site you can get a bit of code to detect the flash plugin.

Build a non-flash site, change the default page on the server to your new html page, add flash detection to this sites pages, IF flash is detected then direct the user to the flash site.

Spiders dont have flash installed

Warning: Make sure you direct away from the text / html site and not from the flash site!

Now get a PC, uninstall flash and look at your site

Some people have sugested this may be a form of cloaking, but, well its more 'catering for all users'


Am I a spammer? <gulp>

[ihelpyou]

But your statement as it is, just totally alienates the entire SEO industry, spammers or not spammers.
That is what I am hung up on as well.

I also believe the engines dictate the rules. Anyone else doing it would not be as enforced. The engines should write the "principals" with consultation from a few.

We can see how tough this really is. With every point made, there can be a counterpoint. With every "rule", there can be an excuse.

With the way the paper is written, I am a true spammer and am unethical. I doubt the majority out there thinks this.

[MazY]

OK Alan

Having read it all with an open mind and with no limit of time to restrict my understanding, yes, I can say that we agree on about 99% of the content.

Just that small matter mentioned above. If you can do something about that then I'm sold!

Sorry guys. It is a fundamentally sound document that really does subscribe quite a bit to what I believe the majority of us believe, in essence. There are just a couple of poorly or innapropriately worded definitions that are just bound to cause alarm.

However, I do like the simplicity of "if there were no search engines, would it be done?". It makes sense.

[Alan Perkins]

Alienating SEOs is not what I want to do. I think the issue is the word "artificially". I should clarify what I mean by it.

Long ago in this thread, I qualified "artificial" a bit by saying that if you could find any reasonable excuse for doing something, then it would not be spam. That's not to say, in this imperfect world, that you wouldn't be penalised for spamming. But that, if you were penalised (and could somehow find out..) that you could present a reasonable excuse for doing it and thus have the spam penalty removed... That's the principle.

I can't think of a search engine that says on its Webmaster help pages that inappropriate use of heading tags is spam. But you and I agree that it is spam. If an SEO receives a spam penalty for it, should their defence be "But you don't list that technique on your Webmaster help pages"? No. It should be "Look, I did that because..." and finish the sentence without saying "I wanted to rank higher on your search engine." If there is no other reason for doing it, it is spam.

OK, I guess you're looking for the role of SEO. Well, it's everything you do now, unless you are spamming. You are spamming if the modifications you make to pages or sites, or the new pages or sites you create, are purely for increased rankings and do not in any way add to the "visitor experience". Look at the fundamental problem your client has, and it's likely to be one of information architecture or site accessibility. In short, the role of the SEO is to fix the fundamental problem.

Alan

[Mel]

Hi Mazy:

well...... I too agree that it is a simple statement but I think it is too simplistic.

"If there were no search engines" there would be no web pages (or very few at least) and so by my thinking if there were no search engines I would not publish web pages at all, and therefor the very act of publishing web pages could be defined as spam, even though there there is a nice escape clause to wriggle out of the strict definition.
To define an important concept such as spam in regard to terms which are not applicable to the real world is not only dangerous, but not logical.

Sorry but when I try to carry this defintion to its logical conclusion it results in chaos.

[MazY]

Originally posted by highman
Some people have sugested this may be a form of cloaking, but, well its more 'catering for all users' [/i] Am I a spammer? <gulp>

Following from yesterday's post about my own very personal strict definition of spam, I would ask this:

Does it add to the "Visitor Experience"? - MY new company buzz phrase for 2002. (As can be seen on the two new sites.)

My answer is yes it does add to the visitor experience. In that without it, they would have a little ugly mess where your Flash animation once lived.

But, I would also encourage them to download the plug-in to avoid further instances of it so that they can get the full "Visitor Experience". Boy am I pushing that phrase! lol

So, no. In my strict defintion, you are not a spammer but a caring webmaster.

[MazY]

Originally posted by Alan Perkins
You are spamming if the modifications you make to pages or sites, or the new pages or sites you create, are purely for increased rankings and do not in any way add to the "visitor experience". Alan

I better get that TM on my new buzz phrase pretty quickly! :D

Ya think I could have The "Visitor eXPerience"? LOL

[Alan Perkins]

Highman:
No, you are not a spammer, as long as the non-Flash version of your site is designed to be used by humans who, for whatever reason, don't have Flash. Many SEOs would call you a spammer, parrot fashion, for using "redirection" technology. If they do, point them at the White Paper.

All:
Help me out. I sense we all mostly agree on the principles. Write the one line defintion of spam that you would be happy with and which would not alienate all SEOs.

Alan

[MazY]

Originally posted by Alan Perkins
Highman:
No, you are not a spammer, as long as the non-Flash version of your site is designed to be used by humans who, for whatever reason, don't have Flash. Many SEOs would call you a spammer, parrot fashion, for using "redirection" technology. If they do, point them at the White Paper.

All:
Help me out. I sense we all mostly agree on the principles. Write the one line defintion of spam that you would be happy with and which would not alienate all SEOs.

Alan

:D I think you are a long way from getting full support. With me you are about 99.9% there. Just that one issue remaining which I have already made my suggestion for.

Mel:

I understand your point about that if search engines didn't exist then people may not create web pages. So let me ask it in a slightly different manner. What if they did exist but didn't apply any kind of ranking? Things that were relevant were listed to the user in a totally random manner?

Oh, Lord. I'm starting to sound like Alan now! ;)

[ihelpyou]

Our client's problem is they cannot get ranked on their targeted keyword phrases so they hire us to help them. Since this is why we exist, of course we may do things to improve ranks. We also improve the information architecture and site accessability. Let us not kid ourselves with this.

No matter what we agree to on that paper, there will be those who certainly do not agree and will abuse it.

[Mel]

Alan:

My respect for you increases the more we discuss this, but you still do not convince me.

How about: Any attempt to artificially influence a sites ranking on a search engine?

[MazY]

Originally posted by Mel
Alan:

My respect for you increases the more we discuss this, but you still do not convince me.

How about: Any attempt to artificially influence a sites ranking on a search engine?

Isn't that what is says now, in essence? And don't we all do that now? or are you looking at it from the SEO/Relevancy as opposed to SEO/Ranking aspect?

[Mel]

Maz
I'm sort of looking at it from the SEO/Ranking point of view as a first approximation. I know that when I am optimizing a site I do no think in terms of increasing the sites relevancy but in increasing its ranking.

Then we need to define "artificially" which is where we may run into stiffer problems.

In fact after rereading Alans paper for the fifth time, I can see that if the word "relevance" was substituted by "Ranking" the whole thing would go down easier. I hold that the determination of relevance should be reserved for the user.

SEs may use whatever methods they want to try to determine what the users like and order their results by rankings, but the relevance of a particular page is in the eye of the beholder.

I'm still thinking about "if there were no search engines...." idea - I can see the theory but when I try to imagine the application I run into problems.

[Alan Perkins]

Doug:
"No matter what we agree to on that paper, there will be those who certainly do not agree and will abuse it." Agreed. But I want the White Paper to be of value to professionals like yourself, not spammers like them. I know you want search engines to write it, but a search engine would not (maybe even could not, for various reasons) engage with you the same way I have here. If you want a consensus opinion, this is the way to get it.

Mel:
I can agree to your definition because, as MazY says, it is essentially the same as mine. I will consider defining rankings before spam in the WHite Paper, so I can use your definition. Did the cola analogy help when defining the relationship between relevancy and rankings?

MazY:
Every one of your recent posts is improving my visitor experience to this board. :D

Alan

[MazY]

Alan - quit using my "Visitor Experience" :D <Calls the TM office>

More seriously....

Might I suggest posing the same question that I asked above. Regarding the search engines returning results in a totally random manner. This, I think, negotiates Mel's valid point about it being a ludicrous suggestion as pages would perhaps not be developed.

The next most obvious question is, subject to you ironing out the creases, what exactly are you hoping the paper to do or achieve? Where are you targeting it, if at all? How are you going to push it out to the big wide world, if at all? Etc.

[ihelpyou]

Here is another problem. Take this page for an example:

http://www.merimage.com/

IMO this is spam. The problem is with your paper that there is a way out for the spammer by simply saying that the design is pleasing to the user, and if I put words visible on the page it would not be as pleasing anymore. The hidden words are also relevant, hence, Not spam at all.

You see? It is bad spam IMO, but according to the paper, it is not spam because they have an excuse.

[MazY]

Originally posted by ihelpyou
Here is another problem. Take this page for an example:

http://www.merimage.com/

IMO this is spam. The problem is with your paper that there is a way out for the spammer by simply saying that the design is pleasing to the user, and if I put words visible on the page it would not be as pleasing anymore. The hidden words are also relevant, hence, Not spam at all.

You see? It is bad spam IMO, but according to the paper, it is not spam because they have an excuse.

I totally disagree with you there, Doug.

I don't think that the document gives any webmaster a way out. Show me anyone that could possibly agree it adds to the "Visitor Experience" (Whoops there it is again. ) :D

And why would it not be as pleasing without it? That makes no logical sense. If you can't see it then how can it not being there make it any more or less pleasing? <Scratches head>

At the end of the day, the determined spammer will find an excuse for anything. Even if the document were 100% watertight. The question is whether experienced and ethical SEO professionals agree with that excuse, given a basic and fundamental definition of spam.

[Mel]

Alan:

No the cola analogy did not help much either the first or second time I read it, since there are lots of extraneous things in there.

As you say it is the user who will vote for the usefulness of the ranked page to him (relevancy???) by using that search engine or not, which may in turn have an effect on the survival of that engine, and they may change their way of doing things if the users are not satisfied with their results.

But ultimately the search engines exist to serve users (OK this is somewhat of a circular argument, but there is lots of that going round) and the users definition of what is useful or relevant to them is what is important. Therefor to define what should or should not be done without considering the user does not sit right with me.

It is up to both the SEO and the SEs to help give the user what he wants, thus if the SEO uses some trick or technique to bring to the users and search engines attention a very useful site then that should not be considered spam. I know I know, how do we control and measure that?

[ihelpyou]

I agree with you Maz, but that is what the site would say. That is their way out according to the paper. That is why they are doing it in the first place. We know they are doing it to get words indexed by the engines. They could say they are doing it because their pretty graphics and the purpose of the front page would be distracted from.

[Alan Perkins]

Mel/MazY:
I'm going to put some concentrated thought into the "relevancy/ranking/spam" debate. I'll get back to you on that.

Doug:
There are principles, then there are applications of principles:

(SE):
merimage, I am penalising you you for spam.

(Merimage):
"The design is pleasing to the user, and if I put words visible on the page it would not be as pleasing anymore. "

(SE):
It looks to me like you have put the words on the page solely to improve ranking without improving the visitor experience at all, In fact, if anything, you have worsened the visitor experience for a considerable set of visitors. The words are incoherent and not designed to be read by any human visitor to the site. If a human visitor without graphical abilities or font tag support visited the site, they would receive a page of garbage. Our blind users have complained about the way their text-to-speech converters have reacted to this page. Why do you think it is more important to design a page for our robots, rather than those humans? We still contend that this is spam.


Alan

[ihelpyou]

Without "listing" a bunch of rules, maybe the paper could simply include a few techniques of spam we all agree IS spam? This way things like the above would not have an "out"?

[ihelpyou]

I agree with that Alan. What if the hidden text was completely coherant if it was visible? What if it was in nice paragraphs and the copy was very good? IMO it would still be spam, but the site would have an out.

This is what I mean.

[Alan Perkins]

Doug
IMO it would still be spam, but the site would have an out.

IMO it would not be spam! It would be a poor attempt at making the site more accessible, but it WOULD make the site more accessible. See what I mean? What you describe would actually make the site more useful to a set of visitors to that site. That is why we have to be very careful using phrases like "invisible text is spam".

The question then moves on to "In determining relevancy (i.e. ranking, Mel) how much weighting should a search engine give to invisible text, versus visible text"?

The answer to that is one for the search engines. But I believe it is incorrect to label well structured, coherent good copy as "spam", if there is a set of human visitors that can derive value from that text.

Alan

[ihelpyou]

Tell me again which user would benefit from the invisible text in the example above?

I find it hard to believe that kind of thing could ever not be considered spam and is only used to influence ranks.

[MazY]

OK. This may be a long one so forgive me in advance if it is.

I just read through the entire thread again and decided to address again some points that have been made.

When a client calls me wanting high rankings, I do not instantly think right, how can I make their site rank higher. I do instantly think, right what can I do to make their content more relevant to their field of business. My own experience tells me that this will generally work in improving their ranking. Given that the SEs are generally quite good at determining their own definition of relevancy.

An interesting thing is that very few of us optimise for a specific engine and yet I cannot believe I am the only one that has enjoyed good rankings from most of the major SEs with just one page. So, we can determine that to an extent, each SE has a "relatively" constant definition of relevancy. I say "constant" with a degree of caution.

Now as Mel points out, just because the SE determines a page to be relevant to the phrase typed in to their engines, this does not mean that the user will feel the same.

What we can be certain of is that by the use of hidden text etc, the readable content of the page does not become any more relevant. Ergo, it is redundant to the visitor. Further, if we assume that there are only two parties to satisfy, the search engines and the visitor, then we must logically deduce that its [the hidden text] sole purpose is to deceive the search engine spiders. Spam.

So does the addition of the hidden text make the content any more relevant to the subject matter? If one wanted to be technical then yes it does. Does it make the visitor experienced content any more relevant? Absolutely not. Does it benefit the visitor in any way? Mel may say that the information that is readable may be of great interest and relevant to the reader's search phrase and may not have been spotted were it not for the hidden text. (Forgive me if I have misunderstood you, Mel.) His point, if I read it correctly, is again correct. However, one must draw the base line somewhere. Something which I feel the Spam White Paper does. A level playing field is what the search engines try to provide from which to have their quality judged. I think the white paper does a reasonable job at clarifying how level that playing field is or is not and how far one can kick the ball around without hearing the ref's whistle.

[ihelpyou]

So you would then also say that the invisible text could also include invisible links that lead to higher PageRanked sites or simply links leading to a page the site wants indexed and not actually visited or links leading to anything. Simply because the copy is relevant and coherent, none of this would matter?

[MazY]

Originally posted by ihelpyou
So you would then also say that the invisible text could also include invisible links that lead to higher PageRanked sites or simply links leading to a page the site wants indexed and not actually visited or links leading to anything. Simply because the copy is relevant and coherent, none of this would matter?

Doug

I thought I was quite clear. I would say that invisible links do not add to the "Visitor Experience" (TM - 2001 MazY Industries) and so serve no purpose for the point of the page - to provide relevant content to the reader.

[Advisor]

IMO it would not be spam! It would be a poor attempt at making the site more accessible, but it WOULD make the site more accessible. See what I mean? What you describe would actually make the site more useful to a set of visitors to that site. That is why we have to be very careful using phrases like "invisible text is spam". So, Alan, are you saying that a site that used invisible text that was well-written and on target to the site, is not spam? And if so, then wouldn't a cloaked page that also uses relevant, well-written text that only the search engines see, also NOT be considered spam? And what about a page using the zero frame, with relevant, well-written copy in the noframes tag? Is this also not spam?

I personally agree that some of the above examples may not be spam if they are completely relevant and the company's in question have a reason why they do not want to (or can't) put said copy on their page. But I didn't think you would agree according to your paper. Perhaps I'm misunderstanding something. I must admit, this conversation is definitely getting a bit too deep for me!

Jill

[ihelpyou]

No Maz, I was commenting to Alan. :)

I find it extremely troubling that any hidden text is not spam simply because it is coherent and well written. Even when it has hidden links, etc.

[MazY]

Hang on, I'm confused. If I read correctly, Doug said that what if the text were visible, well formatted and with very good copy? Alan is right in my mind. It then ceases to be spam and just a desperate attempt at making things more relevant.

Maybe I'm just getting confused by the term "good copy"?

[Alan Perkins]

Let's not digress too far on this invisible text example. I would say that search engines should not place too much emphasis on invisible text and invisible links when determing relevancy/ranking. I don't want to get technical on you by showing you some sample lines of a relevancy calculation algorithm. Think instead of a weighted opinion. In the absence of any visible text anywhere on the web, then coherent, well structured invisible text should rank above no text, not just be dismissed out of hand as spam. A text-only browser with no support for the font tag will display that text (e.g. a PDA), and users of that text browser will be happy to read it and get something from it. A text-to-speech converter will read the text out loud and blind users will be happy to hear it.

It is not spam if it is well structured and coherent copy, valuable to some human users. I know that's difficult. I'm having fun getting that across to the search engines themselves! But, for me, any attempt at improving accessibility is better than no attempt.

To give you an analogy, a similar scenario exists with frames/noframes content. Not many browsers these days support noframes, but some do (like the two I have mentioned above). If you put spam in the noframes section, users of those browsers get garbage. If the noframes content is well structured, coherent good copy, then it is not spam and the users get something of value. However, noframes copy will probably be ranked lower than if the same copy was put on a page that was more likely to be seen, such as a page on a frameless site.

Eventually, maybe, search engines will be able to tailor their results according to the platform, language, location and ability of their users. Then "invisible text" may be given a greater weighting in the instances when a search engine detects that a particular user will be able to see that text.

Alan

[MazY]

Then the simple answer is for the search engines to not demerit a site for invisible text but just not to allow it to feature within the ranking algorithm.

After all, your points about why it should not be discounted are viable. But, there is absolutely no reason in the world why it cannot be made visible!

[Alan Perkins]

Jill:

"Are you saying that a site that used invisible text that was well-written and on target to the site, is not spam?" Yes! Whether it ranks well is another matter. But the search engine can see it, interpret it as invisible, and make its own judgement as to how relevant it is. The fact is, it is relevant to some of the search engine's users.

"And if so, then wouldn't a cloaked page that also uses relevant, well-written text that only the search engines see, also NOT be considered spam?" No! Because it does not improve the experience of ANY of the search engines users.

"And what about a page using the zero frame, with relevant, well-written copy in the noframes tag? Is this also not spam? " Correct, IMO.

Alan

[MazY]

Hang on, Alan. Yer losing me again.

One can't realistically think of a web with no text and use that as a base to reason with. That is simply not the case, never has been and never will be.

You are going to have to do a little better than that. We can all pull extreme hypothetical arguments from the air but they must be based on realistic reason. At least they must if they want me to consider them.

[Alan Perkins]

MazY:
"Then the simple answer is for the search engines to not demerit a site for invisible text but just not to allow it to feature within the ranking algorithm. " Demerit it if it is spam. Little or no merit (but no demeriting) if it is not spam but only appeals to a small subset of the SEs audience.

Alan

[ihelpyou]

All invisible text is spam no matter what ya all say in my book.

Again, you would then also have to say that all invisible links are NOT spam as well.

I am troubled by this immensly and cannot agree with you at all on the paper if this is what you are saying.

Gee whiz. NO SEO ARE NEEDED then. Right? All any page has to do that does not want text to be seen is to simply make it invisible. NOT right.

All we or you have to tell sites is to simply use invisible text and not bother to hire an SEO to help. Why would we be needed then if all invisible text is okay? If all invisible links are okay? As long as the invisible text is readable by that 1% of people, then it is just fine.

[MazY]

Originally posted by Alan Perkins
MazY:
"Then the simple answer is for the search engines to not demerit a site for invisible text but just not to allow it to feature within the ranking algorithm. " Demerit it if it is spam. Little or no merit (but no demeriting) if it is not spam but only appeals to a small subset of the SEs audience.

Alan

Right so that is something else we agree on. I think. :D I'm getting a headache now!

[highman]

So, no. In my strict defintion, you are not a spammer but a caring webmaster.

Phew ;)

[MazY]

Originally posted by ihelpyou
Gee whiz. NO SEO ARE NEEDED then. Right? All any page has to do that does not want text to be seen is to simply make it invisible. NOT right.

All we or you have to tell sites is to simply use invisible text and not bother to hire an SEO to help. Why would we be needed then if all invisible text is okay? If all invisible links are okay? As long as the invisible text is readable by that 1% of people, then it is just fine.

Wrong. Both Alan and I have said, repeatedly that it is spam by search engine defintions. However, there "may" be instances where it is justified. Though I sway a little on that as I fail to see why any text that is invisble cannot be made visible if it has a valid purpose. However, we have both said that the "invisitext" would not gain any benefit in the ranking system. Though to be be fair, this was clarified as you were typing your response I think.

As Alan quite rightly pointed out (Am I starting to sound like the Alan fanclub?. Whack me a few times if I am.) the search engines can read the invisitext. What they do with it is under their control. I believe what is being proposed is that it just doesn't matter whether it is there or not as it won't get any benefit in the rankings.

Given that latter scenario, it certainly adds no benefit to the Visitor Experience and so, SPAM!

[ihelpyou]

The only way it would not be included in the algo as a boost is if the engines reprogramed the robots completely. I do not think that would happen anytime soon.

Invisible text is spam.

If not, you are saying that as long as all cloaking cannot be seen by the user, then it is not spam as well. I disagree completely.

Cloaked pages are and will always be subject to abuse. If you allow it, then the engines would have to hire a "cloaking" department so they could review each and every cloaked page to make sure it was relevant to the site's pages which are seen by the user.

[MazY]

OK. let's turn it on its head a moment. So what if the SEs suddenly announced today:

"We believe that there are some users that may benefit from invisible text on web pages. Ergo, we will now feature invisible text within our algorithms to determine relevancy."

What you going to do? I know what I am going to do. Nothing at all. Because I still feel that if it can be written invisibly then it can be written visibly and add to the visitor experience of the majority of my visitors.

[Alan Perkins]

MazY:
I don't want to lose you ... so tell me what you mean by

"One can't realistically think of a web with no text and use that as a base to reason with"

All I'm saying is that the use of font tag with the same colour as the background page, or a noframes, noscript, or div tag, or an alt, title or longdesc attribute of another tag, is never, in and of itself, spam. It is very often used to hide spam. There are some visitors on some platforms who will see the information in those tags. And if it improves their experience, then the tags are worthwhile.

The issue of how much weighting the search engine places on those tags is a separate issue. But if you listen to some SEO experts, you will be put off making your site more accessible in fear of being labelled a spammer.

Myself, I would never use a font tag with the same colour as the background page. But I might use agent-based delivery, which is a superior way of doing much the same thing. E.g. I would deliver a graphical page to platforms I knew supported graphics, and a text-only page to other platforms. The important thing is not to deliver content solely designed to improve search engine rankings.

Alan

[MazY]

Originally posted by Alan Perkins
MazY:
All I'm saying is that the use of font tag with the same colour as the background page, or a noframes, noscript, or div tag, or an alt, title or longdesc attribute of another tag, is never, in and of itself, spam. It is very often used to hide spam. There are some visitors on some platforms who will see the information in those tags. And if it improves their experience, then the tags are worthwhile.

Alan

OK. Right I'm back with your train of thought now. But we must accept that it is spam according to the majority of the search engines. But I do understand what you are saying.

[ihelpyou]

That is right Alan. That is exactly why agent based or IP based is not right.

What you are saying is that a SEO is not needed as a site could simply use agent based cloaking at all times. Puts things up for abuse and again, the engines would have to hire people simply to review each and every page that is cloaked.

I think I read in the WmW thread that you offer software which is agent based? Maybe I am wrong here, but I read something about that awhile ago.

[Alan Perkins]

Doug:
You will not find a more fervent anti-cloaker than me. You should have been in San Francisco! But I also fiercely protect the use of HTML to make a site more accessible. Spam makes a site less accessible, by taking accessibility features and subverting them purely to achieve rankings, delivering garbage to people who needed the site to support those features. I hate spam.

Alan

[MazY]

Given Alan's last response, I found myself asking myself why I wouldn't use "invisitext". The answer to be is very simple.

More people would benefit from me making it visible. That simple. All platforms can read it when it is visible so why bother to make it invisible?

[Advisor]

Originally posted by Alan Perkins
Jill:

"Are you saying that a site that used invisible text that was well-written and on target to the site, is not spam?" Yes! Whether it ranks well is another matter. But the search engine can see it, interpret it as invisible, and make its own judgement as to how relevant it is. The fact is, it is relevant to some of the search engine's users.

"And if so, then wouldn't a cloaked page that also uses relevant, well-written text that only the search engines see, also NOT be considered spam?" No! Because it does not improve the experience of ANY of the search engines users.

"And what about a page using the zero frame, with relevant, well-written copy in the noframes tag? Is this also not spam? " Correct, IMO.

Alan Ok, then you've converted me!

Jill

[ihelpyou]

All platforms can read it when it is visible so why bother to make it invisible?
EXACTLY. What would be the point to use invisible? No point. No point to use agent based or ip based cloaking for this very reason.

[Alan Perkins]

Doug:

"That is exactly why agent based or IP based is not right." Please read the White Paper. it covers this in minute detail. Agent-based delivery was invented in 1990, along with the Web itself. There are perfectly good, valid reasons for using it, beyond search engines. Therefore, just using agent-based delivery cannot be spam. Using it to deliver unique content to search engines is spam, however.

"What you are saying is that a SEO is not needed as a site could simply use agent based cloaking at all times." No, I do think SEOs are needed. Whether SEO is the right phrase is another question. But somebody who really understands search engines will improve traffic by making the site more relevant (by improved accessibility and maybe by modifying its information architecture to match keyword research) and more obviously relevant (through writing good titles and descriptions). There are lots of other roles for the SEO, e.g. handling directories, PPC campaigns, etc. Personally I think focusing on rankings is inappropriate and I've never run any rank checkers, but that's just me.

"I think I read in the WmW thread that you offer software which is agent based? Maybe I am wrong here, but I read something about that awhile ago." No, I don't offer an agent-based piece of software. I wasn't on WmW to offer our software, and I'm not here to offer our software.

Alan

[ihelpyou]

Jill, I am not sure you know what you are agreeing to here. :eek: You are saying by agreeing that you are not needed at all. Any site can simply use agent based cloaking or hidden text that is coherent, and do away with you.

[MazY]

Originally posted by ihelpyou
No point to use agent based or ip based cloaking for this very reason.

I agree about IP based but there are some very valid reasons for user-agent that can significantly improve the Visitor Experience.

Try to remember that I look at these things from a designer aspect too as well as an SEO basis.

My menu on the two new sites, for example, uses a "sniffer" JavaScript to detect the browser that the visitor is using. It then customises the menu (Still HTML code, well DHTML to be precise.) so that they get to see exactly what they should see. Without this small script then believe me, their experience would not be a quality one. As it is, I can cater for NS4.0 and upwards, Opera and IE. Without it? Imagine....

[Advisor]

Originally posted by ihelpyou
Jill, I am not sure you know what you are agreeing to here. :eek: You are saying by agreeing that you are not needed at all. Any site can simply use agent based cloaking or hidden text that is coherent, and do away with you. Not at all, Doug. You and I could be done away with anyway, if every webmaster got it in their head finally how to create a good, information-rich, easy to navigate site. But we all know that will never happen. Our jobs are secure!

Jill

[ihelpyou]

huh? You either sell agent based software or you do not sell it. From your answer you said

No. I don't offer agent based software. I am not here to offer OUR software.

Do you sell the software?

Okay. WHo do you propose will monitor EVERY web page on the net that will suddenly be using this agent based software? Who is going to make sure that all hidden pages made by this software complies with the rules? To much room for spam and abuse.

[MazY]

Originally posted by ihelpyou
Okay. WHo do you propose will monitor EVERY web page on the net that will suddenly be using this agent based software?

Perhaps the same person who handles all the compaints from the millions of people who are sick to death of seeing blank spaces, faulty layouts, dead animations, etc, etc, etc, because the webmaster couldn't be bothered to cater for their browser and point them to a page which would better satisfy their experience? ;)

[ihelpyou]

No one can tell me that this software that hides a page from that visitor to be read by a robot would NOT be built with the search engines in mind.

If this paper is going to allow this software across the board then the engines would have to hire MANY more people than they have now to police just this software.

[MazY]

Originally posted by ihelpyou
No one can tell me that this software that hides a page from that visitor to be read by a robot would NOT be built with the search engines in mind.

Maybe that is the problem? Nobody can tell you. Your views seem pretty fixed and that is ok. Mine were too at the start of reading this thread. However, logic and reason have big impacts with me and Alan has offered both in his defence. That is all I ask from anyone.

Remember, any technology that has a valid purpose can be abused. There is no way on God's earth that you are ever going to change that. It does not, however, mean that the negative use should negate the positive uses.

[ihelpyou]

At least the way I do SEO can be easily seen by ANY user at anytime. I do not hide anything. If one wishes to see the code, they simply "view source". Anyone can do this. NOT everyone would be able to view an agent based software built page. Too much room for abuse and would have to have an army of SE people simply to review every page.

[MazY]

So hang on then Doug. Both HighMan and I have "confessed" today to using a form of user-agent detection. Does this mean we are to be shot at first light or commended on our good webmastership in giving our visitors a better experience on our sites?

According to your rational, we are both spammers.

[ihelpyou]

Besides, maybe my views would not be as fixed if Alan were truly an outsider and not involved with this industry. Because he does sell agent based cloaking software, this cannot be the case and no matter how hard one may try, views could be a little biased.

Alan seems like a GREAT guy and is very logical and can express himself very well. Just like I am biased in protecting my ways of doing SEO, Alan cannot help to be a little biased in his agent based software.

[ihelpyou]

No Maz, not saying that at all. I know what and why you are doing that.

What I am saying is that there would be too many ways for abuse by simply letting every page on the web use this software to build pages that are search engine friendly.

Again, the engines would have to hire an army to view every one. There would be no policing like there is now from regular viewers. The engines would have to police.

[Alan Perkins]

Doug:
Chill out. I know from earlier in this thread that our opinions are quite well aligned. I am not a spammer and I don't think you are a spammer.

I know the White Paper is quite big, and it's difficult to hold it all at once. That's why, at heart, there are just a couple of principles to fall back on. Here is the key sentence it contains for the state of mind you are in at present:


"...consider why content spam is possible. It is possible because the same URL can deliver different content (or the same content displayed in different ways) to different visitors to that URL. Even the simplest versions of HTTP and HTML support this, and therefore offer the opportunity to deliver spam. For example, IMG support and ALT text within HTML means that image-enabled visitors to a URL will see different content to those visitors that, for various reasons, cannot view images. Whether the ability to deliver spam results in the delivery of spam is largely a matter of knowledge and ethics."


I used ALT text as an example. Agent-based delivery is another example. You cannot lump agent based delievery and IP cloaking together. Agent based delivery was invented before search engines, and has perfectly valid uses beyond search engines. IP Cloaking was invented simply to trick search engines.

A year ago, when I had this discussion on WmW, I left with the opinion that cloaking and spam needed to be defined very much better. The White Paper is the result.

We do sell software, yes, but not agent-based software. You imply that I am here for commercial gain but I am not. I will tell you what we do - edit this out if you think it is too self-promoting! We used to be in the SEO "game" and developed our own tools to help us. We discovered that other SEOs liked the tools. We moved out of the "game" and started training and consulting for blue chip clients on search engines, information architecture and design for accessibility. We started selling the tools and we use them in our training sessions and for our consultancy work. We don't actively seek SEO work any more. Our product sells under a different brand to SEOs, small businesses and Webmasters.

Alan

[Advisor]

Does this mean we are to be shot at first light I say we shoot him. :eek:

Doug, I don't think Alan sells cloaking software. He's as anti-cloaking as one can get!

Jill

[Advisor]

Doug, I believe that agent based delivery just helps tell if someone's using an older browser, and if so, makes the page more readable for them. For instance, you may have a navigation menu that works only with newer IE browsers, but still want your menu to be seen by those with older browsers and the search engines (which are like older browsers) so you put in some code that enables this. This is a common practice, I believe. In fact, even the noscript tag works this way. On my new dhtml menu that Maz created for me, I wanted to put in the noscript tag the actual text version of the menu so that it would be accessible to all. You don't believe that something like this is spam, do you?

Jill

[ihelpyou]

Okay. Since this will help the thread, what is the Url to this software? What is it exactly?

I still content that anyone can view my work whenever they wish. With cloaking, of any kind, the engines would be the only ones that would be forced to review it if all cloaking is always allowed. There is too much room for abuse. At least with the way I do it, anyone can see it. Not hidden from anyone.

[MazY]

Originally posted by ihelpyou
At least with the way I do it, anyone can see it. Not hidden from anyone.

But in fairness nor are you a designer who has to contend with the multitude of problems and imlications that get thrown at you with todays multitude of platforms, plug-ins and visitor expectations.

I can't help thinking that if you were then you would see things a little more broadly. Apologies if that sounds patronising.

[ihelpyou]

NO Jill, I do not.

Not what I am saying at all.

Again. There is room for much abuse that the general public/user could not police like they do now. The engines would be the ONLY ones who could police it if everyone used it.

Of course there is room for abuse with agent based as the robots and people with older browsers are the only ones to see it.

[ihelpyou]

MAZ. You are NOT understanding what I am saying. Did I say that what you or Jill are doing is spam? No.

I simply content that there is room for abuse if ALL cloaking is allowed for every page. Who will police it?

[MazY]

Originally posted by ihelpyou
MAZ. You are NOT understanding what I am saying. Did I say that what you or Jill are doing is spam? No.

Easy, Tiger. We've been here before I seem to remember. In essence, yes you did say that what we were doing is spam. :) Luckily I realise that you didn't mean it that way. So rest easy, Tiger and do as Alan suggested - chill a little. (Sounds weird coming from me!) :D

[ihelpyou]

Listen. I do agree that agent based has to be used in many instances. All I am trying to say is that to have a paper that says agent based is okay is way too sweeping a statement.

[ihelpyou]

hey Maz, I am very fine and very laid back here. :)

I am just very direct when making posts and have always been that way.

[MazY]

That's what I like to hear. I don't do emotion. Gets me all confused and stuff. :) I'm auditioning for the next Spock part. :)

[Alan Perkins]

Doug
I have no problem viewing a page using any user agent I want. There is a lot of software around to do this (I wrote my own, but I don't sell it :D ). Likewise, if a search engine wants to see the same page a Mozilla agent sees, it only needs to use a Mozilla user agent in its request. It does not need to obtain a new IP address, and use that IP address covertly until it falls into cloakers' hands.

You work the way you do because you are working in the search engine industry. But you can't label something as spam when the person that implemented it may have done so before Internet search engines were even invented, or may not care one iota about search engines.

I also have various ways of de-cloaking IP cloaked pages, including Babel Fish, Google cache, etc. I have spoken to search engines about all of them making something like the Google cache option available, to allow self-regulation of IP cloaked pages. They cannot agree, for similar reasons to those Google give for implementing the noarchive tag. If we permit agent-based delivery, but not IP cloaking, it offers the opportunity in future for a self-regulating industry body. Agent-based delivery allows peer review. IP Cloaking does not. The White Paper goes into far more detail than would be appropriate here.

As for our product, the URL for download is

http://www.searchmechanics.com/prepare/

Alan

[MazY]

I downloaded and test-drove the software the other day. Thought it was quite good, for the record. Though the demo seemed very limited.

[ihelpyou]

Okay. I agree with you on the agent based thing.

[Alan Perkins]

Edited at posters request.

[ihelpyou]

oh no. I want you to explain why yours is better? This is Not promotion. I am simply confused as to why you think automated is better than manual? :)

[MazY]

Doug

Clearly Alan doesn't want to sell it here and we should respect that. Perhaps in private may be better?

[ihelpyou]

You have to understand Alan that this forum is unlike any other. If I feel that something will help our members, then POST IT. I do not care whose product, or whose url it is.

I am for "helping".

[Advisor]

In other words, Doug doesn't mind if you sell your product here, as it's relevant to the discussion. This is not like some other forums in that respect. (Don't hit me, Doug, for mentioning other forums!!! :eek: )

Jill

[Alan Perkins]

Okay. I agree with you on the agent based thing.


Excellent! That's Doug, Jill and MazY. A pretty heavy hitting team. Mel's in bed and I would guess Barry is doing some paying work!

I'll tell you the other good thing about agent-based delivery Doug. It gives the IP cloakers nowhere to hide. With agent-based delivery in your toolchest, the only reason you can possibly have for IP cloaking is to prevent humans from reading your pages. That, by definition, is spam.

I'm off to work on that new definition of spam, relevancy and ranking I promised Mel and MazY.

Alan

[ihelpyou]

I am leaving the choice for Alan to explain the product up to him. I edited out the prior two posts.

You are free to do as you wish. Jill is very right, though. :)

[Advisor]

Mel's in bed Yep, that sounds like 'ol lazy Mel ;)

J

[MakeMeTop]

For the purposes of this paper - and to move this forward - I will agree with you on the user agent/cloaking definition and agree (if this comes to pass) there is no room for IP delivery based cloaking.

I'm coming around to Jill's way of thinking on this - so maybe we are getting somewhere.

[Alan Perkins]

OK, apologies for this being somewhat off-topic, but you asked for it!

As an SEO, you want to diagnose, fix and check you have fixed problems with your clients' Web sites before and during your engagement with your client. Search Mechanics will help you do that. It crawls a Web site, much like a search engine spider. It reports back to you, the SEO, what it has found. This can be in the format of overview reports, which you get with the free version, or detailed reports, which you get with the subscription version. A one week license costs $29 and a one year license costs $199, both for an unlimited number of sites.

The reports allow you to diagnose what you need to do in order to make your clients' sites more search engine friendly. You then go away and do it. You then run the spider again, and you can check that you haven't made any mistakes.

Sample reports, for our site, can be found at:

http://www.searchmechanics.com/smsummry.htm
http://www.searchmechanics.com/smreport.htm

Search Mechanics does not create new gateway/doorway pages. It expects you to work with, and improve, what you've got. It does create a "Search Engine Map", which is a preview of the host: or site: lookup you may expect to get had the site already been indexed. Here is an example:

http://www.searchmechanics.com/seindex.htm

Search Mechanics will crawl dynamic sites. It has several ways of avoiding dynamic spider traps, one of which (user assisted crawling) is not open to search engine robots. Once it knows about dynamic pages, they appear on the Search Engine Map. The search engine map, as a static page, can then be used to guide search engine robots around the dynamic parts of a Web site. Use with caution, though!

Once you are certain that the site is "ready", Search Mechanics helps you to submit. We force you to hand submit, taking you directly to all the Add URL pages, populating your clipboard with the information you need to submit so you can just paste it in, and recording what you have done so you can show your clients and avoid over-submitting.

Search Mechanics is always under review and we're improving it all the time according to the feedback we get. The software is self-updating, but you don't pay for the upgrades. You just pay the license fee.

That's enough trumpet blowing. I'm a techie anyway ... if you want to find out more, it's all there on www.searchmechanics.com - useful hints for Web designers, too, e.g self-referencing framesets:

http://www.searchmechanics.com/learn/srf/

Alan

[Alan Perkins]

Barry:

I will agree with you on the user agent/cloaking definition and agree (if this comes to pass) there is no room for IP delivery based cloaking.

That's a big step and I appreciate you taking it.

Alan

[MazY]

All very nice but it's Simpsons time. Now given the choice between reading sales hype or listening to the wordly wisdom of Homer Simpson, which do you think is going to win?

"Get me the number for 911!" - Homer Simpson.

<Runs off to watch Simpsons.>

[ihelpyou]

Looks like a good product Alan. I just do not like things telling me what to do as I do each site in a personal way. I also do not need a submitter thing as very rarely do I submit anywhere, even for a brand new site. Except, of course, to a couple directories.

I can see where some would find this useful as they might not have knowledge of SEO. BUT, that would then leave me in the cold. :) I still think there is and will be a place for the personal service type guy. :) I would also say that what the software may say that is wrong, I might not feel the same. I am very set in my ways, until, of course, I feel the need to change something based on the engines.

[Alan Perkins]

Search Mechanics is just a tool. Like all tools, it takes a craftsman to use it properly and get the most out of it. But there are DIYers who can't afford craftsmen, and we cater to those too. Goal: Improve the Web.

I use it a lot and I have knowledge of SEO. Other SEOs use it a lot. To be honest, I don't think many SEOs use the submission features. They tend to have their own thing going there. But to analyse a site before and after you've done your work, it's very useul (IMO!).

"I would also say that what the software may say that is wrong, I might not feel the same. " You can over-ride the default limits, so it reports whether you are wrong in your own opinion...

Thanks for the opportunity to promote Search Mechanics. What I'm most pleased about, though, is that everybody appears to be on board with spam, cloaking et al. :)

Alan

[Blue]

WOW! This is one fast moving and DEEP thread!

OK, I guess I'll jump in here and give an "outsiders" POV.

I am a designer. Not an SEO by any means, and when I was lured into these forums (thanks Doug! :) ) way back when they started (seems like eons from the amount of time spent reading), I simply wanted to gain as much knowledge about ethical SEO as possible in order to give my clients the best service possible.

First, I must applaud Alan for his efforts. Your white paper is a great step in the right direction and one that I hope gets the chance to serve the purpose it was researched and written for.

Next, I commend all those who participated in this most excellent thread, all of whom I have a great deal of respect for. You have helped Alan determine the "worth" and "accuracy" of his effort as well as to help him refine it.

Now, the reason I'm doing all this applauding and commending is because, as a designer, this paper, and hopefully the positive results it garners, will be of tremendous import to me and those designers like me, who take ethical SEO seriously.

I constantly hear about designers who have no knowledge of or interest in SEO. Well, there are some of us out there (and in here) who DO take it seriously; enough to join forums like this to gain pratical knowledge (and from the amount of knowledge I've gained from just being in this forum I feel I should have a USEO degree, not quite a professional but enough not to sink my clients completely in the depths of the SE's).

Anyway, for what it's worth, my vote is a yea (although I respect and understand Mel's POV and hope that his issues can be addressed to his satisfaction).

You've all done a great job and it gives me hope both as a designer and a searcher that some day in the not so distant future, the SE's will have much better relevant rankings.

:cheers:

[ihelpyou]

Yes Blue. I do agree with you. This was a great thread!

[Mel]

Hi All;

Apologies in advance for this lengthy post, but I need to clarify my position better, especially since this would appear to be the dissenting opinion.

At the outset let me say that if it has served no other purpose, Alans White Paper has, for me at least, caused much careful thought about the inter-relationship of users, search engines and SEO, and in particular about using search engines in general. I also acknowledge the shortcomings of the following in that it does little or nothing to provide practical guidance to SEOs. Rather I am proposing a set of broad ethical guidelines for managing the SE/SEO relationship, and fully understand that it does require more detailed implementation in order to make it workable.

This is also an attempt to explain my rather obstinate opposition to Alans principle that search engines should determine relevancy. It may be that at our present technology level it is true that in practical application search engines do determine relevancy, but I would like and expect to see that change.

The first guideline that I would like to propose is:

1. It is the ethical SEOs responsibility to help insure that his clients site receives rankings in ratio to how useful that site is to a user for a particular search term or terms.

This does not prevent him from also making the site more useful at the same time, but it should prevent him from seeking rankings better than the published site deserves. But how to measure this?

The responses search engines return in response to a users search are far from perfect, and the terms that users search for are also not particularly helpful to the search engines in providing a useful response. By this I mean that, for instance, when a user types the phrase "search engine optimization" into the search box he may be looking for something as simple as a definition of the term, or he may be looking for a firm to do search engine optimization for him or he might be looking for detailed information on how to accomplish SEO. As any experienced SEO knows, it may be that another user looking for the same or similiar information may/will use a different search term.

The search engine has no way of knowing which of these possibilities are in the users mind when he searches for that term and thus returns its best approximation for that term ranked in order of what it perceives as the best approximation first. This is where the first of many possible errors starts to creep into the searching process.

We currently have no mechanism for getting feedback from the user as to how useful the response was to him or what he would have preferred to see, and thus we have no way of defining how useful (or relevant, Alan) the results were.

I have gone to some length explaining this since my second guideline would be:

2. The measure of how useful a search engines ranked responses are should properly be determined by the user.

If we are to materially improve the search mechanism the SEs have to address this problem or, as now is the case, the only (very indirect) measure is the search engines popularity over a period of time. Popularity is not very useful as it has a long lag time between the search and the measure of its success, its not very precise as to what improvement is needed, but at the present it may be all we have.

I think Search Engines can and will address these shortcomings. PageRank in particular is a response to the search engines attempts to get feedback, although indirectly. Remember that what seemed impossible five years ago the SEs do routinely now.

The third guideline would then be:

3.Anything done to achieve a higher ranking than the user visible content of the site deserves should be considered as spamming the search engines.

Again I have proposed a guideline that is difficult or impossible to measure, given the current state of SE technology, but one that we should aspire to.

This explains my opposition to the proposed principles, but I understand that there is no practical way at present to implement these guidelines, and so we do need current and workable definitions (principals?) that all can agree to today, but IMO these should be phrased in such a way as to work toward the ideas outlined above.

For what its worth, I agree that agent delivery is sufficient to handle legitimate usability concerns, and I also feel that (inside the Body tags) anything not useful to the user is spam, and I would generally include invisible text in that category.

[Mel]

Jill:
Yep, that sounds like 'ol lazy Mel

I resemble that remark!

[Alan Perkins]

Hi Mel


1. It is the ethical SEOs responsibility to help insure that his clients site receives rankings in ratio to how useful that site is to a user for a particular search term or terms.


I think that's too big a jump - if you can split that into two guidelines, I'd be happy.

1a. It is the ethical SEO's responsibility to structure their client's site without using spam techniques to give it a chance of receiving rankings.
1b. It then becomes the search engine's responsibility to determine how useful a page/site is to its users in response to a particular query and to assign rankings accordingly.

2. The measure of how useful a search engines ranked responses are should properly be determined by the user
Agreed. I do think the popularity of the search engine is the best measure. I also think it is important that not all search engines deliver the same set of results to the same query. There is no "right" answer to a query. Put another way, there is no absolute relevancy. Niche search engines find niche users, and vice versa. Market forces.

3.Anything done to achieve a higher ranking than the user visible content of the site deserves should be considered as spamming the search engines.
I agree with the sentiment of this guideline, but it is too subjective to be a principle.

Alan

[Mel]

Hi Alan:
These are not intended as principals such as yours are, but guidelines for how things should work in an ideal world.

It is the intent of the first guideline that an ethical SEO should strive obtain reasonable rankings but cannot use techniques which would result in a more than fair rank, whatever they may be. The suggested splitting removes that obligation. I feel it unnecessary to mention the search engines will give the ranking as that is pretty well understood by everyone.

As to using the search engines popularity to measure how correct the rankings are I cannot agree for the reasons given - its much to slow and imprecise to be of any use to the search engines except in a very general way. We need to devise a technology that can provide more or less real time feed back on a search by search basis. Again these are goals to shoot for, not ways of doing things.

Unless and until all search engines have all the pages in the web indexed the results will be different, thus I think this is not a problem.

[Alan Perkins]

OK, Mel, we'll agree to differ

:cheers:

Alan

[Mel]

Alan:

Yes we disagree on some things, but not everything. In particular (even though it may not sound like it) I appreciate your approach as the way you are going at it there is a possibility that the SEs just might listen to you. I prefer to needle them into doing bigger and better things.