Traffic analysis tools

[The Ney]

I have a question regarding the traffic analysis tools available for SEO company.

As an SEO i have faced the Traffic analysis problem, because, giving the SEO service to different companies, often your client`s traffic analysis tool (provided to him by his server service or domain service) does not give you enough of quality information, needed to produce ranking reports, and other traffic analysis, invaluable to SEO.

My question is, how do people on the forum face this problem ? I have thought about trying to analyse log files but i suspect that will not give me the information needed (i wont be able to get different time range analysis, or to see search phrases refferals rather than search phrases cut into single words, the way some traffic analysis tools do, etc.)

[ihelpyou]

That is always a problem and might always be one. I simply do not worry about it. I will sometimes ask the client if I can access his.

Since I do not price based on clicks or ranks, It is something I do not need. The client tells me how many hits they are getting. Like yesterday, a client said that visitors have doubled in the last two months. That is rare for so soon, but I am satisfied with that info. I do not need to know anymore.

The tracking I use for this forum might be an option for you. Ask your client to paste the code and you both can have access to the stats. Of course, many sites would not want this on their site, but it is free.

[The Ney]

Well, we dont price according to rankings or clicks either... i dont think that is a correct way of pricing.
But i do find reports very valuable...first of all it can give me a clue about my success at certain search engines.
Second of all i can get another input about keyword popularity, and just for those keywords that have brought me visitors. According to these data, i can learn a lot about SE algorithms, changes in them, shifts in your ranking and blah blah blah

[ihelpyou]

Well, having stats can be good, but I simply ask the client if I can have access to the stats. If not, he will tell me how things are going and what phrases gets him visitors and anything new as well. The positions the client achieves on the terms we are tracking lets me know how things are going as well. I already have an idea how many times a term is searched on thru wordtracker. I just do not feel the stat hassle is anything to worry about. Every stat program you use shows different results anyway and can be misconstrued.

[ihelpyou]

A program I would be interested in would be one that shows me spider info for the client site. The only way to do that would be to have your own server with a logs analysis program and could moniter all your sites. Of course, the sites might have to be on the same server as well. Or, the server load would be very great.

[Navarone]

Try the following web site http://www.mediahouse.com/

We use the live stats program, it contains a lot of information on a variety of web site stats. Your clients can review this information as well.


Navarone

[ihelpyou]

Yes. That one is good but pricy. It also cannot track the spiders which no online program can do.

[The Ney]

Yes, but again, these solutions are very good, in case you give web hosting services together with SEO. But if you are not, you need something that doesnt have to be installed on a specific server.
Log file analyser is the only solution that serves this kind of SEO companies, as far as i know.... or i dont know enough...

[ihelpyou]

Please post the Url to it the ney.

[The Ney]

URL to what ?

[ihelpyou]

I do not understand your prior post, I guess. :eek:

[Advisor]

Ney,

The perfect solution to what you need is Web Position **** http://www.webposition.com

Not only will you be able to check your rankings in the major search engines, but they have a new traffic analyzer solution that is very good. I just started using this new solution, and so far am very impressed. I had been using their old traffic analyzer which was okay, but it didn't give as much detail as the new one. WPG is reasonably priced and any SEO would be crazy not to have it, for what it gives you. The new traffic analyzer will be an additional fee, but from what I understand, it's not an unreasonable amount if you're a professional working with clients that need this information to justify their ROI.

Jill

[ihelpyou]

There is one problem with their anaylzer Jill. I would use it but I can click on a referring Url for that stats program and be taken directly to all the users stats information. It is not secure.

Yesterday, I saw a hit from a visitor, I clicked on the Url to see where it came from. I was taken to the WPG stats site with ALL the info of all the traffic that site is getting. I could click into any stats report I wished. Just thought you should know.

[Advisor]

Doug,

I believe that's their old traffic analyzer where that happens. And you're right, that's a big ole pain in the butt. I've taken to copying the link and clicking it in a clean browser. However, the new stats are totally different. They're outsourced by a different company...can't think of the name right now. It's very similar to the reports you'd get with WebTrends. Very good, and I do believe, very secure (although I haven't tested that so I'm not gonna say that for sure.)

Jill

[ihelpyou]

If that is the case, it may be worth a look.

It is amazing the number of online stats programs out there that are NOT secure.

I can also go directly into the Paid version of the ExtremeTracking program and view all stats.

There are many more I can view of other programs as well. I would have to know for sure that the WPG is secure before using it. Many clients would not wish their stats to be viewed by every Tom, Dick, and Harry. :)

[Navarone]

Sorry to complain here but I think we may have a problem with the email notification process., I am set up to recieve notification but sometimes I get more than one email for the same post/reply, which is correct if someone has replied but sometimes I open the link and I don't see anything thats been posted new, as i am writing this another email has come in on this very post. Maybe someone can check it out.

[ihelpyou]

You could be set to receive emails on new threads and new posts both. I am not sure. Go to your control panel and inbox to see what is in the the notification. The inbox will show you what you have subscribed to.

[ihelpyou]

Also, if you are subscribed to the thread/forum, whenever you make a post, make sure the box is unchecked to receive notification. Otherwise, you will receive two emails as well.

Actually, there are a number of reasons you are getting two emails. You have to check all possiblities in the control panel, inbox, and when you post and reply.

[Advisor]

Doug,

I just did a quick test, and it appears that it is secure. If someone clicks on the hitlinks referrer URL, it brings you to the page where you have to put in your user name and password. So that way no one can get right into your stats. Looks very secure to me unless someone was able to somehow hack into your password.

You should check it out. If you have a registered version of WPG you get a free 90 day trial. After that I think it might be $10 a month. Or you can use an image banner for free.

Jill

[Navarone]

Ok here are my settings



Messaging & Notification
Allow Bulletin Board Administrators and Moderators To Send You Email Notices? yes
Hide Email Address?
Clicking yes hides your email address in your public profile, so that it is not accessible by other users. yes
Use 'Email Notification' by default?
Using this option emails you whenever someone replies to a thread that you have participated in. yes
Enable Private Messaging?
Allows people to send Private Messages to you yes
Send you an email when you receive a Private Message? yes
Pop up a box when you receive a Private Message?
This will pop up a small warning box when you receive a Private Message asking whether you want to view the message. no

[ihelpyou]

Okay. Got it. Uncheck the default to receive email when you participate as you can always check the box when you make a post. Sometimes you will forget to uncheck the box and then you will get two emails. By unchecking the default in your settings, you will avoid this.

[ihelpyou]

Hey Jill, is there details on all the reports that it gives you on their site? I mean details as I use Superstats right now, which is very detailed.

[Navarone]

Well, here is what I think is happening. I just replied sending my settings. Then, I received an email telling me that Navarone (me) has just replied to xxxx post. Why would it be send me an email notifying my that I just replied to a post?

Thanks

[ihelpyou]

Because you just posted above that your settings are set to receive email on ALL posts made that you have participated in. CHANGE that to NO, not yes.

[Advisor]

Yes, Doug, it's very detailed. Tells everything you'd want to know I think. Not sure about spiders or not though. I haven't seen that, but I just started using it. I know that Web Trends tells you about spiders. Does the SupreStats one? This one may not be much different from your free one, which would mean there would be no reason to bother switching. I don't know.

Jill

[ihelpyou]

oh no. The stats is on my other site and is not free. Although it cost 20 a month.

I could offer the WPG one for much cheaper.

[Mel]

Hi Jill and Doug:

I am using the Hitslink service from WebPosition and find them ok so far.

They have quite a number of reports availabe, have master accounts for seos to use for many websites, and perhaps best of all they offer a free option in return for a link.

For more info go to HITSLINK (http://counter.hitslink.com/webposition/statistics-reports.asp)

[ihelpyou]

hey Mel, that looks very interesting. It mentions subaccounts for your clients but does not tell you what the pricing is? If I am paying $1000 or more per month, I would want a break for the other accounts. Do you see where they tell you? Still lookin.....

[Advisor]

Doug, I remember seeing somewhere that it was going to be $10 per month, but more depending on the amount of traffic you get. Don't know where I saw it though!

Jill

[highman]

Heres something I came across the other day in 'another forum' try a search on google for '
Unlimited Multiple Page Tracking' or any other phrase your stats program puts out.

Moral: Beware of linking to your stats files

Can provide some great insight into your competitors keywords :D

[ihelpyou]

hey Mel. I am reviewing the link you gave me for hitslink. I am very concerned about the "hidden link" that the code starts with. Why is this? I submitted an email to them inquiring about this as this link is Highly unnecessary in my opinion and as nothing to do with the code between the <script> and <noscript> tags. This could serve to be a big detrement to rankings.

[Advisor]

Doug, I'm not sure, but I think that the invisible .gif is the whole basis of how the program works. If you used their version with the viewable graphic, it wouldn't be invisible. I don't believe there's any problem with having an invisible (transparent) graphic on your site, if this is what you're talking about.

Jill

[ihelpyou]

NOpe. I am not talking about the invisible gif. That is there as well. I am talking about this link:

<a href="http://www.toolshack.com"><script language, etc, etc. <noscript></a>

That is an invisible link. There is an invisible image also in the code for tracking.

[ihelpyou]

I took out that blatant advertising hidden link they stuck in the code! That is horrible, terrible, no words can describe it. That code will hurt every site that uses their service. sheesh.

Hopefully, it still works now. Will know real soon.

[Advisor]

Let me know if it still works. I assumed it was necessary for it to run correctly. Why do you think it will hurt sites to have it there?

Jill

[ihelpyou]

Because it is a hidden link that only has javascript code in between the starting <a> and the ending </a>. The script in between the spiders cannot read and basically ignore. So, according to the spiders, all they see is the link with Nothing in between. It is there only for advertising that site with no regard to it being a hidden link. In my opinion anyway.

The spider filters are programmed to detect hidden links, so once they spot it, the site would be penalized in some way, of which you would never really know.

[Advisor]

I may be wrong, but couldn't it be put there for browsers that aren't javascript capable? So that the traffic analyzer can do its job even if someone is using an older browser? I don't think it's a blatant ad, as what purpose would it serve as an invisible ad like that?

I have no idea and might be totally off base, but I also wouldn't jump to conclusions without knowing for sure. Perhaps a programmer might be better able to interpret what it means for us.

Jill

[ihelpyou]

I don't think so Jill. Mainly because if a browser is non-javascript, your stats will tell you that. Besides, only about 1% of all your visitors will not have a javascript capable browser, at least according to my stats over the past year.

I will go look at my stats now to see if it is still tracking.

[ihelpyou]

Tracking is just fine without that link.

[ihelpyou]

See, since I have used a couple different online tracking services, I noticed that link right away as soon as I was given the code. I had never seen a link there before from two prior services. The script itself is basically the same but in the past NO link was ever at the start.

Also notice that the link has nothing to do with the script. The script is calling from the domain of:
counter.hitslink.com

and not from toolshack.com

Very strange indeed.

I am quite surprised that WP has not noticed it before, and if they have, why have they allowed it there, or maybe they do not realize that the engines penalize for hidden links?

[ihelpyou]

Just received back this reply from ToolShack about my question regarding the hidden link:
Good catch. It is there since some accounts display an image and use the link. For the accounts that use the invisible image, I agree it shouldn't be there. I will make sure it gets taken care of.
Makes me wonder how long all sites have had the hidden link on all their pages??

[ihelpyou]

I do respect the complete honesty with his response, as he could have danced around the issue, but did not.

[Advisor]

It sounds like it was not intended as anything sinister. Good to know it can be removed, however.

Doug, the surf+ crap has made you sooooo jaded! :bandit:

Jill

[ihelpyou]

Oh yes... can be removed easily. I am simply shocked that no one caught it til now. They had to change up the script anyway if a client is paying. Not only did the script maker not notice the hidden link but all others viewing the script, including WP did not notice the link?

It took me all of one second when I clicked on "add html" and I noticed immediately. And believe me, I could not write a script for any amount of money. :)

[Advisor]

Looks like they've already fixed it too! :4:

Jill

[markymark]

Yeah, I know this thread is a month old, but I am always looking for better trackers and log file analysis programs.

I would still recommend the following, though.

Happy Log for good log file analysis. I'm sure there are possible better analysers out there and the help file is poor, but it is very customisable and does give an accurate indication of what's going on in a site. It's only $33 and is available from www.axolot.com/happylog . I think that's the URL. There's lots of filters so you can cut out all the c**p.

Also AdvLogger from www.perlonline.com . Only works with Unix (unless you can hack a script better than me, which you probably can). There's a Javascript and an SSI option. The SSI option captures spider visits and bookmarks, and it really is the biz for tracking search engine referrals. Oh, it's free and is the best CGI tracker I've ever used.

I'm sounding like an affiliate marketer now, but I would recommend these to any one. I am very interested to know what people think as it's taken me about a year to come down to these two.

[JuniorHarris]

Thanks for the update...I've used the free version of Happy Log and was pretty impressed with it. I don't use it regularly, as we have our own custom stats built into the web application. But it is fairly easy to use and customize.

[rmridgew]

I just signed up with these guys and have put the code on all my pages could you help me fix them so they dont look sinister

[dina]

Hi!
What about all that talk on forums that SE frown on programs like WPG? Maybe even more than frown for WPG had some problems with some SE in older vrsions (true, for I've also used old one recently)? Supposedly it took authors 3 months to change it so it can work again with some SE (I had to update to get resulst from some). Can it do any harm to users of WPG, especially on Google, couse of their rather strict rule against checking of that kind?

[Susan Goodson]

Hi Dina

If used properly, WebPosition **** will definitely not get you banned or penalized in the search engines. Just the opposite is true, since the Page Critic will warn you not to do things that might get you in trouble--like using too many keywords, over submitting, or other things that a search engine is known to dislike.

If you've heard that WebPosition **** will get you banned, then that is based on outdated information. There was a short period of time where AltaVista decided they didn't like the generic doorway page format that prior versions of WebPosition **** generated. Therefore, we quickly revised the program to generate pages in a format that is different for every customer. There's no longer any difference between our generated pages and any other page on the Web as long as you use it to create unique, content rich, quality pages.

For awhile AltaVista appeared to be discriminating against pages with certain "boiler-plate" phrases on them that were the same phrases as other customer pages that used the default template. We quickly revised the program in version 1.40 and higher to generate pages solely based on what you, the user enters. We even added a template feature so you can completely customize each and every page you generate.

Therefore, there's nothing different about WebPosition generated pages than a page created in say Microsoft FrontPage. The only "difference" is the Page Generator is designed to step you through the process of creating a page that will rank well on the search engines. It will make sure you have all the recommended elements such as a title tag, meta tags (if the engine favors them), heading tags (if they favor them), etc. You could use your own HTML editor to do this if you like and then run the Page Critic against it for advice on fine-tuning it. Or, you can use the Page Generator.

In addition, none of the search engines have ever had a problem with the WebPosition Page Critic, Submitter, or Traffic Analyzer. We also take extra precautions to avoid potential discrimination by designing the modules to emulate a popular browser so WebPosition looks like everyone else on the Web when it submits or does searches. The only current issue with WebPosition **** is that Google does not allow automated searches. If they detect you as abusing their service by performing too many searches concurrently or even a few searches but running them too often they can block you from searching their engine. If you email them directly they will unblock you with a warning to not use automated search programs of any type. They do not however penalize your site's ranking regardless of whether you use the Reporter, Submitter or any other module of the program. You can see this for yourself on Google's own site. They state they do not penalize sites for submissions or automated queries, but they can and will block you (specifically your IP) from searching their engine for using automated queries:

http://www.google.com/webmasters/facts.html

If you keep your automated searches minimal and run them during the off-peak hours you should be okay. There is really no need to run constant queries, usually once per month is often enough.

Therefore, WebPosition **** is not going to get you banned or penalized in the search engines if you are cautious in your approach. If anything, it can help keep you from getting banned, or just as bad, never showing up in the top 10 to 30 matches. You really can't afford to not optimize your Web site in today's market. The problem is that the rules differ for each search engine, and can change from time to time, which can be maddening to sort out on your own. That's where the software steps in to make your job easier and less time consuming. It warns you about techniques such as using hidden text, redirects, multiple keywords side by side and many other things that can get you banned or penalized.

- * -

Another issue that is drawing some confusion lately is whether "doorway pages" are "bad" or not. The exact definition of a doorway page varies widely. Our definition is that it's simply a page on your Web site that you have optimized or fine-tuned to rank well on a certain keyword. Since people end up finding your site through these pages, they act as "doorways" to your site. These pages could be a product information page, your home page, or whatever other page you have. If people are finding that page in their searches, it is an entrance (doorway) to the rest of your Web site. However, many do not like the term "doorway pages", and I tend to agree it now seems to have a negative connotation. A better way to think of them is just good, quality pages on your site that do not contain any tricks or techniques the engines might find objectionable and that are optimized for the key term you are targeting.

Under our definition, search engines do NOT have a problem with doorway pages as long as you follow their various rules, which, unfortunately, tend to change from one month to the next. The trick is to know exactly what the search engines like to see and what they don't like to see on a page (which is the kind of information and advice our monthly newsletters and the WebPosition **** Page Critic supply). Used effectively, this information will help you improve your search engine rankings and tell you what things to avoid to keep from getting dropped by an engine.

Around February 2000 AltaVista in particular started cracking down on pages they consider to be spam. They now claim they don't like "doorway pages" in general. Therefore, it's wise to avoid using that terminology when talking to their support people.

However, AltaVista's definition of doorway pages differs from our own. They consider a doorway page as a "page that lacks significant content." If they decide you have too many pages that don't have "significant content" then they may drop you or ban you from their index.

They do not ban you for simply optimizing a page to rank well if the page has good quality content, or for simply using WebPosition ****. We know optimizing your page is not against their "rules" because their own FAQ page gives Webmasters a number of basic tips on how to create pages that are relevant for keywords you want to be found on (i.e., a doorway page). They don't go into any great detail on improving your rankings, but they do demonstrate that designing your page so it will be favored by their ranking algorithm is not a bad thing.

In summary, read the MarketPosition Newsletters:

http://www.webposition.com/newsletters.htm

Then read the advice in the Page Critic. If you're concerned about following AltaVista's latest rules, be sure to read the article in the April newsletter at:

http://www.webposition.com/mp-0400.htm

You might also check for any newer articles that may be available. The Page Critic in WebPosition **** will always have the latest advice for each search engine, including AltaVista. It will analyze your page and suggest changes to make so your pages will be favored by the search engine.

Hope that helps.

[MazY]

Ignoring all the above - apologies in advance for that...

I have to say that the webposition newsletters are amongst, if not the best newsletters there are out there for the SEO industry.

Always a good balance of technical and generic information. Keep it up and don't go changing the format any time soon. It's perfect. Well, for me at least.

[Advisor]

I second that! Brent always does a great job!

Jill

[dina]

Thanks, Susan, for all these reassuring details. Although I really use exclusively Traffic Analyzer and not very often. Not that I think other tools are not useful, but I happen to be manual kind of type. :) Anyway, it's good to see you care about your clients.

[Alan Perkins]

Hi Susan

Our definition [of a doorway page] is that it's simply a page on your Web site that you have optimized or fine-tuned to rank well on a certain keyword.My definition is that it's a page that has zero (or one, from a "hallway page" :)) inbound links from other pages on the same site.

[Alan Perkins]

Actually, that's not quite accurate, but it is simple. The accurate definition of a doorway page is recursive:

a page that has zero inbound links from other pages on the same site; or, a page whose only inbound links from other pages on the same site are from doorway pages.

So, if Page A was a doorway page that linked to Page B, and no other page on the site linked to Page B, then Page B would also be a doorway page.

I think that an alternative, equivalent definition would be a page on a Web site that cannot be reached by starting at the home page of that site and following links within that site.

[Advisor]

a page that has zero inbound links from other pages on the same site; Shouldn't that be zero "visible" links?

Most people get around your definition simply by creating invisible links to their doorway pages.

Jill

[The Ney]

BTW, regarding the blocking of automated queries, i've been having problems with Top Dog lately. Cannot get any results for Yahoo and Google. Yahoo Google is still available, but those two not. Anyone else ?

And if we speak about newsletters, i am not subscribed to Webposition (yet, but that has been fixed) but my vote for the best newsletter goes to...... Rankwrite Roundtable.

[Alan Perkins]

You are correct Jill - but invisible links are spam!

There is no need to "get round" my definition. I don't think doorway pages are spam per se (they often contain spam...), so why get round the definition by using spam?

I'm, beginning to like my alternative definition (which I only thought of to illustrate the point) - people can't follow invisible links, so how about this:

Doorway page: a page on a Web site that cannot be reached by a person starting at the home page of that site and following links within that site.

[ihelpyou]

Good def. Alan! and that would be a page I would call Spam.

Well, I should say in Most cases it would be spam. But say that you have an order form on a secure server and when a visitor submits, he/she is taken to a "thank you" page on your site. That page is not accessible any other way. Obviously, that page is not spam. There are not too many other ways though. Most pages by that definition would be spam.

[Alan Perkins]

and that would be a page I would call Spam.

I wouldn't call such a page spam. There could be lots of reasons for doing it (e.g. referrer tracking, microsites, pseudo-secure areas, etc.).

If done purely for search engines (how could you tell...?), then some might call it spam. I still wouldn't. Sad? Yes. Spam? No.

[ihelpyou]

Yes. I modified my post. Microsites? They are not on the same domain so do not qualify for this.

[Alan Perkins]

Doug

We obviously need to define "microsite", too!

How about this for a doorway page:

Doorway page: a page on a Web site designed to be the first page that a user might see on that Web site, that will never be returned to by a person starting at that page and following links within that Web site.

That's very formal but sums it up pretty well, I think. The feature that makes a doorway page a doorway, IMO, is that once you've moved away from it, the only way to get back to it is to:

1) Use the back/history/bookmark features of the browser
2) Navigate back in from an off-site location
3) Type its URL directly in

[Advisor]

Originally posted by The Ney
And if we speak about newsletters, i am not subscribed to Webposition (yet, but that has been fixed) but my vote for the best newsletter goes to...... Rankwrite Roundtable. Why, thank you The Ney! That's the kind of thing that keeps us doing it every week!

Jill

[rmridgew]

I have recently begun using hitlinks,,,tonight I noticed a problem with the Opera 6.x browser for some reason the java wasnt picking up the cookie placed by my index page as the visitor went to multiple pages of my site....every time he opened a new page it assigned him a new unique identity,,i did notice that he was using java1.3,,,does this ring a bell with anyone and should i tell hitlinks to disregard opera browsers in the future (an available feature)

[Susan Goodson]

I would recommend you inform Hitslink regardless, as they should be made aware of what you are seeing in case they do not already know of it.

[The Ney]

Just to continue on the original topic. Have been using the Happylog for quite a while (after it took us some time to get log files) and find it great!. Takes a little time to adopt it to the search engines and spiders that are new and do not exist in Happylog database, but other than that it is very good.

[markymark]

Wow, somebody actually took my advice ;) Yeah, I like HappyLog too - but they really should update the search engine file themselves once in a while. I keep recommending it to clients, then spending ages updating the search engine spider files for them.

[kneelsit]

See this thread started way back but one point made concerned me - namely the plug for wpg. Seems this software is not currently popular with Google and other SEs because of the way
it hogs their bandwidth. Could be that sites using it might find themselves downgraded by google in the near future.:6:

Personally I have always used the excellent free version of "extreme"
i.e. http://w.extreme-dm.com Also available in a paid version which is not public.:)

[ihelpyou]

hey kneelsit, users are just fine if they simply use the software when they have to and not go overboard. They should use it late at night and only check ranks once per month. There is NO reason to check ranks more often than that. This goes for all rank checking software and not just WPG.

Of course, with all software there is risk involved of which the user inherits by using it.

[kneelsit]

Hi Doug,

I stand corrected ;) as to detail of amount of time spent using it.
However, having used the trial version for a time I feel that the advantages claimed for the software are overrated.

Just my opinion:)

[ihelpyou]

Know what you mean. Most use the software to only check ranks with..... sparingly.

I know some SEO's use the other features, but if I used it, I would only use the rank checking feature. I like to do the rest completely on my own.