marcus-miller
07-03-2002, 06:22/06:22AM
Hi all,
The company I work for pays another small firm to submit our pages to the inktomi database. ( I have just picked up on this task so excuse me if some of my info is a bit ropey )
<<<<<BACKGROUND INFO>>>>>
The pages are dynamic and use a function of my own to parse the .htm urls to populate the neccessary variables with data needed to query the databases and build the dynamic aspects of the pages.
the urls are in the format of
www.domain.com/page/ - with no vars appended to the .htm string
-or-
www.domain.com/page/3-category-name/
where /page/ is a .php page hosted on a linux box running apache server and /3-category-name/ is exploded to give a category number 3 and extra details (category name) for dynamic meta tags, titles and body text
Where the default url above is called the php script searches for the presence of the variables < if (!isset($var)) { > where non exists it gives a default value.
----------------------------------------------------------------------------------
The problem we are having is detailed in the following text from an email from the company doing the submission:
We contatcted Inktomi and here is the response returned.
Their server is actually serving our crawler an HTTP 406 error code because
it seems it doesn't like the Accept: header which we send (Accept: text/*).
Normally this happens when people try and serve dynamic content based on the
Accept/User-Agent combination, and don't code for crawler accesses. If you
can get them to check their content-negotiation code for errors and update
it so that it can handle text-only clients then that should solve the
problem.
Support
xxxxxxx xxxxxxxxxxx
Im not sure if anyone can help, but a little birdy mentioned something to me about some apache dark-arts going by the name of 'mod_rewrite' that may be a solution to this.
Please excuse me if i have overstated some details here but this is my second post, and I have just had this problem dropped on my desk and dont know where to start....
all replies appreciated..
Thanks in advance
Marcus
:cheers:
The company I work for pays another small firm to submit our pages to the inktomi database. ( I have just picked up on this task so excuse me if some of my info is a bit ropey )
<<<<<BACKGROUND INFO>>>>>
The pages are dynamic and use a function of my own to parse the .htm urls to populate the neccessary variables with data needed to query the databases and build the dynamic aspects of the pages.
the urls are in the format of
www.domain.com/page/ - with no vars appended to the .htm string
-or-
www.domain.com/page/3-category-name/
where /page/ is a .php page hosted on a linux box running apache server and /3-category-name/ is exploded to give a category number 3 and extra details (category name) for dynamic meta tags, titles and body text
Where the default url above is called the php script searches for the presence of the variables < if (!isset($var)) { > where non exists it gives a default value.
----------------------------------------------------------------------------------
The problem we are having is detailed in the following text from an email from the company doing the submission:
We contatcted Inktomi and here is the response returned.
Their server is actually serving our crawler an HTTP 406 error code because
it seems it doesn't like the Accept: header which we send (Accept: text/*).
Normally this happens when people try and serve dynamic content based on the
Accept/User-Agent combination, and don't code for crawler accesses. If you
can get them to check their content-negotiation code for errors and update
it so that it can handle text-only clients then that should solve the
problem.
Support
xxxxxxx xxxxxxxxxxx
Im not sure if anyone can help, but a little birdy mentioned something to me about some apache dark-arts going by the name of 'mod_rewrite' that may be a solution to this.
Please excuse me if i have overstated some details here but this is my second post, and I have just had this problem dropped on my desk and dont know where to start....
all replies appreciated..
Thanks in advance
Marcus
:cheers: