There's no such thing as a stupid question, but they're the easiest to answer.
JoinTour
Login
 
Web Design & Development
Tag Cloud
audio blue screen boot bsod computer cpu crash dell desktop driver drivers error excel external hard drive firefox format freezes freezing hard drive hardware hijackthis internet internet explorer itunes laptop malware motherboard mouse network networking outlook 2007 power printer problem ram router screen slow sound spyware trojan usb virus vista vista 32-bit windows windowsxp windows xp winxp wireless
Search
Search in:
 
Advanced Search
Tech Support Guy Forums > Internet & Networking > Web Design & Development >
http status codes and spiders?


Computer problem? Tech Support Guy is completely free -- paid for by advertisers and donations. Click here to join today! If you're new to Tech Support Guy, we highly recommend that you visit our Guide for New Members. Enjoy!

Closed Thread
 
Thread Tools
freedumb's Avatar
Junior Member with 5 posts.
 
Join Date: Oct 2004
Experience: Beginner
24-Oct-2004, 03:44 PM #1
Unhappy http status codes and spiders?
Hi,

I'm very much a beginner at webdev and this question is possibly a very stupid one. Please forgive me if it is. I have checked around the web and haven’t been able to understand much of what I've found.

Anyway, I have an account with a host and my site is at ieig.net.

The site files are actually stored in a sub dir (www.ieig.net/ieig/) and I have a permanent redirect set up in .htaccess. This is the redirect:

"Redirect permanent /index.php http://www.ieig.net/ieig/index.php"

I also have an account on linkmarket and it has been suspended for a reason only they know of. My attempts to contact them have all failed, most likely because I'm not a paying customer. From "cPanel - Latest Visitors" I can see that the linkmarket spider is still visiting my site on a daily basis but it has a status code of 301. Also msn spider visits occasionally and it also has a status code of 301 as do the majority of spiders that have visited my site. When I visit my site with IE I get a status code of 200. So I'm wondering, does the fact that these spiders are getting a 301 on my site mean they are unable to access it? Could this explain my account being suspended on linkmarket or is the 301 ok? The only reason I ask is because I get the 200 status code as do some other spiders and surveyors.

BTW, Linkmarket spider is for verifying reciprocal links so if it wasn't able to access my site this may explain my account being suspended.

Cheers
Sequal7's Avatar
Computer Specs
Distinguished Member with 2,380 posts.
 
Join Date: Apr 2001
Location: Around the corner!
Experience: Including today?
24-Oct-2004, 04:45 PM #2
301 usually means that the spider has detected a broken link or loop in your files. Do you by chance have a custom 404 error page?
These are most likey the problem.

A 301 redirect is the most efficient and spider/visitor friendly strategy around for web sites that are hosted on servers running Apache (check with your hosting service if you aren't sure). It's not that hard to implement and it should preserve your search engine rankings for that particular page. If you *have* to change file names or move pages around, it's the safest option.


Search google for 301 redirects, or click here
__________________
Good Luck on your fix

My real hobby..JoyCo
My real Job..(Second Hobby) IAFF Local 1865
Like the sites? My hobby is the one that created them!
freedumb's Avatar
Junior Member with 5 posts.
 
Join Date: Oct 2004
Experience: Beginner
24-Oct-2004, 10:21 PM #3
The site dosen't have any custom error pages and the "check server header" tool on http://www.seoconsultants.com/tools/ indicates that the redirect is configured correctly(from what I've read). First I get a 301, then a 200. I am assuming that I should always be looking for a 200 when spiders crawl my site. Again, this assumption is based on what I could understand of what I've read on this subject.

Here's an example of one of the result in Latest Visitors:

Host: 195.92.95.94 Url: / Http Code : 301
Date: Oct 25 09:19:40 Http Version: HTTP/1.0" Size in Bytes: 0
Referer: http://www.netcraft.com/survey/ Agent: Mozilla/4.0 (compatible; Netcraft Web Server Survey)

as opposed to my own access:

Host: 159.134.79.78 Url: /ieig/index.php Http Code : 200
Date: Oct 25 09:03:50 Http Version: HTTP/1.1" Size in Bytes: 20313
Referer: - Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

All the 301 are http/1.0

I have made a very minor change to my root .htaccess today and also removed front page extentions which cleared out a ton of directives from the file, Will see what the spiders think of it before doing anything more.

Thanks for the response.

Last edited by freedumb : 24-Oct-2004 10:33 PM.
freedumb's Avatar
Junior Member with 5 posts.
 
Join Date: Oct 2004
Experience: Beginner
27-Oct-2004, 08:03 AM #4
My redirect is still showing status 301 for http/1.0 clients. I'm gonna go with a manual hyperlink for redirecting to the top level directory, this seems to work much better and gives 1.0 clients the 200 code. My linkmarket account is now unlocked since I've set up the manual link and I don't get 301 anymore. Can anyone tell me how to do one of those automatic redirects found on many download pages on the web. The ones that usualy have a time period and a "click here if your browser dosen't automaticly redirect you" type of thing going on.

Cheers
Closed Thread

THIS THREAD HAS EXPIRED.
Are you having the same problem? We have volunteers ready to answer your question, but first you'll have to join for free. Need help getting started? Check out our Welcome Guide.


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
WELCOME TO TECH SUPPORT GUY! Are you looking for the solution to your computer problem? Join our site today to ask your question -- for free! Our site is run completely by volunteers who want to help you solve your computer problems. See our Welcome Guide to get started.



Thread Tools


You Are Using:
Server ID
Advertisements do not imply our endorsement of that product or service.
All times are GMT -4. The time now is 06:41 AM.
Copyright © 1996 - 2008 TechGuy, Inc. All rights reserved.
Powered by vBulletin, Copyright © 2000 - 2008, Jelsoft Enterprises Ltd.
Search Engine Optimization by vBSEO 3.1.0
Powered by Cermak Technologies, Inc.