URL status codes: A complete guide for SEOs

Most internet users have encountered webpages saying things like ‘404 error’ or similar. But there are many different types of status code, and they perform a number of important functions, from redirecting readers smoothly to indicating server timeouts and much more.

In this article, you’ll learn what is behind these codes, how you can avoid error pages, set up custom 404 pages and other important technical SEO info. And if you’re interested in the topic of redirects, don’t miss our beginner-friendly guide.

Each search inquiry on the Internet sets a process in motion: the communication between the user, client (browser), and the server. Moreover, each process with an HTTP status code from the server is acknowledged. In this way, the user can see if his or her inquiry was successful, and, if not, why a given element is not being displayed.

HTTP status codes consist fundamentally of three figures and are divided into various status classes. The first of the numbers designates the respective class. These codes are the servers’ response as soon as a search inquiry is processed. Put briefly, the numbers indicate the following problems:

• 1xx: Information is being processed.

• 2xx: Inquiry successfully completed.

• 3xx: Site was redirected or forwarded to a new domain.

• 4xx: There is an error in the inquiry of the client (browser).

• 5xx: There is a problem with the (websites) server.

• 9xx: Error from the network.

Not all these status classes require an action; some you will never see on your screen. Following, you will find detailed information about the codes.

1xx Status Codes

100 (Continue):

The server has correctly received the inquiry and is waiting on further instructions from the web browser. If it receives the instructions, it can successfully complete the action.

101 (Switching Protocols):

(Rare) Are used when the server receives an inquiry with “upgrade” header field and changes to another protocol.c

102 (Processing):

This status code prevents a timeout (time limitation) on an inquiry. It is used above all when the inquiry requires a lot of time.

2xx Status Codes

200 (OK):

This HTTP status code is the norm and is used when there are no problems in the requisition process. It shows the server that the search for the URL was successful. The queried website appears in the browser, the 200 does not. This means that the server must accept the inquiry of the client and then be in a position to send the requested site back to the client. Moreover, the resources must exist on the server.

3xx Status Codes

301 (Moved Permanently):

This HTTP status code shows that the site can no longer be reached under the queried URL. It is permanently shifted to another URL (redirect). The old URL is no longer valid; however, Link Juice is transmitted to the new address. If a successful redirect is established, the visitor is automatically forwarded to the new URL.

302 (Found; Moved Temporarily):

Status code 302 indicates that a site has been only temporarily shifted and can, in the meantime, be found under another URL. The original site is still valid. The Googlebot crawls and indexes the original storage location with a temporary 302 forwarding. Therefore, URLs reachable under a new address must be permanently redirected with the status code 301.

307 (Temporary Redirect):

Short-term redirection of websites can be effected with the 307-redirect. This often happens when it is necessary to wait for a server. For this, a php file is written that leaves a message for visitors to the website. This file is then saved in the root directory. Finally, a new htaccess file needs to be created with the name htaccess.307.

4xx Status Codes

410 (gone):

If the user receives this status code it means that the resource is no longer accessible and/or has been erased.

401 unauthorized:

With this server request, the client must authorize themselves. This usually happens with a login. If the user still attempts to access the password protected resource, the status code 401 (unauthorized) will appear with instructions as to what to do.

404 (Not found):

The HTTP status code 404 indicates to web browsers and search machines that a resource is not (or is no longer) reachable. This can be a URL or an element within a subpage – for example a video, an image, or a PDF file. The error message is also displayed when the user enters a URL in the address line that does not exist or that has been incorrectly entered.

The status code 404 is displayed:

• when the location of an element or a file has changed but the external references referring to it were not informed and are still inked to the old address (“dead links”);

• when the resource was moved, but the webmaster has not adjusted the appropriate link;

• when a URL is requested that does not exist: for example, if it was entered incorrectly in the address line.

Figure 1: Typical error sites in the Chrome browser.

5xx Status Codes

500 (Internal Server Error)

The code 500 is a “collected status code” for internal server errors. With this display, the requested URL or element is not achieved. If it is not clear wherein this lies, you should check whether there are errors in the .htaccess file.

503 (Service Unavailable):

The code stands for unexpected server errors. The server is temporarily not available, for example, if it is overloaded or is currently being maintained. The web browser can be informed when the request is possible again. The expected time of availability will appear in the header field “retry-after”.

9xx Status Codes

Some software creators use numbers above 900 for proprietary status codes. This code range is, however, not mentioned in the relevant RFC documents and there are numerous examples of proprietary status codes beyond this range.

Check HTTP Status Codes and Fix Errors

Because browsers do not display the status code, you need special tools to check them, for example, browser extensions such as live Http headers for Firefox or online tools such as web sniffer.

How Do I Remove 404 Errors?

If you are changing your content or the file structure, you can encounter 404 errors. It is best to leave the already existing URLs where they are, or to transfer to them using 301 redirect. If, however, you must restructure your URLs to make them more SEO friendly, for example, you need a strategy to prevent error sites or to keep their number as low as possible.

Work with Google’s tool in its Search Console: Under the heading “Crawling”, you will find the option “Crawling errors”. If you enter your domain, you will learn what internal and external links are not functioning and must be adjusted. If there is a problem, the following message will be shown:

Figure 2: Crawling tester in the Google Search Console.

If there are URLs with 404 errors, you will find them under the sub-item “Not found”. If you click on the incorrect URL, detailed information on the linking will appear. You should regularly call up these lists and fix errors.

More bothersome than “normal” error sites are sites with an improper or “soft 404 error“. They do not show what the user has searched for, but they also do not issue the HTTP status code. A soft 404 error appears when a user requests a non-existent or incorrect website with his or her browser and the server erroneously sends a “200 Ok” or “302 Found” HTTP status code to the client (browser). The user receives a 404 error code on their screen, but the server sends another answer code in the communication between the client and the web server. The content of the website is not connected with the HTTP answer of the server.

This can affect the crawling and the indexing of the requested website. It is also sometimes frustrating for the user because the server answer does not fit his or her search inquiry. Similar designations for a soft 404 error are wrong 404 or soft 303 error codes.

What are User-defined Error Sites?

Most browsers show standard sites delivered by the web server. You can, however, also set up user-defined 404 sites to share additional information about the causes of the error message. In this way, you will improve usability and will direct the attention of the user to your other subpages – and he or she will ideally remain on your web presence and improve the click-through rate (CTR).

Airbnb shows its customers very creatively what they are getting when they remain on the site. Instead of a simple, standard 404 rejection site without added value, those interested find the complete navigation menu.

Figure 3: User-defined error sites from Airbnb.

If you operate an online shop with a strongly fluctuating product range, 404 status codes can become a problem. By establishing original error sites, you will offer frustrated customers an alternative with added value.

How Do I Create a Custom 404 Page?

To create a user-defined error message, you first need to create an .htaccess file and store it in the root directory on the server. If a server cannot locate the website it is searching for, it will look in the .htaccess file. The file directs the server behavior and issues the error page in the case of a 404 error. The following line should be entered (on an Apache server):

Then you create a similarly named file in the editor: “error-404.html” and save it in the root directory as well. This file explains why the error message is appearing and what the visitor can do next to find the desired information. This is where it can really come in handy to have installed a search function on your site. A simple but user-friendly option is available from the company amCharts.

Figure 4: Error Site by amCharts

The “error-404.html” code in the amCharts file contains the following elements:

the actual error message:

the prompt for the search function:

the link to the correct starting page:

While the amCharts example contains the most important information for the reader, it is not very charming and is poorly designed. Make sure your error site matches the design and style of your web presence, including navigation for better orientation, and a link to the starting page. Creativity is welcome, but jokes or inappropriate comments such as “Good job, you crashed my website!” are not. Remain polite, explain to your readers why the page is being displayed (e.g. page no longer exists; address has been changed) and offer them precise information and alternatives for further steps.

How Do I Properly Forward Websites with the 301 Redirect?

Restructuring URLs, or even completely changing the domain, presents great challenges for webmasters. If you don’t want to risk your site sliding immediately into the black hole of search results, you must pay attention to a few things.

1. The oldest subpage should be forwarded to the new subpage 1:1, not to the start page. In this way, you can avoid URLs with valuable external links losing their Linkjuice.

2. Long forwarding times or chains are counterproductive, because the Googlebot will give up after the fifth redirection, at the latest.

3. With a redirect, the old address can be a relative URL. The goal of the redirect should, however, be an absolute URL.

Example of a relative URL (path to the linked document after the forward slash):

Example of an absolute URL (including the domain):

To send a status code 301, you can, for example, create a suitable header in PHP-based apps and enter the following PHP code into the old file “the-old-name.php”.

Because the site can be called up using “domain.de/the-old-name.php”, visitors will be redirected to the new URL “domain.de/the-new-name.php”. In this way, the HTTP status code 301 Moved Permanently is given.

If you are working on a Linux server with an activated Apache mod-rewrite module, you can send the status code 301 with an .htaccess file. The .htaccess file must contain the following lines of code:

Conclusion

You should not underestimate the significance of error messages for your search engine optimization. You should always know which status codes will be called up for your website. Careless mistakes such as spelling errors in the URL or carelessly restructured directories can have fatal consequences for your visibility in the SERPs.

Particular care is demanded in domain changes. The forward from the old to the new site should always take place via 301 (Redirect), not through 302 (Moved Temporarily). If this isn’t your thing, leave it to your programmer and take care of the creative side: the design of your completely individual error page!

To end this article, we have put together for you a graphic overview from the status codes discussed above. Enjoy!