Google Webmaster Guidelines

The Google webmaster guidelines define the actions that webmasters can take to make their websites better indexable or crawlable. Moreover, the webmaster guidelines include a list of procedures that Google considers breaches of these guidelines which may result in the devaluation of a website or even exclusion from the Google index. The search engine Bing also has their own Webmaster Guidelines, based on the same principles as the Google Webmaster Guidelines.

Conditions for the validity of the guidelines

Google webmaster guidelines apply to all websites that Google has added to its index. If you want your website to be permanently displayed in the SERPs, you should be striving to comply with these requirements.

The guidelines are based on three central aspects. Webmasters should support Google in finding websites (crawling). Secondly, webmasters should help Google to classify and "recognize" the content (indexing). Thirdly, webmasters should support the users of the websites in their use (usability and user experience). Essentially, according to the Google Webmaster Guidelines, a website should support the search engine in crawling and indexing and ensure a good user experience.

Content and design

When designing the content, the Google Webmaster guidelines emphasize that a website should be primarily created for users and not for search engines. A page should be clearly structured - this applies to both the navigation an the heading structure. These aspects should also accurately reflect the content of a website. Here Google refers to the practice of some webmasters who optimize their content for keywords and neglect the information value of the text. However, the guidelines also state that webmasters should include keywords on their page where they anticipate their users will be searching for these queries. 

Links play a major role as well. Google recommends that each page should be accessible with at least one incoming link, but the total number of links on a page should also remain at a "reasonable level." Google specifically references the work involved for the bot for following each link. If it encounters too many links on a page, it cannot crawl a website in its entirety. Google advises webmasters to create a summary page for their visitors which includes all possible subpages. The term “sitemap” is a bit confusing in this context, since the way it is used by Google in the guidelines has nothing in to do with an XML sitemap. Broken links should also be avoided whenever possible.

As a general rule, a website should offer all important information in text form, as the crawlers cannot yet read images. Images can be optimized using ALT attributes.

Technology

In the technical part of the Google webmaster guidelines, the robots.txt file plays an important role. Google explains the advantages of this file and mentions that the crawling of certain directories can be prevented. The same file can also be used to index AdSense content. In addition, the robots.txt file is meant to prevent visitors being presented with irrelevant content in the SERP.

Another section of the technical guidelines addresses the verification of your published pages. Google recommends using a text browser to check whether all of the content can be displayed and that its display is not hindered by JavaScript or other functionalities. It should also be checked whether a bot can easily crawl the page, without being misled by session IDs or other parameters. Finally, any webmaster should test whether their website pages can be opened with different browsers and select a CMS with which the pages can be indexed easily.

Great importance is also placed on the loading time of a website. Since it is a ranking factor, every webmaster should take this aspect seriously and provide fast access to their website.

Quality guidelines

In the “Quality Guidelines” section, the Google webmaster guidelines lists all common forms of SERP manipulation - SEOs should ensure that they are not following these manipulative practices. Google also encourages users to report spam sites to the Google anti-spam team.

The most cited guideline is this:

“Make pages primarily for users, not for search engines.”

A Google webmaster guideline frequently mentioned by Matt Cutts is:

“Think about what makes your website unique, valuable, or engaging. Make your website stand out from others in your field.

In a nutshell, SEO can be described with just these two statements. But they don’t specify the methods to be used.

Here are some actions that are NOT recommended:

• Auto-generated content
• Link exchange schemes
• Cloaking
• Hidden text
• Inserting copied content from other sites on your own pages

If a website is declassified by Google or a penalty is levied, then there is usually a prior warning and the indication that the webmaster has to change something. If a domain was completely removed from the index, a reconsideration request can be submitted so that the affected website can be rechecked and ideally be re-indexed.

Violations of the Webmaster Guidelines

If webmasters violate Google's guidelines, the consequences range from a temporary penalty for individual URLs to the exclusion of an entire domain from the index. Typically, Google Search Console provides users with hints about penalties, such as a warning about unnatural links. After the aspects described in the warning have been resolved, webmasters can use a so-called "Reconsideration Request" to request a new check of the URLs concerned.

Guidelines for AMP Pages

With the introduction of the Accelerated Mobile Pages, Google has extended its Webmaster Guidelines to include guidelines for these websites. The guidelines refer to the specific requirements that an AMP page must meet in order to be indexed by Google and displayed in the mobile SERPs.

Web Links

• Google Webmaster Guidelines