Quality Rater Guidelines

The Quality Rater Guidelines are very extensive guidelines in the form of questionnaires for manual quality inspection of websites that Google issues to “Search Quality Evaluators” (short: raters) who are part of the Google staff. The Google search results are supposed to be improved based on the results of such website evaluations. Typically, these guidelines are not intended for the public. But there are always leaked versions that are anonymously put on the Internet.

Background

The quality of the Google search has evolved steadily, thanks to the development of the search algorithms in recent years. Although search engine robots can automatically analyze and evaluate link structure, design, technical requirements, and content based on numerous algorithms, Google still to date, relies on the work of people to check the quality of websites. Despite all the advanced technical capabilities, a review with human eyes is often more precise and on point.

The work of the Search Quality Evaluators is not, however, primarily geared toward examining existing websites, but rather to evaluate the effectiveness of existing algorithms based on search results indirectly and improve them through manual checks. Accordingly, the work of raters provides important data for future Google updates. One could therefore also call them “Search Algorithm Checkers.” According to rumors, several thousand Search Quality Evaluators work for Google. Each quality rater is used for checking the SERPs in his native language.

In order to standardize and simplify the raters’ work, they receive a comprehensive document with criteria for evaluating websites. This document is the Quality Rater Guidelines. The content of these guidelines is actually reserved for internal use. In 2013, Google released just one version of the Quality Rater Guidelines with the version number 1.0 (as of October 2014). Since then the guidelines remained eyes-only for site evaluators. But in 2014 a new version of the Quality Rater Guidelines was leaked. The document circulated like wildfire on the net.

Webmasters and SEOs could draw conclusions about their own optimizations based on the specifications formulated in the Guidelines.

The Quality Rater Guidelines are composed of two major sections, the Page Quality Rating Guideline and the Utility Rating Guideline.

Structure

The central question that an SQR (Search Quality Rater) is supposed to ask themselves is whether the visited website is trustworthy and of high quality. Based on this basic, individual page elements are examined and the page rated accordingly.

Classification of websites

For the assessment of a site the raters have five different ratings to choose from.

• High Quality Pages serve their purpose and satisfy the user expectations well.

• Highest Quality Pages fulfill their purpose and the user expectation extremely well.

• Low Quality Pages do not or inadequately fulfill their purpose and give an unsatisfactory user experience.

• Lowest Quality Pages harm users or disappoint them. They only exist to generate money without any user benefit.

• Medium Quality Pages have no particular errors in the eyes of the examiner, but are nothing special.

Criteria for the evaluation of websites

To arrive at a rating for a website, quality raters must examine them based on various aspects. These aspects are in turn based on sub-sections.

Rating of content

• Main Content (MC): This area includes all content elements, i.e. text, images or videos that serve the actual purpose or object of the site.
• Supplementary content (SC): This content provides a good user experience.
• Advertising: The site should be rated based on the relationship of its main purpose and the on-screen advertising.

Review of navigation and legal information

In the Quality Rater Guidelines, raters are instructed to look for imprints or information on the authors of the websites. It should also be checked how easy it is to return to the homepage or get to individual categories. The option to establish contact with the operators of the site is included as part of the assessment.

Review of the website maintenance and web design

This analysis part answers the question of whether a website is regularly maintained. This includes whether the links are functional, the content is updated, and compatibility with new browsers. Similarly, the functionality of the web design is analyzed and assessed.

Review of the reputation of the website

This part concerns less an analysis of the site itself, but rather research for external and credible reviews of the examined domain. Google gives its employees at this point many tips on how you can search for clues to the reputation of a website and how to rate those sources.

The rating scale

Every single aspect can be evaluated by a Quality Rater with a slider. The range includes five different evaluation stages and intermediate gradations are possible.

The evaluation steps are: Lowest, Low, Medium, High, Highest

E-A-T

An important element of the Quality Rater Guidelines is the E-A-T-formula. These are three key components of a high-quality website, including its expertise (E), Authority (A) and Trustworthiness (T). In short, anyone running a high-quality website must be very familiar with the topic of his website and convey it, should be highly regarded for their knowledge, and must provide trustworthy content.

The assessment of these different areas in turn depends on the respective industry of the site.

Review of the utility of a website

The second part of the guidelines deals with the assessment of the utility of a website. The focus is on understanding user intention and to assess the content based on the presumed user intent.

The ratings get subdivided in relation to the “result blocks” in the SERPs. The quality raters enter a keyword in the search bar and get a result. From the snippets shown and the clicked link, they can then assess whether the user expectation is fulfilled or not.

Five different rating levels are available.

• vital
• useful
• relevant
• slightly relevant
• off-topic or useless

In a separate division, a rater can mark, whether a target link has pornographic content, content in another language or inaccessible content. In a further step, it can also be tested, how good the results are in terms of local searches.

Benefits for SEO

Anyone familiar with the content of the Quality Rater Guidelines, can use this advantage in optimizing their website. However, many areas of the guidelines are already “common sense” in website optimization. Accordingly, no great secrets are being revealed no matter how much hype there is about the leaked versions.

Web Links

• http://moz.com/blog/16-insights-into-googles-rating-guidelines insights into Google’s rating guidelines