All marketers must ensure their websites are indexable to get more users and traffic to their websites, to ensure higher conversion rates. This article helps you ensure your website is indexable.
<meta name="robots" content="noindex, follow"/>This tag can be a useful way to avoid duplicate content, and can also be used for example before a domain transfer, to test the website before the actual launch. (Although when your site then goes live, the Noindex tag should of course be removed.) Under Search Engine Optimisation in the report "Indexability", you can check with a few clicks which pages are indexable.
Figure 1: Check your indexability with Ryte
Figure 2: Check your robots.txt with Ryte
RewriteEngine OnRewriting require using:
RewriteBase /Define the rule that the server is to execute:
RewriteEngine On RewriteBase / RewriteRule seitea.html seiteb.html [R=301]Of course, it’s possible that the file may have been named incorrectly and therefore cannot rewrite or redirect URLs. As a result, both users and search engines will not be able to access the pages, and they are therefore not crawled or indexed.
Figure 3: Check canonical tags with Ryte
Figure 4: Server monitoring with RyteTip: Check the HTTP status codes of your site regularly to see if 301 redirects work correctly or if 404 status codes exist. Pages with this status are untraceable to potential readers and web crawlers. Links referring to such pages are called "dead links".
Figure 5: Find pages without incoming linksRyte Website Success shows you orphaned pages quickly. To do this, click on the "Pages without inbound links" report in the "Links" section.
Figure 6: Check your sitemap.xml for mistakes with RyteYou can also check your sitemap for errors with the Google Search Console - a warning notice indicates possible problems with indexing.
|1||Noindex-Tags||Check your URLs for the no-index tag. Unless this tag is completely necessary, change it to "index, follow".|
|2||Robots.txt||Check your robots.txt file and see if important directories are excluded from crawling.|
|3||.htaccess||Check this file for incorrect redirects or syntax mistakes.|
|4||Canonical-Tags||Check that these tags correctly refer to the canonical URL.|
|5||Server availability and status code registrations||Monitor the availability of your server and check the status codes of your URLs.|
|6||Orphan pages||Find pages without incoming links and create internal links.|
|7||Content theft||Check whether external websites use your content. Create canonical tags and avoid relative URLs.|
|8||Internal nofollow-Links||Search for nofollow-tags on your website, and remove them. Alternatives are canonical- or noindex-tags.|
|9||XML-Sitemap||Check whether your sitemap contain all URLs to be indexed, and check the status codes of the URLs.|
|10||Hacking||Look out for warnings regarding hacked pages in the Google Search Console, and, for example, change your login details.|
Check your website's indexability with Ryte FREEStart now!
Published on 07/04/2020 by Olivia Willson.
Who writes here
After studying at King’s College London, Olivia moved to Munich, where she joined the Ryte team till 2021. She was previously in charge of product marketing and CRO, and also helped out with SEO and content marketing. When she's not working, you can usually find her outside, either running around a track, or hiking up a mountain.