Introduction of our crawler and service
We are Ryte - a Website Quality Assurance Software. We fight for a cleaner web. No more broken pages, no more links to deleted contents and better performance for all. Please help us to make the web a little better.
If you block our crawler, our clients might think that they are referencing (aka linking to) a broken page. In order to keep their website up to date and clean, they might be inclined to remove this reference to your page based on this false information.
By default we use the UserAgent "Googlebot 2.1 (compatible; OnPageBot; +https://ryte.com/bot/)". But our users can set any useragent in their project settings.
We recommend our clients to use the Useragent "Googlebot" in order to reveal so called "cloaking" which is prohibited in the Webmaster Guidelines of Google. By visiting the website as Googlebot we offer our clients a way to identify if this technique is used on their domain (or referenced pages).
Our clients can schedule the crawler. By default the crawler starts on demand only. Clients can schedule a time slot for crawling their website via the crawl settings. Crawling is possible at any time.
We use a wide-range of IP-addresses to crawl the website of our clients. If you need further guidance regarding whitelisting our service please contact email@example.com.
If your page is beeing crawled without your permission and you wold like to stop then please contact us at firstname.lastname@example.org.
The speed of our crawlers is also determined by the project settings. The default setting are 10 parallel requests - simulating 10 users clicking from page to page on our clients domain. This setting can be decreased. If a user has verified the ownership of a domain he can also increase the amount of parallel requests (and with that the amount of requests per second).
This is our monitoring service. Our clients can define their most important pages (by default their homepage) which get constantly monitored. This is done to inform our users if their website is facing an issue (like a server downtime, bad database performance, etc.).