Semetrical Launches Free Robots.txt Tool
Here at Semetrical, we’re excited to announce our latest release to market, Tomo, a new, free robots.txt monitoring tool.
What is Tomo?
Tomo gets its name from the Japanese word for ‘friend’ and is a free SEO tool that helps marketers avoid negative traffic impacts and crawling issues on their website caused by an incorrectly configured robots.txt file. It quickly shows users which of their website’s URLs are available to multiple crawlers, including Googlebot, helping users to improve their overall crawl efficiency.
The Tomo app allows users to configure tests for multiple URLs and user agent tokens, which can be run against their live website robots.txt file to see if it produces expected results. Tomo can also be used for testing new robots.txt files in development before they go live.
Tomo’s unique ability to alert users on live robots.txt changes quickly, allows for a rapid response to mitigate the impact of erroneous or unexpected changes.
What are the benefits of Tomo?
Having a well-configured robots.txt file is crucial to ensure websites are crawled efficiently by search engine crawlers. By preventing low-value URLs from consuming crawl budget, users can enhance the indexing of their website in search engines and reduce server costs. Moreover, with Tomo, you can avoid robots directives that are preventing valuable pages from being crawled and indexed.
5 benefits of Tomo technology
The main benefits of using Tomo technology are:
- You can avoid any unexpected robots.txt changes going unnoticed, without wasting time on false positives
- You gain the ability to optimise crawling efficiency to improve website indexing and reduce server costs by blocking low value URLs
- You are able to avoid negative impacts on your search engine traffic by identifying URLs that are blocked by mistake
- You can update and improve your robots.txt files with confidence that there are no errors
- You can easily manage notifications for a large number of websites using project grouping
What do the experts say?
Tony King, Director of Digital Strategy and co-founder of Semetrical, comments:
“The robots.txt file is the first port of call for a search engine when visiting your website. It’s a text file that lists instructions for different user agents and essentially tells web crawlers which parts of a site should be crawled and which should be ignored. The main instructions used in a robots.txt file are specified by an “allow” or “disallow” rule.
“Many SEO’s will already be comfortable working with simple robots.txt files, but large sites can command complex robots.txt files and managing a portfolio of sites can be unwieldy. Tomo can help efficiently manage and test your portfolio and avoid any negligent traffic drops or potential search engine blocks. Tomo will help you accurately produce and manage your robots.txt file, whilst helping you avoid any issues in the future.
“Tomo alerts can be set up through email, Slack or Teams notifications to alert the right people in your team quickly and efficiently”.
