Checks · index

Blocked by robots.txt but Linked

Warning index.blocked_robots

What it means

Internal links point at a URL that robots.txt disallows. Crawlers see the links but cannot fetch the page, so the link equity goes nowhere.

How to fix it

Either allow the URL in robots.txt, or remove the internal links to it. Note that robots.txt blocks crawling, not indexing — to keep a page out of the index use noindex on a crawlable URL.

Where to find it in Truelint

Open the Issues pane on the right of the workspace and select Blocked by robots.txt but Linked. The grid filters to every affected URL and the detail pane shows the specific finding for each. Export the list with Export → Current view, or from the API:

GET /api/v1/crawls/{handle}/issues/index.blocked_robots