Guides · 4 min read · Updated 2026-09-08

XML sitemap best practices: what belongs in it, what doesn't, and how to keep it honest

A sitemap only helps when search engines trust it. The rules for what to include, the mistakes that make Google ignore your sitemap, and how to generate a correct one from a crawl.

An XML sitemap is a list of URLs you’d like search engines to crawl, with optional hints about when each last changed. It doesn’t make pages rank and it doesn’t force indexing. What it does is help discovery on large sites, on new sites with few links, and for pages deep in the architecture, and it gives you a signal in Search Console: how many of the URLs you submitted actually got indexed. A sitemap full of URLs that redirect, error, or carry noindex teaches search engines to discount it, and the useful signal disappears.

What belongs in a sitemap

Only URLs that meet all of these:

• Return 200. Not redirects, not errors.
• Are indexable. No noindex, not blocked by robots.txt.
• Are canonical. Each URL’s canonical tag points at itself. Never list the duplicate; list the canonical.
• Are the final form. HTTPS, the right hostname, trailing slash consistent with the rest of the site.
• You want indexed. Not internal search results, not filtered listings, not account pages, not tag pages with two posts.

That’s the whole list. Everything else is optional metadata.

What doesn’t belong

• Redirecting URLs, including old URLs after a migration. The sitemap should reflect the site as it is.
• Paginated pages beyond the first, unless they carry unique content you want indexed (they usually do for blogs, rarely for categories).
• Parameter variants and sorted views.
• Pages returning soft 404s.
• URLs on other hostnames (a separate sitemap per host, referenced from that host’s robots.txt).
• Images and videos, unless in a dedicated image or video sitemap with their extensions.

The metadata, honestly

lastmod is the one field Google uses, and only if it’s accurate. If every URL says today’s date, or the deploy date, Google learns it’s meaningless and ignores it. Set it from the content’s actual last change; if you can’t, leave it out.

changefreq and priority are ignored by Google and have been for years. Including them is harmless but a sitemap that sets every page to priority=1.0 is a small tell that nobody maintains it.

Structure and limits

• 50,000 URLs or 50 MB uncompressed per file. Beyond that, split into several sitemaps and list them in a sitemap index.
• Gzip is fine (sitemap.xml.gz).
• Reference the sitemap (or index) in robots.txt with a Sitemap: line, and submit it in Search Console and Bing Webmaster Tools.
• One sitemap per hostname and protocol; a sitemap on https://www. can only list https://www. URLs.
• UTF-8, with & escaped as & in URLs.

The mistakes that make it worthless

Generated from the database, not the site. The CMS lists every record, including drafts, unpublished translations and products that return 404. The sitemap should be generated from, or validated against, what the site actually serves.

Never updated after a migration. Old URLs stay for years, redirecting. Each is a wasted fetch and a trust loss.

Includes noindexed pages. A page that says “index me” in the sitemap and “don’t index me” in its head is a contradiction; the head wins and the sitemap looks wrong.

Orphans only in the sitemap. Pages with no internal links, discovered only through the sitemap. They get crawled and rarely rank. The fix is links, not sitemap entries.

Missing pages. The opposite problem: important pages that the crawl finds but the sitemap omits, because the generator doesn’t know about that content type.

Sitemap versus crawl: the audit

The useful audit is a comparison of three sets: URLs in the sitemap, URLs the crawl found, and URLs that are indexable. Four findings fall out:

1. In sitemap, not indexable (redirects, errors, noindex): remove from the sitemap or fix the page.
2. In sitemap, not found by crawl (orphans): add internal links or remove.
3. Found by crawl, indexable, not in sitemap: add to the sitemap.
4. In sitemap, canonical elsewhere: replace with the canonical.

Truelint’s Sitemaps tab computes all four from the sitemaps referenced in robots.txt and the crawl, with counts and lists. The sitemap.orphan finding is the one most sites have never seen.

Generating a correct sitemap from a crawl

The simplest way to get a sitemap that is right by construction is to build it from a crawl: indexable, 200, self-canonical pages, split at 50,000 with an index, lastmod from the Last-Modified header where the server sends one. Truelint does this from the Export menu or with truelint sitemap site.crawl --out ./sitemaps/. Regenerate after each significant deploy, or schedule it.

Special cases

News sites use the News sitemap extension for articles from the last two days. Image-heavy sites benefit from image sitemaps listing images per page. Video likewise. Hreflang can be declared in sitemaps instead of page heads, which keeps large multilingual sites consistent, provided the sitemap and the pages don’t both declare it differently.