The Complete Sitemap Guide

Introduction to Sitemaps

A sitemap is a file where you provide information about the pages, videos, and other files on your site, and the relationships between them. Search engines like Google read this file to more intelligently crawl your site.

A sitemap tells the crawler which files you think are important in your site, and also provides valuable information about these files: for example, for pages, when the page was last updated, how often the page is changed, and any alternate language versions of a page.

Do you need a sitemap?

If your site's pages are properly linked, Google's web crawlers can usually discover most of your site. In that case, a sitemap can still improve the crawling of your site, particularly if your site meets one of the following criteria:

• Your site is really large. As a result, it's more likely that Google web crawlers might overlook crawling some of your new or recently updated pages.
• Your site has a large archive of content pages that are isolated or not well linked to each other. If your site pages don't naturally reference each other, you can list them in a sitemap to ensure that Google does not overlook some of your pages.
• Your site is new and has few external links to it. Googlebot and other web crawlers crawl the web by following links from one page to another. As a result, Google might not discover your pages if no other sites link to them.
• Your site has a lot of rich media content (video, images) or is shown in Google News. If provided, Google can take additional information from sitemaps into account for search, where appropriate.

Formats Supported by Search Engines

Google supports several different sitemap formats:

1. XML (Extensible Markup Language)

The most common and highly recommended format. It is strictly structured, machine-readable, and allows for additional metadata like the last modification date. Read our deep-dive on XML sitemaps.

2. RSS, mRSS, and Atom 1.0

If you have a blog with an RSS or Atom feed, you can submit the feed's URL as a sitemap. This provides Google with a feed of your most recent updates, though it typically only contains recent URLs rather than your entire site structure.

3. Text

If your sitemap includes only web page URLs, you can provide Google with a simple text file that contains one URL per line. This format does not support metadata like <lastmod>.

XML Sitemap Limitations

Regardless of the format you choose, sitemaps are subject to standard limitations across all search engines:

• Maximum URLs: A single sitemap file can contain a maximum of 50,000 URLs.
• Maximum File Size: The file size cannot exceed 50 MB uncompressed.

If you have more than 50,000 URLs, you must split your list into multiple sitemaps and use a Sitemap Index File to submit them together.

How to Submit a Sitemap

Creating the file is only half the battle. You must ensure search engines can find it:

1. Google Search Console: Navigate to Sitemaps under the Indexing section, enter your sitemap URL, and click submit.
2. Robots.txt: Add the directive Sitemap: https://www.yourdomain.com/sitemap.xml anywhere in your robots.txt file. This helps all web crawlers, not just Google, discover your file automatically.