How to Use the XML Sitemap Validator

What a valid XML sitemap looks like

An XML sitemap is a file that lists the important pages of a website. It helps search engines discover and crawl all the pages on a site. A valid XML sitemap must adhere to a specific structure. The root element is <urlset>, which contains one or more <url> elements. Each <url> element includes a <loc> element specifying the URL of a page. Other optional elements like <lastmod> (last modification date), <changefreq> (how frequently the page is likely to change), and <priority> (page's priority relative to other pages on the site) can also be included.

Here is an example of a simple XML sitemap structure:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://www.example.com/</loc>
    <lastmod>2023-10-26T10:00:00+00:00</lastmod>
    <changefreq>daily</changefreq>
    <priority>1.0</priority>
  </url>
  <url>
    <loc>https://www.example.com/page1</loc>
    <lastmod>2023-10-25T15:30:00Z</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.8</priority>
  </url>
</urlset>

The xmlns attribute specifies the XML namespace. This attribute is required and should not be changed.

The LemRank xml sitemap validator checks for correct XML syntax, valid URL formats within <loc> tags, and the presence of required elements. It also ensures that the sitemap uses the correct character encoding, which should be UTF-8.

Google's sitemap size and URL count limits

Google imposes specific limitations on the size and content of individual XML sitemap files. Each sitemap file must not exceed 50,000 URLs and its uncompressed file size must be less than 50MB. These limits are in place to ensure sitemaps are manageable for search engines to process efficiently.

If a website has more than 50,000 URLs or its sitemap file would exceed 50MB, it is necessary to split the sitemap into multiple smaller sitemaps. A sitemap index file is then used to list these individual sitemaps. The validator will identify if your sitemap exceeds these limits and recommend using a sitemap index file if applicable.

When a sitemap exceeds these limits, search engines may not fully process it, leading to some pages not being discovered or crawled. Adhering to these limits helps ensure comprehensive coverage of your website's important pages by search engines.

Common validation errors

The LemRank xml sitemap validator identifies various structural and content errors that can prevent search engines from effectively processing your sitemap. Understanding these common errors can help you maintain a healthy sitemap.

Error Type	Description	Example
Invalid XML Syntax	The sitemap file does not conform to the XML standard. This can include issues like unclosed tags, incorrect character usage, or missing XML declaration.	Missing <?xml version="1.0" encoding="UTF-8"?> declaration or an unclosed <urlset> tag.
Missing Required Elements	An <url> entry is missing the mandatory <loc> element.	<url><lastmod>...</lastmod></url> instead of <url><loc>...</loc><lastmod>...</lastmod></url>.
Invalid URL Structure	The URL specified within a <loc> tag is not a valid URL format or contains invalid characters.	<loc>example.com/page with spaces</loc> (spaces are invalid in URLs without proper encoding) or <loc>htp://invalid-url</loc>.
Incorrect Namespace	The xmlns attribute in the <urlset> tag is missing or incorrect.	<urlset> instead of <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">.
Unsupported Encoding	The sitemap file is not encoded in UTF-8.	A sitemap saved with ISO-8859-1 encoding.
Duplicate URLs	The same URL appears multiple times within the sitemap. This does not always cause an error, but it is inefficient and can indicate an underlying issue with sitemap generation.	<loc>https://example.com/page1</loc> appearing twice in the same sitemap.
URLs not on the same domain	All URLs within a sitemap are expected to be on the same domain as the sitemap itself.	A sitemap hosted on www.example.com includes a URL for www.another-domain.com.
Sitemap Size / URL Count Exceeded	The sitemap exceeds Google's limits of 50,000 URLs or 50MB uncompressed file size.	A sitemap containing 50,001 URLs, or a sitemap file size of 51MB.

The validator provides detailed feedback for each error found, including the line number and a description of the issue, enabling you to pinpoint and fix problems efficiently.

Checking for dead URLs (404s and 5xxs)

A critical aspect of sitemap validation is identifying dead URLs, which are pages that return HTTP status codes indicating an error (e.g., 404 Not Found, 500 Internal Server Error). While the LemRank xml sitemap validator does not crawl every single URL, it performs a spot-check on up to 100 sampled URLs within your sitemap to identify common HTTP status codes.

Here is a breakdown of common HTTP status codes:

HTTP Status Code	Description	Impact on SEO
200 OK	The request was successful, and the server returned the requested resource.	Ideal. Indicates a healthy, crawlable page.
301 Moved Permanently	The requested resource has been permanently moved to a new URL.	Generally positive if implemented correctly to pass link equity to the new URL. The sitemap should eventually be updated with the new URL.
302 Found (Temporary Redirect)	The requested resource has been temporarily moved to a different URL.	May dilute link equity. For permanent changes, 301 is preferred. The sitemap should list the original URL if the move is truly temporary.
404 Not Found	The server cannot find the requested resource.	Negative. Search engines may de-index the page, and users encounter broken links. High rates suggest poor site maintenance or outdated sitemap entries.
410 Gone	The requested resource is no longer available and will not be available again.	Similar to 404, but explicitly communicates that the resource is intentionally removed. Search engines will typically de-index faster.
500 Internal Server Error	A generic error message, given when an unexpected condition was encountered.	Negative. Indicates server-side issues preventing access to the page. Can severely impact crawling and indexing if widespread.
503 Service Unavailable	The server is currently unable to handle the request due to temporary overloading or maintenance.	Negative if prolonged. Search engines may temporarily pause crawling. Indicates a temporary issue, but extended periods harm SEO.

Sitemaps with a high rate of dead URLs (404s or 5xxs) can negatively impact your website's SEO. Search engines may crawl your site less frequently if they repeatedly encounter non-existent or inaccessible pages. This can lead to slower indexing of new content and a potential decrease in search rankings. It is therefore crucial to regularly review your sitemap for dead URLs and update it to reflect the current status of your website's pages.

Sitemap index files for large sites

For large websites that exceed Google's sitemap limits (50,000 URLs or 50MB uncompressed file size per sitemap), a sitemap index file (sitemap.xml) is necessary. A sitemap index file acts as a list of other sitemap files, allowing you to organize your site's URLs across multiple sitemap files while still providing a single point of entry for search engines.

The structure of a sitemap index file is similar to a regular sitemap, but instead of <url> elements, it uses <sitemap> elements, each pointing to an individual sitemap file.

Here is an example of a sitemap index file:

<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://www.example.com/sitemap1.xml</loc>
    <lastmod>2023-10-26T10:00:00+00:00</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://www.example.com/sitemap2.xml</loc>
    <lastmod>2023-10-25T15:30:00Z</lastmod>
  </sitemap>
</sitemapindex>

Each <sitemap> element must contain a <loc> tag specifying the full URL of a sitemap file. <lastmod> is an optional element indicating the last modification time of the sitemap file.

The LemRank xml sitemap validator can process both individual sitemap files and sitemap index files. When you submit a sitemap index file, the validator will check its structure and then attempt to validate each listed individual sitemap file for errors and dead URLs, providing a comprehensive report for your entire site.

Submitting the fixed sitemap to Google

After validating and correcting any issues in your XML sitemap, the final step is to submit it to Google Search Console. This process informs Google about the updated sitemap, prompting it to crawl and index your website's pages more effectively.

1. Access Google Search Console: Log in to your Google Search Console account. If you have not already, add and verify your website property.
2. Navigate to Sitemaps: In the left-hand menu, click on "Sitemaps" under the "Index" section.
3. Add New Sitemap: In the "Add a new sitemap" section, enter the relative path to your sitemap file (e.g., sitemap.xml or sitemap_index.xml).
4. Submit: Click the "Submit" button.

Google will then attempt to process your sitemap. The "Sitemaps" report in Search Console will display the submission date, the status of the sitemap, and the number of URLs discovered. It may take some time for Google to fully process the sitemap and reflect the changes in its index. Regularly checking this report can help you monitor the health and indexing status of your website.

Related

• How to Use the Robots.txt Tester