Why Google isn’t indexing your website: diagnose and fix the cause

A new or redesigned website does not automatically appear in Google. If important pages are not indexed, people may not find them through organic search. The cause could be a technical restriction—or simply a page Google has not discovered or processed yet.

This guide explains how to check the status, distinguish common causes, and troubleshoot methodically. Remember: crawling, indexing, and ranking are separate steps. A page can be crawled without being indexed; an indexed page can still rank poorly.

1. Check the status before guessing

Open Google Search Console for your site and use URL Inspection on a specific affected page. Check whether Google knows the URL, whether it is indexed, and what information is shown about its latest processing.

Distinguish these situations:

• Google does not know the URL: It may not have discovered it yet.
• Crawled, currently not indexed: Google fetched it but did not add it to the index.
• Excluded from indexing: Search Console may report a reason that needs investigation.
• The URL is indexed: The issue may be ranking, query relevance, or visibility rather than indexing.

Search Console data can lag, so interpret a status in context.

2. Common causes and checks

An unintended noindex

A page can be excluded through a meta robots tag or an X-Robots-Tag HTTP header. Inspect the HTML and HTTP response. If the page should appear in Google, remove the unintended directive—but only after confirming the exclusion was not deliberate.

robots.txt blocks crawling

A robots.txt rule may prevent Googlebot from fetching a URL or resources. Check /robots.txt and the relevant rule. Note that robots.txt controls crawling; it is not a reliable way to remove a URL from the index. In some circumstances, a blocked URL can still appear without its full content.

The canonical points elsewhere

A canonical tag indicates the preferred version of a URL. If it mistakenly points to another page, Google may select that destination instead. Confirm that the canonical is intentional and points to a relevant, accessible URL.

Redirects or HTTP errors

A redirected URL may not be indexed as a separate page. Check redirect chains, loops, and status codes such as 404 or 5xx. Do not redirect a removed page to an unrelated destination simply to avoid an error.

The page is new or poorly linked internally

Google first needs to discover a URL. Link important pages from relevant, accessible pages and include canonical URLs in a current XML sitemap. A sitemap can help discovery, but it does not guarantee indexing.

Content is thin, duplicated, or unhelpful

Google may choose not to index a page that is very thin, duplicated, or of limited use. Check whether it has a clear purpose, answers the search intent, and provides distinct value. Adding more words or keywords alone is not a guaranteed fix.

The site is unreliable or blocks legitimate crawlers

Server errors, unstable responses, or security controls that block legitimate search crawlers can interfere with processing. Review logs, status codes, and security rules. Avoid disabling protections broadly; test narrowly and preserve your security requirements.

3. A practical troubleshooting sequence

1. Choose the affected URL. Inspect the actual important page, not only the homepage.
2. Review Search Console. Read the indexing status and URL Inspection details.
3. Test the live page. Confirm it is reachable and returns the intended content.
4. Check indexing directives. Review robots.txt, meta robots, and X-Robots-Tag.
5. Check canonicals and redirects. Ensure the desired URL is the appropriate canonical version.
6. Review internal links and sitemap. Link to the page naturally and include it in the sitemap if it should be indexed.
7. Assess the content. Improve originality, clarity, and usefulness for the intended audience.
8. Recheck after fixing. Use live URL inspection and, if appropriate, request indexing in Search Console. A request is not a guarantee and should not be repeatedly submitted without reason.

4. Pay special attention after a redesign

A redesign can remove old URLs, introduce new ones, or leave redirects missing. Map old URLs to their replacements, test important redirects, and review the sitemap. Also check whether a staging setting—such as noindex—was accidentally carried over to the live site.

If many pages disappear from the index at once, look for a shared cause, such as a sitewide robots rule, template mistake, or server issue, rather than treating every URL as an isolated problem.

5. Checklist

• Inspected the specific URL in Search Console.
• Live page returns the expected status and content.
• No unintended noindex directive.
• robots.txt does not accidentally block important pages.
• Canonical points to the intended, relevant URL.
• Redirects work and do not loop.
• Page has sensible internal links and, where appropriate, appears in the sitemap.
• Content is useful, distinct, and aligned with search intent.
• Status was checked again after the fix.

Diagnosis in ten minutes: the right order

Work from the outside in and you save time:

1. Open Search Console and enter the affected address in URL inspection. Note the status shown.
2. Reachability: does the page load without errors (status 200), without login and without a redirect chain?
3. Look for blocks: is there a noindex in the source or the HTTP header? Does robots.txt block the path?
4. Check the canonical: does the page point to itself or to a different address?
5. Sitemap and linking: is the address in the sitemap and linked internally?
6. Judge the content: is the page distinct, helpful and not nearly identical to another?

Fix one cause at a time, then have the address inspected again in Search Console.

Conclusion

When a website is not indexed, start with a specific URL and reliable status information. Separate crawling, indexing, and ranking; inspect technical directives; and improve content if it offers little distinct value. Fix the underlying cause and check again—without promising instant indexing or higher positions.

Official resources:

• Google Search Central — How Search works: https://developers.google.com/search/docs/fundamentals/how-search-works
• Google Search Central — Robots meta tag: https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag
• Google Search Central — robots.txt: https://developers.google.com/search/docs/crawling-indexing/robots/intro
• Google Search Console Help: https://support.google.com/webmasters/