Tip

Discovered or Crawled but Not Indexed: The Fixes That Worked

The exact sequence that moved a new site out of the not-indexed buckets — two invisible server bugs, crawl invitations, and the pages-deserve-it rule.

8 min read

“Discovered – currently not indexed” and “Crawled – currently not indexed” are where new sites go to wait. This is the exact sequence we ran on a three-week-old tools site to move pages out of those buckets — including two bugs nobody talks about, because they are server-side and invisible in the content.

What the two statuses really mean

“Discovered” means Google knows the URL exists but has not crawled it yet — a budget and quality-queue problem. “Crawled” means the page was fetched but not added to the index — a judgement problem. The fixes are different: discovered needs crawl invitations and internal links; crawled needs the page to deserve more than a fetch.

Bug one: the sitemap that could not be read

Search Console reported “Couldn’t fetch” on a sitemap that loaded perfectly in a browser. The cause was invisible: the server answered HEAD requests without a content-length header, and Google’s fetcher validates with HEAD first. One line of server config, weeks of mystery. Lesson: test your sitemap with a HEAD request and a Googlebot user-agent, not just your browser.

Bug two: the stale failure state

After the fix, the old sitemap URL kept its failed status — Search Console caches fetch outcomes per URL and retries on its own schedule. The pragmatic escape: publish a byte-identical alias (sitemap-all.xml next to sitemap.xml) and submit the fresh URL, which starts with no history. Both stay served forever so nothing 404s.

Inviting the crawl

URL inspection → Request indexing for the money pages, ten a day, no more. IndexNow for instant Bing pickup. Internal links from already-indexed pages to stuck ones — a linked page is a page with a reason to be fetched. Fresh publishing on a visible cadence: a sitemap whose lastmod moves honestly every week tells the crawler the site is alive.

Making pages deserve the index

For “crawled – not indexed”, volume of text is not the fix; uniqueness is. Every page needs something a competitor cannot copy-paste: a tool that runs, original numbers, first-hand experience. Template pages with swapped nouns get folded into duplicates and wait forever. We rewrote the weakest ten tool pages with real use-cases and FAQs before touching anything else.

The waiting rule

New domains get a probation window; indexing speed roughly doubles between week three and week eight if the site keeps publishing. Resubmitting sitemaps daily or panicking at day four wastes the energy the content needs. Check weekly, publish weekly, measure monthly.

The weekly ten-minute checklist

One: Indexing report — count moved? Two: sitemap status green and last-read recent? Three: ten URL inspections on the pages that matter. Four: one new published page or one substantially improved page. Five: internal links from a strong page to the weakest indexed neighbour. Ten minutes, every week, beats any tool you can buy.

Related tools

Frequently asked questions

How long does not-indexed usually last?
On a new domain that keeps publishing, expect movement between weeks three and eight. If it stalls past that, look for server-side fetch problems before blaming content.
Does requesting indexing every day help?
No. Ten URL inspections a day is the practical ceiling; repeating the same URL daily adds nothing and wastes the routine.
Is a sitemap alias a legitimate fix?
Yes — serving a byte-identical copy under a new URL resets a stale failed-fetch record without hiding anything; both URLs stay public.

More tips

Tip

Google Business Profile SEO: The Checklist That Moves the Local Pack

Category, services, review answers, weekly posts and UTM-tagged links — the profile fields that actually move local rankings, in one pass.

Tip

How to Use Google Keyword Planner Without an Ads Budget

Ranges over numbers, indexed competition, bids as intent, clustering, and the monthly Search Console loop — the free-tier workflow that built this site’s plan.

Tip

Spreadsheet Work Without Excel: Nine Browser Jobs Done in Seconds

Dedupe, sort, trim, CSV-to-JSON, averages and date maths — the cleaning jobs that open a workbook today, closed in a browser tab instead.

Tip

AI Generators vs Browser Tools: Which Machine for Which Job

Predicted text or computed answers — where AI drafts win, where deterministic tools win, the privacy line, and the hybrid workflow that uses both correctly.

Tip

Instagram Engagement Rate Benchmarks 2026: What Is Actually Good

Engagement rate benchmarks by account size for 2026, the last-10 measuring rule, and the three levers that lift ER without buying anything.

Tip

Instagram Captions and Hashtags That Still Work in 2026

The 125-character hook rule, keyword placement Instagram actually indexes, three-to-five hashtags, and a fifteen-minute caption workflow.

Tip

YouTube SEO in 2026: Titles, Descriptions and the Retention Loop

The two signals that actually rank videos — clicks and retention — and the metadata workflow around them: 60-character titles, indexed descriptions, chapters that win SERP space.