Crawled, Currently Not Indexed: What It Means and What I Actually Did
Of every message Search Console has given me this year, "Crawled, currently not indexed" is the only one that stings. The others are administrative. This one is a verdict, and it took me a while to be willing to read it properly.
Here is the distinction that finally made it click. There are four reasons a page of mine was sitting outside Google's index. Three of them are about whether Google can get in. One of them is about whether Google, having got in and read the page, decided it was not worth including. That last one is a judgment about my work, and I had exactly one page carrying it.
This post is about that page, and about the difference between a site that is hard to enter and a page that is easy to dismiss.
What the label actually means
"Crawled, currently not indexed" means Google fetched the page, read it, understood it, and chose not to put it in the index. Not because it could not reach it. Because it decided the page did not add enough to what it already had.
The phrase "currently" is doing a lot of work. It is not permanent, it is not a penalty, and it is not a message from a person sitting in judgment of your prose. It is a system making an efficiency call with a fixed amount of storage and a fixed amount of patience, and it will revisit that call if the page earns it.
What earns it is the part nobody enjoys hearing. Unique value that exists nowhere else. A clear reason to exist. Evidence that a human wrote it and stands behind it. And the slow accumulation of signals that this domain is worth spending crawls on at all.
The page that got read and rejected
Mine was the Start Here page. Google's last visit was 28 October at 06:24, and I can see the record of it: crawled, read, not indexed.
Two details in that same panel matter more than the label. The first is that the referring page was listed as "None detected". Nothing on my site pointed at it and nothing outside my site pointed at it. Google found it in the sitemap and nowhere else, which means no human path to that page existed at all.
The second is that under Discovery, the sitemap line read "Temporary processing error". At the time I panicked about that and assumed my sitemap was broken. It was not. When I checked the Sitemaps report directly, all five submitted sitemaps read Success, and the main one had 62 discovered pages. So that line in the per-URL panel was an artefact, not a fault, and I had been about to spend a day fixing something that was never broken.
Read the report at the level it was written. The Sitemaps page tells you about sitemaps. A single URL panel tells you about one visit to one address, on one day, and it will happily show you a stale error without telling you it is stale.
Why that page in particular
A Start Here page is the easiest page on any site to get wrong, and mine was wrong in the most common way. It was written for someone who already cared, in a voice that assumed context a stranger does not have, and it pointed at nothing and was pointed at by nothing.
Google's crawler does not read a page and think "this is dull". It compares the page against everything it already holds and asks whether indexing this adds anything. A short page with no links in, no links out, no external references and a handful of general sentences about starting a business is competing against a million pages saying the same thing more thoroughly. It lost that comparison, and it deserved to.
The useful part is that this is fixable in a way that crawl errors are not. A redirect error resolves when the site resolves it. A quality call resolves when the page changes enough to be worth indexing.
The other fifty-five pages were a different problem entirely
Thirty-six pages were in "Discovered, currently not indexed", which is Google saying it knows the address exists and has not decided to spend a visit. Eighteen were redirect errors from visits that happened before I fixed the site, all between 14 September and 6 October. One was an alternative page with a proper canonical tag, which is correct behaviour and nothing to fix.
None of those are judgments about writing. They are about access and appetite. I have written about the redirect errors separately, because that was a mechanical failure I could find and repair. This one is not mechanical, and pretending otherwise is how people spend months submitting forms instead of improving the page.
The number that actually explains everything
When I finally looked at my crawl statistics rather than my feelings, the diagnosis was sitting there. Around 87 crawl requests over 90 days, roughly one a day, and every single one of them a refresh of a page Google already had. Zero discovery crawls. In three months, Google did not once come to my site looking for something new.
That is what a site with one inbound domain looks like from the inside. Not punished. Not blacklisted. Just not interesting enough to explore, so the crawler turns up occasionally, refreshes what it knows, and leaves.
Which means the fix for a page marked crawled and not indexed is not really about that page. It is about whether the site has earned the kind of attention where Google spends a crawl on something it has never seen. I can improve the page. I cannot improve it into an authority it does not have.
What I changed on the page, and what I stopped pretending
I made the page specific. It now names what the diary is, how many entries exist, when it started, what has failed so far, and what the numbers are. It links out to the entries that explain the failures in detail, which gives a crawler a path deeper into the site instead of a dead end, and gives a reader a reason to click.
I also stopped treating submission as a strategy. Up to that point I had requested indexing on around sixty URLs over three weeks, and the visible count went from twelve posts to eleven. That is not a slow improvement, that is noise, and I paused the daily batch rather than keep spending twenty-five minutes a day on something measurably inert.
What replaced it is the dull work. Internal links from pages Google already has to pages it does not. A rebuilt archive page that lists every entry, so nothing is orphaned. A sitemap that agrees with the canonical tag. And, for the first time, external links, because one inbound domain is not a distribution problem I can write my way out of.
The order I would do this in, from here
One. Confirm the page is genuinely reachable. A 200 response, one redirect hop at most from any variant, and present in the sitemap on the canonical host. Mine passed all three, which is how I knew the label was about content rather than access.
Two. Check whether anything links to the page. If nothing does, in or out, that is the first thing to fix, because a page nobody links to is a page nobody finds.
Three. Make the page worth indexing on its own terms. Specific numbers, a named author, a stated position, something a stranger could not get from the ten results above it.
Four. Stop submitting and start publishing things that earn links, then wait longer than feels comfortable. The "currently" in the label is a promise, not a deadline.
How to check your own crawl stats
Most people never open this report, and it is the one that explains a stalled site better than any dashboard summary. In Search Console, open Settings, then Crawl stats. You will see total requests, the breakdown by response, by purpose, and by file type, over the last 90 days.
Look at two lines. The first is requests per day. A site publishing two or three times a week that receives roughly one request a day is not being crawled, it is being checked on. The second is purpose. Refresh means Google came back for something it already knew about. Discovery means it came looking for something new. If refresh is everything and discovery is zero, no amount of publishing will change your index count, because nobody is reading the new pages at all.
When I checked mine, it read about 87 requests over 90 days, all refresh, no discovery. That single sentence explained more than three weeks of index requests ever did. I had been standing at a door asking to be let in, while the person inside was only ever checking whether the lights were still on.
The other thing worth reading there is the response breakdown. If a large share of requests are 301 or 404 rather than 200, the crawler is spending its small budget on detours. My redirect errors showed up here before I understood them anywhere else.
What makes a page worth indexing
After reading a lot of advice that amounted to "write better", I boiled it down to three things I can actually check on my own pages.
The first is a reason to exist that a stranger can state in one sentence. Not "a page about starting a business" but "the page where I list every failure and its date". If I cannot say it in a sentence, the page is not finished, no matter how long it is.
The second is evidence that cannot be copied. Mine is numbers: the counts, the dates, the rejection rounds, the crawl statistics. Anyone can write a general post about indexing. Almost nobody writes down that their validation sat at zero processed URLs for ten days while the interface said Started. That specific detail is the reason a page is worth storing.
The third is a path in and a path out. A page nothing links to has no path in. A page with no internal links has no path out, which makes it a dead end for both the crawler and the reader. A Start Here page with neither was never going to survive a comparison against pages that have both.
Length is not on that list, and I say that as someone whose shortest pages were the ones that got dismissed. Length is a side effect of having enough to say. When I padded, it showed.
What I would tell someone starting today
Do not build the site first and the audience second. That is the mistake this whole diary is built on. I wrote sixty-three posts into a site that had one inbound link, and then spent weeks being confused that a machine with a limited budget did not come and read them.
Decide where your first hundred readers will come from before you publish your tenth post. If the answer is "Google eventually", you are relying on the one channel that requires the thing you do not yet have, which is authority.
Then, separately, accept that a page can be perfectly reachable and still be rejected. When that happens, the label is telling you the truth and the fix is in the page, not in the plumbing. It took me an embarrassing amount of time to be willing to hear that.
And check the crawl stats before you check anything else. The number of visits tells you whether your problem is access, appetite or quality, and those three have completely different cures. I spent weeks applying the access cure to a quality problem and wondering why nothing moved.
What I am watching now
One number, taken the same way every Friday, by searching the site in Google and counting what comes back, rather than reading a report dated three weeks earlier. Today it is thirteen URLs, eleven of them posts, out of sixty-three published.
If that number moves, the approach is right and I keep going. If it does not move in a month, the answer will not be a better Start Here page. It will be that a young domain needs other people to point at it, and I will have to go and ask, which is the least comfortable thing on this entire list.
One page, crawled and dismissed, taught me more than the fifty-five that were simply out of reach. The out-of-reach ones need patience. The dismissed one needed a better page, and the honesty to admit it.
Continue reading the journey
Written by Justice Manyika — Carletonville, South Africa. This diary records my real journey, in my own words.
If this helped, follow the journey — drop your email in the box below and every new entry lands in your inbox. No spam, unsubscribe anytime.
Comments
Post a Comment