Practical guides on websites, online stores, and getting found on Google — written for business owners, not developers.
Tell us about your business today — get a free quote within 24 hours. No pressure, no confusing tech talk.
Tell us about your business today — get a free quote within 24 hours. No pressure, no confusing tech talk.
Google finds, renders, stores and ranks pages in five stages. Learn where pages get stuck, how to check each stage in Search Console, and what AI Mode changes.
A client asked us why their new service page wasn't showing up on Google. It had been live for three weeks. The content was solid. The page was fast. Google just didn't know it existed.
This happens more often than most site owners think. The problem is rarely the content. Something breaks inside Google's pipeline, and nobody knows where to look.
Here's how that pipeline works. Google finds a URL, crawls it, renders it, decides whether to index it, and ranks it when someone searches.
A page can fail at any of those five steps. By the end of this guide, you'll know exactly where yours is stuck and how to prove it in Search Console.
Google officially describes Search in three stages. Crawling, indexing and serving results.
We split it into five. Each stage can fail on its own, and each failure needs a different fix.
Stage | What Google does | Where it fails |
|---|---|---|
Discovery | Learns a URL exists | No links, no sitemap |
Crawling | Googlebot fetches the page | robots.txt blocks, server errors, login walls |
Rendering | Runs JavaScript and sees the final page | Content that only loads through JavaScript |
Indexing | Analyzes the page and stores it | Thin or duplicate content, noindex tags |
Ranking | Orders indexed pages for a search | Weak relevance, low quality |
Google doesn't guarantee that any page will pass all five. Every stage is a filter.
There's no central list of every page on the web. Google has to find pages on its own. It does that three ways.
It revisits pages it already knows.
It follows links from known pages to new ones. A category page linking to a new blog post is the classic example.
It reads your XML sitemap.
Links do most of the work. A page with no internal links pointing to it is an orphan. Orphans get found late or not at all.
Once Google knows a URL, Googlebot may fetch it. That "may" matters. Google decides which sites to crawl, how often, and how many pages to take from each one.
Googlebot mostly crawls as a smartphone. Google uses the mobile version of your page for indexing and ranking. If your mobile layout hides content, Google won't see it.
Googlebot also watches how your server responds. Slow responses and 500 errors tell it to back off.
That's why hosting matters for SEO. Overloaded cheap hosting doesn't just frustrate visitors. It gets your site crawled less. [HOSTINGER AFFILIATE LINK]
robots.txt rules that block the page
Server errors, DNS or network problems
Pages behind a login
Orphan pages with no links pointing to them
Myth: You can pay Google to crawl or rank your site faster. You can't. Google states it doesn't take payment for either. Ads buy ad space, not organic results.
Fetching a page and seeing a page are two different things.
Googlebot first downloads the raw HTML. Then Google renders it with a recent version of Chrome, the same way your browser does, and runs any JavaScript it finds.
On a simple WordPress site, the two are almost the same. On a JavaScript-heavy site, they can be completely different. If your main content, internal links or canonical tag only appear after JavaScript runs, Google may store an empty or half-built page.
You can check this in two minutes. Open Search Console, run URL Inspection on the page, and click "Test live URL." Look at the screenshot and the rendered HTML. If it doesn't match what your visitors see, you've found your problem.
This is also where hacked sites hide. Some malware shows spam only to Googlebot and a clean page to everyone else. The owner sees nothing wrong while Google indexes casino or pharma pages. We covered this in our Japanese keyword hack guide.
After crawling and rendering, Google tries to understand what the page is about. It reads the text, the title, headings, images, alt text and structured data. It also notes the page's language and the country it serves.
Then it decides whether the page is worth storing. Google says plainly that not every page it processes will be indexed.
Most sites have more duplicates than their owners realize.
http and https versions of the same page
URLs with and without a trailing slash
Product variants like ?color=red and ?color=blue
Google groups similar pages into a cluster and picks one as the canonical. Only the canonical usually shows in search results. Your rel="canonical" tag is a hint. Google can ignore it and pick a different URL.
This is the status that frustrates site owners most. Google visited the page and chose not to keep it. The usual reasons are:
The content is thin or adds nothing new
The page is a near duplicate of another page
A noindex tag is blocking it
The canonical tag points to a different URL
Submitting a sitemap won't fix any of these. A sitemap tells Google a page exists. It doesn't make Google want to keep it.
When someone searches, Google pulls matching pages from its index. Then multiple ranking systems sort them. Google ranks pages, not whole websites, and the order changes with the searcher's location, language and device.
Meaning. Google works out what the searcher actually wants, not just the words they typed.
Relevance. Does the page answer that intent?
Quality. Is the content helpful, accurate and trustworthy?
Links. Google's ranking systems guide still lists link analysis and PageRank.
Page experience. Speed, mobile usability and intrusive popups all count.
For years Google avoided saying whether clicks affect rankings. That changed during the US antitrust trial. Pandu Nayak, Google's VP of Search, testified under oath that a system called NavBoost uses aggregated click data to adjust rankings. He described it as one of Google's most important signals.
The lesson is simple. A page that ranks but disappoints searchers tends to slide. Give people what they came for.
Forget the "200 ranking factors" line. It was never an official list.
AI search runs on the same pipeline. Google says there are no additional requirements to appear in AI Overviews or AI Mode. A page needs to be indexed and eligible to show a snippet. That's it.
The difference is how answers get built. Both features can use "query fan-out." Google splits one question into several related searches, pulls pages for each, and writes one answer with links.
That creates a real opportunity. A page that clearly answers one specific sub-question can get cited even if it doesn't rank top ten for the main search. But crawling and indexing still come first. If Google can't store your page, AI can't cite it.
AI Overviews and AI Mode traffic shows up inside your normal Search Console Performance report.
Every stage above leaves a trace in Search Console. Match your symptom to the stage, then fix that stage only.
Symptom | Stage | Where to check | First fix |
|---|---|---|---|
"URL is unknown to Google" | Discovery | URL Inspection | Add internal links, submit sitemap |
Blocked by robots.txt or server errors | Crawling | Page indexing, Crawl stats | Fix robots.txt, fix hosting |
Screenshot shows a blank or partial page | Rendering | URL Inspection live test | Serve key content in the HTML |
"Crawled, currently not indexed" | Indexing | Page indexing | Improve or merge the page |
"Duplicate, Google chose different canonical" | Indexing | URL Inspection | Consolidate duplicates, fix canonicals |
Indexed but no impressions | Ranking | Performance | Rework intent, content and links |
Pages you never made, spam or Japanese titles | Security | Page indexing, | Treat it as a hack, not an SEO problem |
That last row matters. If Google is indexing pages you never created, no SEO work will help. Start with our guide on how to investigate a hacked WordPress site, or let our team handle the malware removal.
Found problems in more than one row? That usually points to a site-wide technical issue. Our SEO audit checks all five stages and tells you what to fix first.
There's no fixed time. Google says crawling can take anywhere from several days to several months, depending on how often its systems think a page needs refreshing. Strong internal links and a sitemap help Google find it sooner.
Crawling means Googlebot fetched your page. Indexing means Google analyzed it and decided to store it. A page can be crawled and still not indexed.
Usually because Google sees it as thin, duplicate, or not better than pages it already has. Check for noindex tags and canonical tags first, then improve the content or merge it with a stronger page.
Yes, in aggregate. Google's VP of Search testified during the US antitrust trial that NavBoost uses click data and is one of Google's most important ranking signals.
SEO & Security Expert, AppDeel
Azizul Hakim is an SEO and website security specialist at AppDeel, the agency he founded and where he still handles client work himself. For more than ten years he has helped business owners and eCommerce brands get found in search and keep their websites secure.
Read more posts by Md Azizul Hakim →
ChatGPT ads cost $3 to $5 a click. Google averages $5.42. We compare cost per lead, buyer intent and early conversion data to show where each one wins.
Host warning you about spam? Here is how to confirm it, find the malicious mailer script on your server, and remove it without deleting the wrong file.
Most WordPress vulnerabilities live in plugins. Learn how to vet, update, monitor and remove them, plus WordPress.org's new automated review.
A WordPress activity log only works if it records the right events, keeps them long enough, and survives an attacker.
Tell us about your business today — get a free quote within 24 hours. No pressure, no confusing tech talk.
