I checked my own robots.txt file. And found a 400 error. I panicked, wondering if Google was blocked from crawling my entire site. Here's what Google's documentation say about it: "If you use a CMS, such as Wix or Blogger, you might not need to (or be able to) edit your robots.txt file directly." So the 400 error wasn't a problem. It's just how Wix works on a non-custom domain. Additionally, when Google encounters a 4xx error on a robots.txt file , it assumes that there ar

So you've built your website. You've written your pages. Maybe you've even submitted your sitemap to Google Search Console. And yet, your pages are sitting there, unindexed. Google knows they exist, but refuses to show them in search results. Sound familiar? It happened to me too. So let me explain what indexing actually is. What's the difference between crawling and indexing? If you've read my previous post on how Google crawls a website , you'll know that crawling is simply

Google uses a bot called Googlebot to discover and visit pages on your site. It follows links, reads your content, and decides what to store. You have a crawl budget which is a limited number of pages Google will crawl per visit. If Google can't crawl a page, it can't rank it. Crawling comes first, everything else follows. When I first started learning SEO, I kept hearing "make sure Google can crawl your site." I nodded along but had no idea what that actually meant. So h