Have you been creating content, without seeing the results you want in Google search or LLM results? There are a few reasons why that might be happening.
1: Google hasn't found your sitemap
Your sitemap makes it easier for Google and other search engine crawlers (including those used by OpenAI and Anthropic) to accurately crawl and index your website. Log into your Google Search Console and click on "sitemap" to see if the link to your sitemap is listed as submitted.
If you don't see a sitemap submitted, paste your sitemap URL in the submission box. Your sitemap can usually be found at yourwebsite.com/sitemap.xml.
2: You're blocking AI search engine crawlers in your robots.txt file
Your robots.txt file (which you can find at yourwebsite.com/robots.txt) is like a gatekeeper that permits and restricts certain web crawlers from accessing your website.
It's normal and good to have select pages of your site restricted from crawlers, such as your WordPress theme files or your admin login page. But if you restrict too much of your site, or fully block the crawlers used by various AI search tools, you won't show up in their results.
Use this free tool to scan your robots.txt file and see if you're blocking too many crawlers from too many pages:
As long as the results say "root" is allowed, you're good. That means it can access your homepage and other core pages, even if some URLs are restricted as explained above.
3: You're blocking Google's AI crawler in Google Search Console
Go to Search Console > Settings > Search Generative AI and check the two buttons shown below:
If you have "exclude" checked, you won't show up in things like the AI Overview.
4: Your content is too vague
This can be an issue with human writing, but I'm seeing it becoming more of a concern with AI-generated content. In order to rank in search results, you need to get specific, clear, and say something new. All search engines, including AI-powered ones, prioritize showing users content that is fresh, relevant, and full of E-E-A-T: experience, expertise, authority, and trust.
I have a free workbook you can grab to learn more about what E-E-A-T is and how to improve it on your own site:
Emily Gertenbach
5: It already exists somewhere else on the internet
You can't post the same content on two URLs/pages and have them both appear in search results. Search engines will pick the one that was published first or the one that is published on the "stronger" domain. For example, if you publish a blog post on LinkedIn and then copy it to your website, the LinkedIn result is likely going to show up in results...not your blog.
I always recommend publishing content on your site first, waiting at least two weeks, and then publishing it on a platform like LinkedIn if you intend to do so.
And if you get the opportunity to guest write a blog post on another website, don't copy the text over to your own. Instead, write an announcement about your guest blog or your experience and link to the feature.
6: You're targeting super difficult keywords
Really short, broad terms (like "computer mouse") are very hard to rank #1 for unless you are THE major player in an industry.And if that's not you, you're going to want to focus more on target terms that are less competitive but still very in line with your audience. Yes, the overall potential audience is smaller—but people will actually find your content.
7: You're trying to do too much at once
Just as targeting one keyword that's too difficult can keep you from appearing in search results, so can targeting too many keywords at once. I'm talking about trying to use one piece of content to rank for multiple unique keywords—even if they're low competition. One blog post can only do so much heavy lifting, even if it's a long one.
8: Your content is old or not authoritative
If you want to be the #1 resource in your industry, then you need to be on your content A game. That means keeping the date on your content current (and actually updating it every few months) as well as attributing it to an experienced/established writer or subject matter expert. This can be a recognized author (for example, I have a Google Knowledge Panel that associates me with my work, clients, and several topics) or a member of your team with deep industry experience.
9: Your content is really new
It can take weeks for new content to rank in search results, especially if you have a small website. There are definitely exceptions—I've gotten my clients' content to show up in AI search results within three hours of publication—but the norm for most people is that it'll take a couple weeks for search crawlers to find your new website page and add it to their database.
Want to learn more about how to plan, use, write, and get more traffic from your content? Check out my free masterclass / webinar library to get the rundown on important SEO topics you need to know more about:
Emily Gertenbach