Rabu, 12 Agustus 2026

Your site might be invisible to AI crawlers

Most sites never check. Here's how to.

Hi Iwan,

Google isn't the only bot visiting your site anymore. ChatGPT, Claude, and Perplexity all send their own bots to crawl your pages before they'll ever cite or recommend you. If those bots are getting blocked, and on many sites they are, you're invisible to AI search before anyone even judges your content.

If AI tools can't see your content, they can't recommend it, no matter how good it is. Most site owners have never checked whether that's already happening to them.

Here is how to check and fix it:

1. Audit your robots.txt for AI-specific blocks

Many CMS platforms and security plugins block AI crawlers by default, without ever telling you.

  • Check your robots.txt for explicit disallow rules on GPTBot, ClaudeBot, PerplexityBot, and Google-Extended.

  • If you don't recognize a rule blocking one of these, it likely came from a plugin update, not a deliberate choice.

  • Test your key pages directly against each bot's user agent using a tool like Screaming Frog's custom user agent feature.

2. Separate your AI-visibility strategy from your Google strategy

Blocking Google-Extended stops your content from training future Gemini models, but does not stop citations in AI Overviews, which pull from the live index.

  • Decide deliberately whether you want your content used for AI model training versus AI search citations. These are different toggles.

  • If you want citations but not training data usage, block Google-Extended specifically and leave Googlebot untouched.

  • Document this decision so it doesn't get silently reversed by a future plugin or developer.

3. Check your CDN and firewall settings, not just robots.txt

Robots.txt is a suggestion. Firewalls and bot-protection services enforce actual blocks that never show up in a standard SEO audit.

  • Review your CDN or WAF (Cloudflare, Sucuri, etc.) for bot-management rules that silently block AI user agents.

  • Check for JavaScript challenges or CAPTCHA walls that stop non-browser crawlers, including AI bots, from rendering your page.

  • Whitelist verified AI crawler IP ranges the same way you'd whitelist Googlebot's.

4. Recheck after every major platform update

A single WordPress security plugin update can silently add new bot blocks overnight.

  • Set a recurring quarterly check of your robots.txt and firewall bot rules.

  • Compare your current file against a saved baseline to catch unauthorized changes fast.

  • Treat AI crawler access like a core technical SEO metric, not a one-time setup task.

Click here to get a free audit of your site's AI crawler access.

To your continued success,

Matt Diggity
30 N Gould St, Ste #4000, Sheridan WY 82801 USA
Don't want to get emails like this? Unsubscribe or Pause Emails
Diposting oleh di Tidak ada komentar:

Senin, 10 Agustus 2026

Setelan privasi baru untuk layanan Penelusuran dan Google Play

Diposting oleh di Tidak ada komentar:

Selasa, 28 Juli 2026

How to rewire an LLM, and other SEO goodies…

Hi Iwan,

New research shows a few hundred mentions can permanently bake your brand into an LLM's training data, changing what it recommends for good. Meanwhile, Google's June Spam Update wiped out entire sites built on AI-generated content, with a new spam policy provision that specifically targets LLM answer manipulation.

This month's Diggity Marketing News Roundup breaks it all down, plus:

  • The exact process for training AI to recognize and recommend your brand

  • Only 25.6% of ChatGPT's cited sources overlap between its reasoning modes

  • An affiliate site that grew almost 200% after Google crushed its competitors

...and plenty more worth your time this month.

Read the full roundup here: https://diggitymarketing.com/news-roundup-july-2026/

Don't miss out on this one.

To your continued success,

Matt Diggity
30 N Gould St, Ste #4000, Sheridan WY 82801 USA
Don't want to get emails like this? Unsubscribe or Pause Emails
Diposting oleh di Tidak ada komentar:

I started a new channel...

It's not about what you think...

Hey Iwan,

Just launched a new channel about personal development, performance, and the inner work.

The things I got into after I burned out hard a few years back and had to figure out how to put myself back together.

First video is on a book that got me through that stretch. From Strength to Strength by Arthur Brooks. Honestly one of the most impactful things I've ever read, and I wish I'd found it 10 years earlier.

Here it is<<

If it resonates, hit like on the video. That's how I'll know whether to keep going with this.

To your continued success,

Matt Diggity
30 N Gould St, Ste #4000, Sheridan WY 82801 USA
Don't want to get emails like this? Unsubscribe or Pause Emails
Diposting oleh di Tidak ada komentar:

Rabu, 22 Juli 2026

The link building mistake tanking your rankings

More links isn't the goal. This is.

Hi Iwan,

Most sites I audit have a backlink profile that looks impressive in volume and does almost nothing for rankings. The problem isn't the number of links. It's relevance.

Here's how to fix your link building approach:

1. Match link sources to your actual topic, not your industry

Google evaluates topical relevance at the page level, not just the domain level.

  • Target sites and articles that cover your exact subtopic, not just your broad industry.

  • A relevant link from a smaller site often outperforms an irrelevant link from a bigger one.

  • Check the linking page's content, not just the domain's overall authority.

2. Prioritize contextual links over directory or profile links

A link buried in a resource list carries far less weight than one placed inside relevant content.

  • Aim for links embedded within a paragraph of related content, with natural anchor text around them.

  • Deprioritize sitewide footer links, badge links, and generic directory listings.

  • Ask yourself if a real reader would actually click the link in context. If not, Google discounts it too.

3. Build links to pages that need authority, not just your homepage

Sending every link to your homepage wastes the ranking power those links could pass to money pages.

  • Identify the specific pages that need a rankings push: commercial pages, cornerstone content, or pages just outside page 1.

  • Pitch content and outreach angles that naturally point to those specific pages.

  • Track which pages are receiving links using Ahrefs Site Explorer, filtered by target URL.

4. Diversify anchor text naturally

A backlink profile with the same exact-match anchor across dozens of links looks manipulated to Google's algorithms.

  • Mix branded anchors, naked URLs, and partial-match phrases across your link profile.

  • Let anchor text vary based on how the linking author would naturally reference your page.

  • Avoid requesting specific anchor text in outreach pitches. Let the linking site choose their own phrasing where possible.

Ten relevant, contextual links will move your rankings further than a hundred generic ones. Stop chasing volume and start chasing relevance.

Click here to get a free backlink audit of your site.

To your continued success,

Matt Diggity
30 N Gould St, Ste #4000, Sheridan WY 82801 USA
Don't want to get emails like this? Unsubscribe or Pause Emails
Diposting oleh di Tidak ada komentar:

Senin, 20 Juli 2026

How a small site out-ranked big lenders...

In a YMYL niche where authority decides who ranks

Hi Iwan,

If you've ever managed a site up against competitors with 10x the budget and domain authority, here's what actually closes that gap.

We took a client in the short-term loans niche, dominated by massive financial institutions, and grew their organic sessions by 73% YoY while increasing their AI Overview keyword visibility from zero to nearly 90 keywords.

Here's the playbook.

1. Clean up content before adding more

Thin and outdated pages drag down the credibility of your entire site in Google's eyes.

  • Audit every page and consolidate or remove thin, low-performing content

  • Set up 301 redirects on removed pages to preserve link equity

  • Refresh your highest-value articles with current data to hold search relevance

2. Fix the technical foundation first

If search engines and LLMs can't crawl your content efficiently, none of your other work matters.

  • Rewrite meta titles and headers on core landing pages to match search intent

  • Repair broken internal links and remove duplicate subpages

  • Implement schema markup so search engines can surface your content clearly

3. Build authority the slow, steady way

YMYL niches punish sites that try to fake authority with a burst of low-quality links.

  • Run consistent monthly outreach instead of one-off link pushes

  • Target topically relevant sites in finance and adjacent sectors

  • Prioritize link quality over volume in a space where trust standards are strict

4. Connect your blog to your money pages

Informational traffic means nothing if it never reaches your highest-intent landing pages.

  • Build a clear internal linking structure from blog content to state-specific pages

  • Reorganize your blog into distinct categories so related content sits together

  • Create content around informational gaps competitors haven't covered yet

The results:

  • Organic sessions: +73.1% YoY

  • Engaged sessions: +66.05% YoY

  • AI Overview keyword visibility: 0 to 87 keywords

If you're competing against bigger players in a trust-sensitive niche, head over to The Search Initiative, and we'll show you exactly where to start.

To your continued success,

Matt Diggity
30 N Gould St, Ste #4000, Sheridan WY 82801 USA
Don't want to get emails like this? Unsubscribe or Pause Emails
Diposting oleh di Tidak ada komentar:
Langganan: Postingan (Atom)