Cloudflare corta el dilema: diga no al entrenamiento de IA y síguase viendo en Google, Apple y Bing

For years, site owners faced an awkward tradeoff: refuse to let your content train AI models, and you might also lose search visibility. The reason is concrete: several mixed-use crawlers do both jobs at once — building a search index for Google, Apple or Bing and collecting text for training.

On 15 September 2026, Cloudflare launched a control built to end that dilemma: Disallow AI Training. In plain terms for a small business: you can say “yes, keep me findable in search” and “no, do not use my site to train models” without switching the whole crawler off.

The SME dilemma (and why the numbers matter)

Cloudflare notes that almost nobody wants to vanish from search: under 1% of sites on its network choose to block Search bots. Training is different: about 17% already use some mechanism to stop content being used for AI training. In simple words: people want Google visits; they do not necessarily want to donate their copy to a model.

A robots.txt file alone is not enough: anyone can publish one, but it cannot reliably identify who is crawling or stop a crawler that ignores it. As a network, Cloudflare says it can publish the preference, identify the crawler, classify why it came, and block operators that ignore the rule — with transparency on Cloudflare Radar.

What “Disallow AI Training” and Accountable mean

The new setting is named after the Disallow preference it publishes via Bot Preference Sync, which writes those rules into the domain’s robots.txt. Mixed-use crawlers Cloudflare labels Accountable may keep crawling for search; other training traffic is blocked.

To earn Accountable status, an operator must meet or commit to: a training opt-out (robots.txt or similar); a path to opt out of AI summaries; URL-level visibility into how content was used; and assurance that refusing training does not hurt traditional search ranking. Apple, Google and Microsoft meet these criteria or have given timelines. Cloudflare also treats Amazon, Anthropic, Meta and OpenAI crawlers that separate search from training as Accountable: blocking the training bot does not break SEO.

Useful Bing detail: robots.txt no-training support is still being built (target early 2027). Today Microsoft offers NOARCHIVE and Webmaster Tools controls; Cloudflare notes Disallow AI Training does not yet automatically convey a no-training preference to Bing via robots.txt.

Warning: “Block” now also stops Googlebot

This is the easy mistake. Previously, Block and “Block on pages with ads” did not apply to mixed-use crawlers because cutting them could remove the site from search. As of 15 September, those modes do apply to mixed-use crawlers — including Applebot, Bingbot and Googlebot. If you only want to stop training and keep SEO, use Disallow AI Training, not Block.

The generic “Block AI Bots” switch is deprecated in favour of three independent controls: Search, Training and Agent. Managed Robots.txt migrates to Bot Preference Sync. The controls are available on all Cloudflare plans, under Security Settings per domain.

What to do today if your SME is on Cloudflare

1. Open the Cloudflare dashboard for your domain.
2. Go to the site’s Security Settings.
3. Under the Training control, choose Disallow AI Training.
4. Leave Search on Allow if you want to stay in Google, Apple and Bing.
5. Do not hit Block “just in case”: that also stops mixed-use search crawlers.

New ad-supported domains get a stricter recommended preset (Search allowed, Training on Disallow, agents blocked on ad pages). You can change it anytime.

Next up: AI summaries (and why they matter to revenue)

Cloudflare says the next focus is AI summaries in search. More than half of consumers already read them; those readers are over 40% more likely to end the search there — fewer clicks to your site. But visitors referred by AI search convert at roughly 3–5× traditional search traffic: fewer visits, often higher purchase intent. A publisher funded by ads may want volume; a retailer may prefer quality. Finer control over how much content enters a summary is the next piece.

Why this is brand-safe for your business

This is not a fight against AI and not an accusation against anyone: it is a product control. You decide whether your blog, catalog or landing page feeds a model, without giving up organic visits. For a Mexican SME that lives on search traffic, that separation — previously impossible with mixed-use crawlers — was the missing piece.

If you manage several domains, review each zone: Training, Search and Agent settings are per domain. And if someone on the team flipped “Block AI Bots” months ago, confirm the automatic migration to Disallow AI Training under Training and Allow under Search; Cloudflare documents migration tables so you are not surprised on change day.

Sources

• Cloudflare Blog — Have it both ways: stay discoverable while disallowing AI training (15 Sep 2026)
• Cloudflare Press — Helps End the Search-or-AI-Training Tradeoff
• Search Engine Journal — Cloudflare lets sites disallow AI training without blocking Googlebot

If your SME needs a clear, fast site ready for these crawler controls, see Presticorp for SMEs: design and hosting so Google can find you — while you decide what trains on your content.