Back to blog
Digital Advertising

Meta’s Terms of Service Update Sparks Scraping Concerns — Meta Says It Applies Only to Ads Products and Respects robots.txt

Meta’s Terms of Service Update Sparks Scraping Concerns — Meta Says It Applies Only to Ads Products and Respects robots.txt

Meta’s updated Terms of Service, taking effect Oct. 30, drew criticism for language that appeared to let the company scrape businesses’ websites. After the backlash, Meta clarified that the terms apply only to its ads and business products and that it respects robots.txt preferences.

What the original language said

As first written, businesses that continue using Meta would authorize it to:

  • automatically crawl their associated domains,
  • use test credentials, and
  • periodically pull updates for training and research.

The clause was flagged by Stephen Beale of The Steampunk Explorer, and legal experts quoted by Nieman Lab warned it could be used to justify widespread scraping.

Meta’s clarification

Meta spokesperson Daniel Roberts told The Verge that the terms apply only to Meta’s ads and business products and that the company respects robots.txt preferences. Meta can still gather data from business websites for ad targeting.

What advertisers should check now

Review your robots.txt policy

Since Meta says it follows robots.txt, make your crawler rules explicit — especially the split between crawlers you allow for AI search visibility and those you block from training.

Audit connected domains

Confirm which domains are linked to your Meta business assets and make sure test or internal pages aren’t publicly exposed.

Takeaways

  1. Don’t skim platform terms. Ad platform terms can change how your site data is used; share the update with legal and engineering before Oct. 30.
  2. robots.txt is table stakes. See Four Technical SEO Fundamentals for AI Search: From robots.txt to Entity Consistency.
  3. Keep watching Meta’s data collection. For a related controversy, read What Personal Data Does Meta’s Muse AI Collect?.

Frequently Asked Questions

When do Meta’s new Terms of Service take effect?

October 30.

Can Meta crawl my business website?

Meta says the terms apply only to its ads and business products and that it respects robots.txt, though it can still collect data from business sites for ad targeting.

What should advertisers do?

Clarify robots.txt rules for each crawler and audit which domains and test pages are connected to Meta business assets.

Where does your own site stand?

To apply what you just read to your own site, start with a free audit of where things are now.

A strategist replies within 24 hours on business days.

Read next