Coaley Peak GEO Research Centre Engine guide
Meta AI How the assistant inside WhatsApp and Instagram finds its sources, and how to be one.
Meta AI is the assistant built into WhatsApp, Instagram, Facebook, Messenger and Threads, used by about a billion people a month at Meta's last count. It now runs on Muse Spark, the first model from Meta Superintelligence Labs, and searches the web with help from its own crawler. This guide covers its timeline, the four layers behind its answers, Meta's crawlers and how to opt out.
Independent guide: Coaley Peak is not associated with Meta Platforms, Inc. Trademarks
- Owner
- Meta Platforms, Inc.
- Launched
- September 2023 (US); UK October 2024; EU March 2025
- Home market
- Dozens of countries, across WhatsApp, Instagram, Facebook and Messenger
- Languages
- Many, including Hindi, French, German, Italian, Portuguese and Spanish
- Open weights
- No for Muse Spark; earlier versions ran on open Llama models
- Crawlers
- meta-webindexer, meta-externalagent, meta-externalfetcher and others
Overview
What Meta AI is, and who uses it
Meta announced Meta AI in September 2023, in beta in the US on WhatsApp, Messenger and Instagram, with real-time information through a search partnership with Bing. In April 2024 it was rebuilt on Llama 3, put into the apps' search bars, rolled out to more English-speaking countries, and began using Google's results as well as Bing's. It reached India in June 2024, the UK in October 2024 and the EU in March 2025, after regulatory delays over training on Europeans' data.
Mark Zuckerberg said in May 2025 that Meta AI had about a billion monthly users across Meta's apps, and a standalone Meta AI app, built with Llama 4, had launched that April. In April 2026 Meta rebuilt the assistant on Muse Spark, a hosted model from Meta Superintelligence Labs, and in July 2026 it said daily users had risen 60% since. Meta says WhatsApp is the leading surface where people use Meta AI.
Meta AI is distinctive for GEO in two ways. Its answers are read inside chats, mostly in WhatsApp, rather than on a results page. And Muse Spark increasingly weaves Reels, photos and posts from Meta's own platforms into its answers, credited to their creators, so a brand's public Instagram, Facebook and Threads content can be cited alongside the open web.
Meta also separates its crawlers by purpose. Meta-WebIndexer builds the index Meta AI uses to cite and link sources; meta-externalagent crawls for training; meta-externalfetcher fetches pages for users and may ignore robots.txt. A site can refuse training and still be cited.
Timeline
Meta AI, from launch to today
Meta announces Meta AI in beta in the US on WhatsApp, Messenger and Instagram, with real-time information through a Bing search partnership.
Meta AI is rebuilt on Llama 3, added to the apps' search bars, rolled out to 13 more countries, and begins using real-time results from Google as well as Bing.
Meta pauses plans to train on EU users' public posts after a request from the Irish Data Protection Commission; the pause also covers the UK.
Meta AI arrives in India, in English.
Brazil's data protection authority suspends Meta's use of personal data for AI training; the suspension is lifted with conditions on 30 August 2024.
Meta's crawler documentation lists meta-externalagent, for AI training and indexing, and meta-externalfetcher, for user-requested fetches.
Meta says it will begin training on UK adults' public Facebook and Instagram content, with an objection form, after engaging with the ICO.
Meta AI launches in the UK, Brazil, Bolivia, Guatemala, Paraguay and the Philippines.
Meta AI begins rolling out across 41 European countries, including the EU, and 21 overseas territories.
Meta announces it will train on EU adults' public content and Meta AI interactions, with an objection form.
Meta launches a standalone Meta AI app, built with Llama 4.
The Meta-WebIndexer crawler, for Meta AI search and citations, appears in Meta's crawler documentation.
Meta introduces Muse Spark, the first model from Meta Superintelligence Labs, to power a rebuilt Meta AI.
Meta launches Muse, a personal AI agent with its own browser, in the US.
Things to know
Things to know about Meta AI
Each Meta crawler has its own robots.txt token. Blocking meta-externalagent (training) does not block meta-webindexer (Meta AI search and citations) or facebookexternalhit (link previews).
Meta says allowing Meta-WebIndexer 'helps us cite and link to your content in Meta AI's responses'.
meta-externalfetcher, which fetches pages for users and helps AI complete tasks on websites, may bypass robots.txt.
Meta's crawlers can cache robots.txt for up to 24 hours, so blocks and unblocks are not instant.
Meta says it honours robots.txt rather than 'non-standard formats like NoAI tags', so noai meta tags are not a documented Meta control.
Meta AI's web search is on by default. It sends a generated query, not linked to the user's account, to 'select web search engines' that Meta no longer names.
Users can tap 'See details' under an answer to see the exact web query Meta AI sent and the sources it used.
Muse Spark answers increasingly include Reels, photos and posts from Meta's platforms, credited to their creators.
In May 2026 Meta removed its crawler IP guidance, a whois lookup of its network AS32934, from its documentation, leaving no published way to verify its crawlers by IP.
Meta has said WhatsApp is 'the leading surface where people engage with Meta AI', so most Meta AI answers are read inside chats.
The four layers
How Meta AI builds an answer
Every AI answer is made in four layers: what the model learned in pre-training, how it was tuned, what it retrieves live, and how it puts the answer together. Here is each one for Meta AI, and what it means for a brand that wants to be cited.
Layer 1
Pre-training: the base (foundation) model
Since April 2026 Meta AI has run on Muse Spark, the first model from Meta Superintelligence Labs: a hosted model rather than open weights, though Meta says it hopes to open-source future versions. Earlier versions ran on Meta's open Llama models. Muse Spark's cut-off is not published. Meta says its AI is trained on publicly available internet information, licensed data and information shared on Meta's products, such as public posts, captions and interactions with AI features; private messages are not used unless shared with the AI.
What it means for you: Meta trains on public content from its own platforms as well as the web, so what your brand posts publicly on Instagram, Facebook and Threads can shape what Meta's models learn about you, alongside what meta-externalagent crawls from your site.
Layer 2
Post-training: tuning and alignment
Meta describes reinforcement learning as one of three ways it scales Muse Spark, alongside pre-training and reasoning at answer time. It attributes refusals in high-risk areas to filtering of pre-training data, safety-focused post-training and system-level guardrails, assessed under its Advanced AI Scaling Framework. A detailed post-training recipe is not published.
What it means for you: The tuning details are thin, but the product direction is clear: Meta AI is built for quick answers inside chats. Content that settles a question in a few clear lines is what it is shaped to deliver.
Layer 3
Retrieval: live search
Meta AI generates a search query from the prompt and sends it to select web search engines, without linking it to the user's account. Meta named Bing at launch and added Google in 2024; it no longer names its providers. It also builds its own index through Meta-WebIndexer. Muse Spark draws on Meta's platforms too, surfacing public posts, Reels and community posts, and Shopping mode mixes Marketplace listings with options from across the web.
What it means for you: Two routes matter: the open web, through Meta-WebIndexer and the search engines behind Meta AI, and Meta's own platforms. A brand with an active, public presence on Instagram, Facebook and Threads gives Meta AI first-party sources to draw on.
Layer 4
The answer: what people read
Users can tap 'See details' below a reply to see the search query Meta AI sent and the sources it used. Meta says Muse Spark answers will increasingly weave in Reels, photos and posts, credited to creators. Developers using Muse Spark's web search get citations that tie spans of the answer to source URLs. Meta's separate Muse agent, launched in the US in September 2026, has its own browser.
What it means for you: Sources sit behind 'See details', so being named in the answer matters more than being linked. A brand that appears in creators' posts and Reels, as well as on the web, has more ways into the answer.
Crawlers
Meta AI's crawlers and user agents
The user-agent tokens to look for in your server logs and to name in robots.txt.
| User-agent token | What it does | robots.txt |
|---|---|---|
| meta-webindexer Documentation | Navigates the web to improve Meta AI's search results. Meta says allowing it helps Meta AI cite and link to your content. User agent: meta-webindexer/1.1, with a link to Meta's crawler documentation. | Follows robots.txt; Meta lists no exception. |
| meta-externalagent Documentation | Crawls for uses such as training foundation AI models, or improving products by indexing content directly. | Follows robots.txt; Meta's own example rule uses this token. |
| meta-externalfetcher Documentation | Fetches individual links at a user's request and supports agentic features, including helping AI navigate websites to complete tasks. | May bypass robots.txt, because it performs user-requested fetches. |
| facebookexternalhit Documentation | Fetches pages shared on Meta's apps to build link previews: title, description and thumbnail. | Generally follows robots.txt, but may bypass it for security or integrity checks. |
| meta-externalads Documentation | Crawls for advertising and other business products and services. | Follows robots.txt; no exception documented. |
Meta's current crawler page shows its user-agent links in relative form after a documentation move, and third-party logs also show browser-style strings that end with, for example, 'compatible; meta-externalagent/1.1'. Match on the token.
Crawl sources
Where Meta AI crawls from, and how to verify it
No longer published. Until May 2026, Meta's crawler documentation said its crawlers' addresses were those announced by its network, AS32934, retrievable with a whois query, and pointed to its peering page and geofeed. The current page still recommends allow-listing by IP address but lists none, and publishes no reverse DNS scheme.
In practice, verify Meta's crawlers against AS32934's announced prefixes, as Meta itself used to recommend, and treat Meta user agents from other networks as unverified. Meta's contact for crawler problems is webmasters@meta.com.
Opting out
How to opt out of Meta AI, or limit what it uses
Refuse training, stay citable
Disallow meta-externalagent to stop crawling for model training and direct indexing, and leave meta-webindexer allowed so Meta AI can still cite and link you. Changes can take up to 24 hours.
Official pageKnow which bots robots.txt cannot stop
meta-externalfetcher may bypass robots.txt for user-requested and agentic fetches, and facebookexternalhit may bypass it for security checks.
Official pageUse robots.txt, not NoAI tags
Meta says it honours robots.txt rather than non-standard formats such as NoAI meta tags.
Official pageRaise personal data in AI answers
Anyone, user or not, can raise a concern about personal information in a response from an AI at Meta. Meta says it does not offer access, correction or deletion of third-party data used for AI, and reviews requests under local law.
Official pageObject as a UK or EU account holder
Adults in the UK (from September 2024) and the EU (from April 2025) were sent notices linking to a form to object to their public content and AI interactions being used for training.
Official pageKeep content non-public
Meta trains on public posts and captions. Private messages are not used unless they are shared with the AI.
Official page
Opting out covers future crawling and training only: models already trained keep what they learned, and Meta offers no way to correct third-party data used for AI. Blocking meta-webindexer gives up citations in Meta AI, so most brands should allow it.
How we help
How we get brands cited in Meta AI
The Coaley Peak GEO Research Centre researches and implements GEO for SEO agencies and in-house teams, with foreign markets as our speciality.
Meta AI visibility audit
We ask Meta AI your customers' questions, in WhatsApp and the Meta AI app, across markets, and record whether you are named, which sources it shows and what it says.
Crawler policy by purpose
We allow meta-webindexer so Meta AI can cite you, set meta-externalagent as you choose, and check that your firewall is not blocking Meta's network by mistake.
Platform presence Meta AI draws on
Muse Spark weaves Reels and posts into its answers. We help you build a public presence on Instagram, Facebook and Threads that states your facts consistently.
Answers that fit a chat
Meta AI is read in WhatsApp. We write content that answers a question in a few clear lines, the shape Meta AI is built to deliver.
Reach in WhatsApp-first markets
Meta AI is strongest where WhatsApp dominates, from India to Brazil. We prioritise it where your customers live in WhatsApp.
More on our research, services and team at the Coaley Peak GEO Research Centre.
FAQ
Meta AI GEO: questions
Which Meta crawler should I allow to be cited in Meta AI?
Meta-WebIndexer. Meta says allowing it in robots.txt helps it cite and link to your content in Meta AI's responses. It is separate from meta-externalagent, the crawler for training.
How do I stop Meta training on my website?
Disallow meta-externalagent in robots.txt. This stops future crawling for training and direct indexing, but does not affect models already trained. Meta says it uses robots.txt rather than NoAI meta tags.
What model does Meta AI use?
Since April 2026, Muse Spark, the first model from Meta Superintelligence Labs. It is hosted rather than released as open weights. Earlier versions of Meta AI ran on Meta's open Llama models.
Does Meta AI show its sources?
Yes, behind 'See details' below a reply, which shows the web query Meta AI sent and the sources it used. Meta says answers will increasingly include credited Reels, photos and posts.
Where is Meta AI available?
In many countries across WhatsApp, Instagram, Facebook, Messenger, Threads, meta.ai and the Meta AI app, including the UK since October 2024 and the EU since March 2025. Meta publishes no single current country list.
Can I verify Meta's crawlers by IP address?
Not from current documentation. Until May 2026 Meta pointed to the IP ranges of its network, AS32934; its current crawler page lists none and publishes no reverse DNS scheme.
Want a free chat about Meta AI?
Tell us which market and brand you are working on. If we can help, we will set up a free, no-obligation chat with Stephen or Ethan.
More engine guides
Related engines
Independence and trademarks. Meta, Meta AI, Facebook, Instagram, Messenger, Threads, WhatsApp and Llama are trademarks of Meta Platforms, Inc. or its affiliates. Coaley Peak is independent: we are not associated with, endorsed by or a partner of Meta Platforms, Inc. This guide is our own research from public documentation and reputable reporting, checked in October 2026.
Reviewed by Stephen Grindley, research centre lead. Last checked .
Document reference: ISO_webpage_geo_engine_meta_ai_v1
Last modified: 7 October 2026
GEO Research Centre·Engine guides·Meta AI