How to Get Your Business Recommended by Claude: Crawlers and Sources
To get your business recommended by Claude, allow Anthropic's search and user-fetch agents in robots.txt. Publish pages that answer a specific question with linked sources and a visible date, and describe the business the same way everywhere Claude might read about it. Then ask Claude the question with web search on and read the citations it returns.
Anthropic publishes less about Claude's search than Google does about its results. It does document its crawlers, its web search feature and the citations its API returns, and this post is built on those pages plus the research into what generative engines cite. When we read the first page for this search, every result was an agency or tool page and none was Anthropic's. Where Anthropic publishes nothing, the post says so.
How does Claude find businesses to recommend?
Claude recommends a business after searching the web for the question, reading the pages its search returns and citing the ones it used. Anthropic's web search documentation says Claude determines when to search based on the prompt, the API runs the searches, and the turn ends with a response that carries cited sources.
The same documentation lists what triggers a search: information that is current or changing, and information about specific organizations, people or products that might have changed. "Which time-tracking app should a freelance designer use" is that kind of question, so Claude searches rather than answering from memory. Anthropic adds that simple factual questions typically take one to three searches, and comparative research can take ten or more.
Web search reached Claude's users in stages. Anthropic's announcement says it opened as a preview for paid users in the United States on March 20, 2025. By May 27, 2025 it was available globally on all Claude plans, with direct citations so a reader can check the sources. As of October 2026, those citations are the window into why Claude said what it said. The way to get found by Claude starts with being readable by it.
Which Claude crawlers matter, and what does each one do?
Anthropic runs three agents, and two of them decide whether Claude can read your site when a person asks. ClaudeBot collects content that may be used for training. Claude-SearchBot indexes pages for search, and Claude-User fetches a page when a user's question calls for it. Each has its own line in robots.txt.
| Agent | What Anthropic says it does | What blocking it does, per Anthropic |
|---|---|---|
| ClaudeBot | Collects web content that could contribute to training | Signals that future materials should be excluded from training datasets |
| Claude-SearchBot | Navigates the web to improve search result quality | Prevents indexing for search, which may reduce visibility in user search results |
| Claude-User | Accesses websites when individuals ask Claude questions | Prevents retrieving content for a user query, which may reduce visibility for user-directed web search |
Anthropic's help article on its crawlers says Claude-SearchBot "analyzes online content specifically to enhance the relevance and accuracy of search responses," and that all of its bots honor the industry standard directives in robots.txt. For a site that wants to be found but not trained on, the file looks like this:
User-agent: Claude-SearchBot
Allow: /
User-agent: Claude-User
Allow: /
User-agent: ClaudeBot
Disallow: /
Our two-minute robots.txt check covers the other companies' agents.
Why do well-sourced pages do better with Claude?
Claude cites a passage, not a page, and the passage has to carry the claim and its evidence on its own. Anthropic's API documentation says citations are always enabled for web search, and that each citation carries up to 150 characters of the cited text.
If the claim is in one paragraph and its figure sits three paragraphs later, the cited passage has one without the other. Put the number, its source and its date in the same sentence, and open each section with the sentences you would want quoted.
The research agrees. Aggarwal and colleagues at Princeton University and Georgia Tech found in 2023 that citations, quotations and statistics raised a page's visibility in generated answers by 30 to 40 percent. Princeton's record of the paper, presented at KDD 2024, puts the headline gain at up to 40 percent.
The same paper found keyword stuffing performed about 10 percent worse than the unchanged page when tested on Perplexity. It tested its own engine and Perplexity rather than Claude, so treat it as the best evidence available, not a measurement of Claude. Our guide to writing a blog post that gets quoted by AI has the full page shape.
What should your pages say about the business?
A page Claude can recommend states, in one sentence, what the business is, what it does and where, and uses the same name every time. An assistant connects a mention on a review site with a page on your domain only when the details line up.
Take Clockbird, a fictional time-tracking app used here as an example. "Clockbird is a time-tracking app for freelance designers that rounds billable hours to each client's rules" gives an assistant something to repeat. "We help creative professionals take back their time" does not. Add the date the facts last changed, because Anthropic's search results carry a page_age field recording when the page was last updated.
Keep the same description on your Business Profile, your directory listings and your social profiles. Every mismatch is a reason for an assistant to hedge rather than recommend. For a business page, prefer the specific:
- The service list, in the words customers use.
- The area served.
- The pricing approach, if you publish one.
- A named person behind the page, with their role.
Where else does Claude look before it recommends?
When the question is "which one should I use," Claude's citations are usually comparison articles, review sites and community threads rather than the vendors' own pages. That is an observation anyone can repeat in a minute, not a statistic. Anthropic publishes no breakdown of the sources Claude cites.
A recommendation is a judgment, and an assistant borrows judgments from pages that already made one. Our post on getting recommended by ChatGPT explains the difference between being cited and being recommended; the same split applies to Claude.
List the business where your customers compare options, in your industry's directories and review sites. Answer real questions in the communities those customers use, under your own name, with the detail only you have. Describe the business identically in each place.
PARCHMNT is a writing tool that researches what ranks for a search, writes the page, checks every claim and tracks where it lands in Google and AI answers. Its Get mentioned plan lists, for one business, the directories, communities and review sites worth being present in, with one specific action for Claude among the assistants.
How do you check whether Claude recommends you?
Ask Claude the questions your customers ask, in two or three wordings, with web search on, and read the citations under the answer. Note separately whether your page is cited, whether your business is named, and whether it is recommended rather than mentioned in passing.
Repeat monthly with the same wordings, because answers change as the web and the model change. In our experience building the visibility check, the set of Claude citations shifts between wordings of the same question. A single ask tells you less than three, which is why we record consistency across phrasings rather than one answer.
If a competitor is recommended and you do not appear in Claude answers, open the cited pages. Usually the competitor is on a comparison page you are absent from, or its page answers in the first paragraph while yours explains, or its facts are dated and sourced and yours are not. Fix that, then check again next month.
Key takeaways
- Claude-SearchBot and Claude-User decide whether Claude can read your site for a person's question; ClaudeBot is the training crawler, and Anthropic describes each one separately.
- Claude searches when a question concerns specific organizations, products or changing facts, and its citations carry up to 150 characters, so each passage must hold its claim and its evidence.
- Citations, quotations and statistics lifted visibility 30 to 40 percent in the research that defined the field; keyword stuffing made pages worse.
- Recommendations are borrowed from comparison pages, reviews and threads; be present on them under one consistent description.
- Ask Claude monthly in the same wordings and read the citations; they show exactly what beat you.
Frequently asked questions
Does blocking ClaudeBot stop Claude from recommending you?
Not by itself, according to Anthropic's documentation. Anthropic describes ClaudeBot as the agent that collects content that may contribute to training. Claude-SearchBot and Claude-User have their own entries with their own consequences for search visibility. A site that blocks ClaudeBot and allows the other two has declined training while staying readable when a person asks Claude a question.
Does Claude search with Google or Bing?
Nobody publishes this. Anthropic's web search documentation describes what Claude does with results and how citations are returned, but it does not name a search index or provider, and this post will not guess. Assume that what ranks in ordinary search is in the running, and that a page no search engine has indexed is not.
Does Claude read structured data?
Anthropic's crawler and web search pages do not mention schema markup, so there is no evidence it is required. Structured data still helps Google and makes your author, date and business explicit on the page. Keep it, and do not expect it to move Claude on its own.
How long after a robots.txt change does Claude notice?
Anthropic publishes no figure. OpenAI says about 24 hours and Perplexity says up to 24 hours. Make the change, note the date, and watch your server logs over the following weeks for requests from Claude-SearchBot and Claude-User. Their presence in the logs is the only confirmation Anthropic's pages offer.
How to get your business recommended by Claude, in four steps
Allow the two agents that serve a person's question, write pages whose passages carry their own evidence, get mentioned where comparisons are made, and ask Claude monthly. If you would rather have the research, the writing, the checks and the monthly asking done for you, try one article free.
Sources
This post was written and checked with the same rules PARCHMNT applies to every page it writes for a customer: researched against what ranks today, answer first, every figure with a source.
Try 1 article free