Free tool: runs in your browser
AI crawler opt-out generatorrobots.txt and TDM rules for artists and writers
TimeBinder adds a layer of protection against AI and human fraud and disputes. Ask AI crawlers not to use your site, and see what that does and does not do.
Nothing you type is sent anywhere. The rules are made in your browser, and they only state a preference.
Make your opt-out files
Pick the crawlers, choose the paths and copy the result. The generator uses the crawler names their makers publish.
Everything is made in your browser. Nothing you type here is sent anywhere.
1. Start from a preset
2. Adjust the crawler list
These files ask crawlers not to use your site. They do not enforce anything: compliance is voluntary, crawlers that ignore the rules are not stopped, and nothing here removes content already collected. Check each company’s current documentation, because crawler names and policies change.
How do you opt your site out of AI training? The short answer
The documented way is a robots.txt file that names each AI crawler and disallows your pages. Companies such as OpenAI, Anthropic, Google, Apple and Meta say their training crawlers read it. You can add a W3C community-group reservation, the TDM Reservation Protocol, to state the same preference in a form built for the EU text and data mining opt-out.
These are requests. They do not stop a crawler that ignores them, and they do not undo what was collected before. Treat them as one step, not a lock.
How it works
- Choose a preset. The default asks training crawlers only. The wider presets add AI search crawlers and user-triggered fetchers, which can cost you visitors or do nothing.
- Set the paths. Use
/for the whole site, or list folders such as/portfolio/. - Paste your current robots.txt (optional). The new rules are added below it and any crawler you already mention is flagged.
- Copy or download the files and publish them: robots.txt at the root of your site, and optionally the meta tags, header or
tdmrep.json.
What an opt-out does and does not do
Opting out is about your site’s future crawling. It is separate from proving when you made a work.
| Part | What it does | What it does not do |
|---|---|---|
| robots.txt rules for training crawlers | Ask named crawlers not to collect your pages for model training. Documented by their makers. | Stop a crawler that ignores robots.txt, or remove content already collected. |
| robots.txt rules for AI search crawlers | Ask AI search tools not to index your pages. | Keep your content out of answers built from other sources. It can also cost you traffic. |
| User-triggered fetchers | Ask tools that load a page on a person’s request to stay away. | Reliably work. Several makers say these fetchers may ignore robots.txt. |
| TDM Reservation Protocol (tdmrep.json, header, meta tag) | States that text and data mining rights are reserved, in machine-readable form. | Bind any company to read it. It is a community-group specification, not a standard. |
| noai meta tag | Signals a preference to the few tools that look for it. | Do much. It is informal and not documented by the major AI companies. |
Where the crawler names come from
Names and behaviour were taken from each company’s own documentation on 12 November 2026. Crawlers and policies change, so check the source before you rely on a name:
- OpenAI: GPTBot, OAI-SearchBot and ChatGPT-User
- Anthropic: ClaudeBot, Claude-SearchBot and Claude-User
- Google: Google-Extended and the other common crawlers
- Apple: Applebot and Applebot-Extended
- Meta: meta-externalagent and meta-externalfetcher
- Amazon: Amazonbot, Amzn-SearchBot and Amzn-User
- Perplexity: PerplexityBot and Perplexity-User
- Common Crawl: CCBot
- W3C Community Group: TDM Reservation Protocol (TDMRep)
This is general information, not legal advice. How opt-outs work in law differs by country; a lawyer can advise on your own situation.
An opt-out is not a dated record of your work
Asking crawlers to stay away says nothing about when you made something. If you may one day need to show that a file existed on a certain date, record the exact file separately:
- Freeze the finished file or the draft you want to be able to show.
- Make a TimeBind of it. The fingerprint is made in your browser and the file stays on your device.
- Keep the file and the Proof of Time Certificate together. Read how artists do it in how artists show when they made something.
A TimeBind shows the exact file existed no later than the recorded time and still matches. It does not show who made it, who owns it, or whether anyone trained on it.
First Pooled TimeBind free, no card, then from US$5. An ASAP TimeBind is US$15.
What a TimeBind shows, and what it does not. It shows that an exact file existed no later than a recorded time and that a copy shown later still matches its fingerprint. It does not show who made the file, who owns it, that its contents are true, or whether AI was used to make or edit it. This is general information, not legal advice.
AI crawler opt-out generator: FAQ
How do I ask AI companies not to train on my website?
Add rules to the robots.txt file at the root of your site that name each AI training crawler, such as GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot and meta-externalagent, and disallow your paths. This tool writes those lines for you. You can also add the TDM Reservation Protocol tags and header, which are aimed at the EU text and data mining opt-out.
Does robots.txt actually keep AI crawlers away?
It asks them to stay away. The crawlers listed here are documented by their makers as respecting robots.txt, but compliance is voluntary. A crawler that ignores the file is not stopped, and a robots.txt rule does not remove anything that was collected before you added it.
Will this affect my Google Search ranking?
Not through the training crawlers in the default preset. Google says Google-Extended does not affect inclusion in Google Search or ranking. The tool does not touch Googlebot. Blocking AI search crawlers such as OAI-SearchBot or PerplexityBot can reduce the traffic you get from those tools, which is why they are a separate option.
What is the difference between training crawlers, AI search crawlers and user-triggered fetchers?
Training crawlers collect content that may be used to train models. AI search crawlers find pages to cite in AI answers. User-triggered fetchers load a page when a person asks an AI tool about it. Several makers say their user-triggered fetchers may ignore robots.txt because a person asked for the page.
I am an artist. Does this protect my images?
It asks the listed crawlers not to use images on your own website, and the reservation tags state your preference in machine-readable form. It does nothing for copies of your images on other sites or platforms, for crawlers that do not follow robots.txt, or for images already collected. Platform settings and your contracts matter there. It also does not change who owns your work.
What is the TDM Reservation Protocol?
It is a specification from a W3C Community Group, not a W3C standard. It lets a site declare that text and data mining rights are reserved, using a file at /.well-known/tdmrep.json, an HTTP header or an HTML meta tag. It was designed to express the opt-out in Article 4 of the EU Copyright Directive (2019/790). Whether a particular AI company reads it is up to that company.
Does the noai meta tag work?
It is an informal convention, not a standard. Meta, for one, says on its crawler page that it prefers industry-standard practices such as robots.txt over non-standard formats like NoAI tags. The tool leaves it off unless you tick it. Do not rely on it.
What about an ai.txt file?
This tool does not make one. An ai.txt file is a proposed convention, and none of the crawler documents we checked for this page mention reading it. robots.txt is the method those companies document.
Where do I put the file, and how do I check it?
Put robots.txt at the root of each website address, for example https://example.com/robots.txt. Subdomains need their own. Then open that address in your browser to confirm it shows your rules. Crawlers can cache the file, so allow up to a day. Many website builders have their own settings for this, so check your builder’s help first.
Does this give me a dated record of my work?
No. An opt-out states a preference about future crawling. A TimeBind is different: it records the SHA-256 fingerprint of an exact file on the Bitcoin blockchain, which shows the file existed no later than a recorded time and still matches. Your first Pooled TimeBind is free with no card, then from US$5.
Related guides and tools
- Developer timestamping hubSource code, releases and game assets
- The shared robots.txt AI-training opt-outWhat the new __usage_ai-training line does, and when companies will honour it
- Glaze, Nightshade and AI opt-outsWhat they do for artists, and what a dated record adds
- How artists show when they made somethingA simple routine
- Timestamping for illustratorsA dated record of your work
- Photo metadata removerSee and remove EXIF and GPS location
- Content Credentials checkerSee whether a file carries a C2PA manifest
- How to protect yourself from AI forgeriesSteps for creators
- Creator timestampingDate drafts, masters and finished work
- Timestamp music filesShow a demo existed by a date
- All free toolsPrivate, in-browser tools
Date your work before you publish it
Add a layer of protection against AI and human fraud and disputes: a dated, checkable record of the exact file you hold.
- Open the app and sign in with email or Google.
- Add your files. They stay on your computer.
- Seal them and download your Proof of Time certificates.
First Pooled TimeBind free, no card, then from US$5. See pricing.