Dataset builders
YandexAdditional is a bot operated by Yandex that crawls pages into a dataset that may be reused or shared with third parties. Retrieves data used for YandexGPT quick answers features. It honours robots.txt, and a request claiming to be YandexAdditional can be checked against a forward-confirmed reverse DNS lookup.
Request says
"YandexAdditional"
Checked against
Yandex's reverse DNS
Live check
Paste the address from the log line. It is checked against what the operator publishes.
Reference
| User agent | YandexAdditionalToken only: match on the substring. |
|---|---|
| Operator | Yandex |
| Purpose | Scrapes/analyzes data for the YandexGPT LLM. |
| Respects robots.txt | Honourssource |
| Verification method | Reverse DNSReverse DNS, forward confirmed Yandex documents a reverse-then-forward DNS check against yandex.ru, yandex.net and yandex.com hostnames. The same check covers the YandexGPT crawlers. https://yandex.com/support/webmaster/robot-workings/check-yandex-robots.html |
| Crawl pattern | Not published |
| Robots.txt tokens | YandexAdditionalYandexAdditionalBot |
| Operator documentation | None published |
Retrieves data used for YandexGPT quick answers features.
robots.txt
Paste one at the root of your domain. Rules are per token.
User-agent: YandexAdditional
User-agent: YandexAdditionalBot
Disallow: /User-agent: YandexAdditional
User-agent: YandexAdditionalBot
Allow: /User-agent: YandexAdditional
User-agent: YandexAdditionalBot
Allow: /
Disallow: /account/
Disallow: /checkout/
Disallow: /searchEdit the Disallow paths to match your own URLs.
Blocking YandexAdditional keeps your pages out of the datasets it builds from future crawls. It does not remove anything collected earlier, and how each operator uses or shares its data varies, so check their policy.
Build a file for every crawlerFAQ
YandexAdditional is a bot operated by Yandex that crawls pages into a dataset that may be reused or shared with third parties. Retrieves data used for YandexGPT quick answers features. It honours robots.txt, and a request claiming to be YandexAdditional can be checked against a forward-confirmed reverse DNS lookup.
Yes. The operator documents that YandexAdditional reads robots.txt and follows Disallow rules for its own token. Rules take effect on the next fetch of the file, which can be hours or days after you publish them.
Yandex documents a reverse-then-forward DNS check against yandex.ru, yandex.net and yandex.com hostnames. The same check covers the YandexGPT crawlers. The reference is published at https://yandex.com/support/webmaster/robot-workings/check-yandex-robots.html.
Add these user-agent tokens (YandexAdditional, YandexAdditionalBot) to robots.txt with Disallow: /. If the crawler ignores robots.txt, match the user agent at your CDN or reverse proxy and return 403 there instead.
Blocking YandexAdditional keeps your pages out of the datasets it builds from future crawls. It does not remove anything collected earlier, and how each operator uses or shares its data varies, so check their policy.