For site owners
YakkeryBot
YakkeryBot reads pages that Yakkery members open or discuss, plus the home pages of widely visited sites, to write a neutral description, place the site in the directory, and find similar sites. It does not crawl the web at large.
How it identifies itself
User agent: YakkeryBot/1.0 (+/bot). Requests come from Yakkery's servers, never from members' browsers. The browser extension never uploads page content.
Limits it follows
| Rate | At most 1 request per second per host |
|---|---|
| Size | Stops reading after 5 MB |
| Timeout | 15 seconds per request |
| Refresh | A page is re-read at most every 30 days |
| Retries | Up to 3 attempts over 24 hours for server errors, honoring 429 and Retry-After |
| Paywalls | Never bypassed; paywalled pages are described from their metadata only |
Opting out
Add this to your robots.txt. YakkeryBot stops fetching your site within 24 hours:
User-agent: YakkeryBot Disallow: /
Your site stays in the directory by its domain name with the label "not summarized at the site owner's request", and members can still discuss it. To correct a description instead, claim your site from its page on Yakkery.