StyneBot
StyneBot is Styne's web crawler. It identifies itself, honours robots.txt and never gets past bot challenges. Crawled pages are never used to train models.
How to recognise it
| User-Agent | Mozilla/5.0 (compatible; StyneBot/1.0; +https://styne.ai/bot) |
|---|---|
| robots.txt token | StyneBot |
| IP addresses | 13.41.17.173, 35.179.27.68, 63.254.168.189, 63.254.168.148 (machine-readable: /bot-ips.json). A request that claims to be StyneBot from any other address is not us. |
What it honours
- robots.txt (RFC 9309): the
StyneBotgroup if there is one, else*. Crawl-delay up to 60 seconds. - At most one request per host at a time, at least one second apart. A 429 or 503 answer slows it down.
noindexand opt-out signals (noai,nosnippet, TDM reservations) in meta tags and headers.- Bot challenges are detected and never bypassed: the page is skipped.
How to block it
User-agent: StyneBot Disallow: /
To remove a site or ask a question, write to hello@styne.ai.