Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It is cloudflare who made the claim that they are well behaved unlike those other bots and that their behaviour can be controlled by robots.txt

If I need to treat cloudflare bots the same as malicious bots, that undermines their claim.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: