# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file User-agent: * Disallow: /media_proxy/ Disallow: /interact/ Allow: /about Disallow: / Disallow: /users/*/followers Disallow: /users/*/following Disallow: /@*/media Disallow: /@*/with_replies Disallow: /@*/tagged/* Disallow: /media_proxy/* Disallow: /emoji/* Disallow: /packs/* Disallow: /sounds/* Disallow: /system/* Disallow: /avatars/* Disallow: /headers/*