# Whitelist User-agent: AdIdxBot User-agent: AdsBot-Google User-agent: AdsBot-Google-Mobile User-agent: AdsBot-Google-Mobile-Apps User-agent: AOLBuild User-agent: APIs-Google User-agent: Applebot User-agent: Baiduspider User-agent: Bingbot User-agent: BingPreview User-agent: Blekkobot User-agent: DuckDuckBot User-agent: FacebookExternalHit User-agent: Google Favicon User-agent: Googlebot User-agent: Googlebot-Image User-agent: Googlebot-Mobile User-agent: Googlebot-News User-agent: Googlebot-Video User-agent: LinkedInBot User-agent: Mediapartners-Google User-agent: MSNBot User-agent: MSNBot-Media User-agent: MSNPTC User-agent: Slurp User-agent: Teoma User-agent: Twitterbot User-agent: Yahoo-Blogs User-agent: Yahoo-MMCrawler User-agent: YandexBot Disallow: /*? Disallow: # Blacklist User-agent: ia_archiver Disallow: / User-agent: * Disallow: /