User-agent: * Disallow: /ajax/ Disallow: /nfl/ranker/ Disallow: /mlb/ranker/ Disallow: /nba/ranker/ Disallow: /api/ Disallow: /json/ Disallow: /xml/ Crawl-delay: 5