如果网站sitemap地址被分成多个文件。Robots.txt应该这么写:
User-agent: Googlebot
Disallow: /nogooglebot/
User-agent: *
Allow: /
Sitemap: http://www.example.com/sitemap1.xml
Sitemap: http://www.example.com/sitemap2.xml
Sitemap: http://www.example.com/sitemap3.xml
Sitemap: http://www.example.com/sitemap4.xml
Sitemap: http://www.example.com/sitemap5.xml
参考地址:Multiple Sitemap: entries in robots.txt? - Stack Overflow