Nginx/Apache/PHP 防止恶意User-Agent方法

------------------------------------------------------------------------

！！！以下配置方法（简单易通）

------------------------------------------------------------------------

通过恶意的User-Agent关键词来屏蔽访问，包含漏洞扫描，无用蜘蛛，采集爬虫，cc攻击，sq注入等。

Nginx 

修改配置文件，内容：

#禁止Scrapy等工具的抓取
if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) {
     return 403;
}
#禁止指定UA
if ($http_user_agent ~* "User-Agent关键词|User-Agent关键词")
{
    return 403;
}
#禁止非GET|HEAD|POST方式的抓取
if ($request_method !~ ^(GET|HEAD|POST)$) {
    return 403;
}

Apache

修改配置文件，内容： 

DocumentRoot /home/wwwroot/xxx
<Directory "/home/wwwroot/xxx">
    SetEnvIfNoCase User-Agent ".*(User-Agent关键词|User-Agent关键词)" BADBOT
    Order allow,deny
    Allow from all
    deny from env=BADBOT
</Directory>

PHP

项目入口文件或者启动文件中，设置内容：

//获取UA信息
$ua = $_SERVER['HTTP_USER_AGENT'];

//将恶意USER_AGENT关键词存入数组
$now_ua = [USER_AGENT关键词];

//禁止空USER_AGENT，dedecms等主流采集程序都是空USER_AGENT，部分sql注入工具也是空USER_AGENT
//判断是否是数组中存在的UA
if (!$ua || in_array($ua, $now_ua)) {
    die('页面错误，请稍后重试！！！');
}

---

最后，网络搜索的恶意User-Agent关键词列表（参考）

'FeedDemon', 'BOT/0.1 (BOT for JCE)', 'CrawlDaddy', 'Java', 'Feedly', 'UniversalFeedParser', 'ApacheBench', 'Swiftbot', 'ZmEu', 'Indy Library', 'oBot', 'jaunty', 'YandexBot', 'AhrefsBot', 'MJ12bot', 'WinHttp', 'EasouSpider', 'HttpClient', 'Microsoft URL Control', 'YYSpider', 'jaunty', 'Python-urllib', 'lightDeckReports Bot', 'Abonti', 'aggregator', 'AhrefsBot', 'asterias', 'BDCbot', 'BLEXBot', 'BuiltBotTough', 'Bullseye', 'BunnySlippers', 'ca-crawler', 'CCBot', 'Cegbfeieh', 'CheeseBot', 'CherryPicker', 'CopyRightCheck', 'cosmos', 'Crescent', 'discobot', 'DittoSpyder', 'DOC', 'DotBot', 'Download Ninja', 'EmailCollector', 'EmailSiphon', 'EmailWolf', 'EroCrawler', 'Exabot', 'ExtractorPro', 'Fasterfox', 'FeedBooster', 'Foobot', 'Genieo', 'grub-client', 'Harvest', 'hloader', 'httplib', 'humanlinks', 'ieautodiscovery', 'InfoNaviRobot', 'IstellaBot', 'JennyBot', 'k2spider', 'Kenjin Spider', 'Keyword Density0.9', 'larbin', 'LexiBot', 'libWeb', 'LinkextractorPro', 'linko', 'LinkScan8.1a Unix', 'LinkWalker', 'LNSpiderguy', 'lwp-trivial', 'magpie', 'Mata Hari', 'MaxPointCrawler', 'MegaIndex', 'Microsoft URL Control', 'MIIxpc', 'Mippin', 'Missigua Locator', 'Mister PiX', 'MJ12bot', 'moget', 'MSIECrawler', 'NetAnts', 'NICErsPRO', 'Niki-Bot', 'NPBot', 'Nutch', 'Offline Explorer', 'Openfind', 'panscient.com', 'ProPowerBot2.14', 'ProWebWalker', 'QueryN Metasearch', 'RepoMonkey', 'RMA', 'SemrushBot', 'SeznamBot', 'SISTRIX', 'sitecheck.Internetseer.com', 'SiteSnagger', 'SnapPreviewBot', 'Sogou', 'SpankBot', 'spanner', 'spbot', 'Spinn3r', 'suzuran', 'Szukacz1.4', 'Teleport', 'Telesoft', 'The Intraformant', 'TheNomad', 'TightTwatBot', 'Titan', 'True_Robot', 'turingos', 'TurnitinBot', 'UbiCrawler', 'UnisterBot', 'URLy Warning', 'VCI', 'WBSearchBot', 'Web Downloader6.9', 'WebAuto', 'WebBandit', 'WebCopier', 'WebEnhancer', 'WebmasterWorldForumBot', 'WebReaper', 'WebSauger', 'Website Quester', 'Webster Pro', 'WebStripper', 'WebZip', 'Wotbox', 'wsr-agent', 'WWW-Collector-E', 'Xenu', 'yandex', 'Zao', 'Zeus', 'ZyBORG', 'coccoc', 'Incutio', 'lmspider', 'memoryBot', 'SemrushBot', 'serf', 'Unknown', 'uptime files', 'BOT/0.1', 'YisouSpider', 'Jullo', 'YandexBot', 'WinHttp', 'Linguee Bot', 'HTTrack', 'harvest', 'audit', 'dirbuster', 'pangolin', 'nmap', 'sqln', 'hydra', 'Parser', 'libwww', 'BBBike', 'sqlmap', 'w3af', 'owasp', 'Nikto', 'fimap', 'havij', 'zmeu', 'FeedDemon', 'Indy Library', 'Alexa Toolbar', 'AskTbFXTV', 'AhrefsBot', 'CrawlDaddy', 'CoolpadWebkit', 'Java', 'Feedly', 'UniversalFeedParser', 'ApacheBench', 'Swiftbot', 'ZmEu', 'oBot', 'jaunty', 'Python-urllib', 'lightDeckReports Bot', 'YYSpider', 'DigExt', 'HttpClient', 'MJ12bot', 'heritrix', 'EasouSpider', 'Ezooms', 'BabyKrokodil', 'netsparker', 'httperf', 'SF'

--------------------------------

！！！仅供参考！！！