我的buyvm小机机要被蜘蛛们抓死了，怎么都一起来了

fwt0125 · 发表于 2011-4-7 22:54:01

220.181.94.216 - - [07/Apr/2011:22:51:47 +0800] "GET /play/?28384-0-0.html HTTP/1.1" 200 14451 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
66.249.67.178 - - [07/Apr/2011:22:49:54 +0800] "GET /good-2807514066.jsp HTTP/1.1" 200 72 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.67.155 - - [07/Apr/2011:22:51:55 +0800] "GET /uploads/allimg/1/1a/1282367566.jpg HTTP/1.1" 200 14612 "-" "Googlebot-Image/1.0"
220.181.94.216 - - [07/Apr/2011:22:51:57 +0800] "GET /play/?28383-0-0.html HTTP/1.1" 200 14825 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.94.216 - - [07/Apr/2011:22:52:07 +0800] "GET /play/?2494-0-60.html HTTP/1.1" 200 38724 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
61.135.249.210 - - [07/Apr/2011:22:52:08 +0800] "GET /content/?24009.html HTTP/1.1" 200 12868 "-" "Mozilla/5.0 (compatible; YoudaoBot/1.0; http://www.youdao.com/help/webmaster/spider/; )"
220.181.94.216 - - [07/Apr/2011:22:52:18 +0800] "GET /play/?28726-0-0.html HTTP/1.1" 200 15690 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.108.156 - - [07/Apr/2011:22:52:18 +0800] "GET /good-4700735893.jsp HTTP/1.1" 200 50206 "-" "Baiduspider+(+http://www.baidu.com/search/spider.htm)"
220.181.94.216 - - [07/Apr/2011:22:52:27 +0800] "GET /play/?28577-0-0.html HTTP/1.1" 200 18282 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.94.216 - - [07/Apr/2011:22:52:37 +0800] "GET /play/?28356-0-0.html HTTP/1.1" 200 13461 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
66.249.67.155 - - [07/Apr/2011:22:52:39 +0800] "GET /uploads/allimg/f/f2/1301745628.jpg HTTP/1.1" 200 6542 "-" "Googlebot-Image/1.0"
220.181.94.216 - - [07/Apr/2011:22:52:47 +0800] "GET /play/?28360-0-0.html HTTP/1.1" 200 13506 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
123.122.94.119 - - [07/Apr/2011:22:52:22 +0800] "GET /attachments/lyh/%E8%80%81%E7%BD%97%E8%AF%AD%E5%BD%95_%E9%AB%98%E5%B1%B1%E6%97%8F.wma HTTP/1.1" 200 2926202 "-" "NSPlayer/11.00.6002.18392 WMFSDK/11.00.6002.18392"
220.181.108.158 - - [07/Apr/2011:22:52:51 +0800] "GET /content/?1930.html HTTP/1.1" 301 328 "-" "Baiduspider+(+http://www.baidu.com/search/spider.htm)"
220.181.108.106 - - [07/Apr/2011:22:52:51 +0800] "GET /content/?1838.html HTTP/1.1" 200 15626 "-" "Baiduspider+(+http://www.baidu.com/search/spider.htm)"
220.181.108.86 - - [07/Apr/2011:22:52:51 +0800] "GET /content/?1930.html HTTP/1.1" 200 13976 "-" "Baiduspider+(+http://www.baidu.com/search/spider.htm)"
220.181.108.85 - - [07/Apr/2011:22:52:55 +0800] "GET /content/?1893.html HTTP/1.1" 301 328 "-" "Baiduspider+(+http://www.baidu.com/search/spider.htm)"
220.181.94.216 - - [07/Apr/2011:22:52:57 +0800] "GET /play/?28361-0-0.html HTTP/1.1" 200 13756 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
220.181.108.111 - - [07/Apr/2011:22:52:59 +0800] "GET /content/?1893.html HTTP/1.1" 200 14633 "-" "Baiduspider+(+http://www.baidu.com/search/spider.htm)"

刹那芳华 · 发表于 2011-4-7 22:54:38

屏蔽啊

fwt0125 · 发表于 2011-4-7 22:56:07

原帖由 刹那芳华 于 2011-4-7 22:54 发表
屏蔽啊

不知道说什么好

xtmp · 发表于 2011-4-7 22:56:23

sougou的spider屏蔽没用，它根本不遵守robot

walkman660 · 发表于 2011-4-7 23:03:00

直接把搜狗IP拉黑

l12ab · 发表于 2011-4-7 23:07:18

在user-agent里面屏蔽spider

Host.HowPick · 发表于 2011-4-7 23:09:41

搜狗是谁家的？我只知道 gg和bd

mslxd · 发表于 2011-4-7 23:11:13

sogou是这样的呃，你要给它吃别的。。。

糕手 · 发表于 2011-4-7 23:49:53

提示: 作者被禁止或删除内容自动屏蔽

wdlth · 发表于 2011-4-7 23:59:03

上缓存吧

		自动登录	找回密码
密码			注册

糕手糕手当前离线积分 3031	发表于 2011-4-7 23:49:53 \| 显示全部楼层提示: 作者被禁止或删除内容自动屏蔽
糕手糕手当前离线积分 3031
	回复支持反对举报

[特价VPS] 我的buyvm小机机要被蜘蛛们抓死了，怎么都一起来了