How to block evil majestic bot

jrmarino

Well-Known Member
#1
I was looking in my logs, here is a sample:

151.48.23.53 - - [22/Dec/2007:10:00:05 -0600] "GET /index.php?id=35&player=1598&&league=4701&day=1025013 HTTP/1.1" 200 50647 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
151.48.23.53 - - [22/Dec/2007:10:00:06 -0600] "GET /index.php?id=19&day=1075011&&league=4701 HTTP/1.1" 200 47855 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
88.156.197.5 - - [22/Dec/2007:10:00:04 -0600] "GET /index.php?id=3&mavid=12&&league=1601&day=1065013 HTTP/1.1" 200 105094 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"
88.156.197.5 - - [22/Dec/2007:10:00:04 -0600] "GET /index.php?id=66&game=5&&league=1601&day=1065002 HTTP/1.1" 200 74373 "-" "MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)"

The logs are filled with this majestic bot, but the robots.txt instructs to disallow all robots to the entire document root. According to this page:

http://www.majestic12.co.uk/projects/dsearch/mj12bot.php

There is an evil botnot spoofing an older version of a legitimate bot. They suggest filtering out the request based on this text "MJ12bot/v1.0.8"

Exactly how do I do that? I am guessing it is the "Filter Request" tab on the server panel?

Can you outline the settings necessary to block this nuisance?
 
Last edited:

mistwang

LiteSpeed Staff
#3
Use rewrite rule or request fileter

Rewrite Rule

Code:
RewriteCond %{HTTP_USER_AGENT} ^MJ12bot/v1.0.8 [NC,OR]
RewriteRule .*  - [F]
filter rule
Code:
SecFilterSelective HEADER_USER_AGENT ^MJ12bot/v1.0.8
 
#5
My apology to this thread (long time): I implementing this to block Firefox/3
But not effect

113.162.128.27 - - [16/Feb/2012:15:01:41 +0700] "POST /forum/login.php?do=login HTTP/1.1" 200 4464 "http://svtoantin.com/forum/forumdisplay.php?f=122" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.8) Gecko/20100722 AskTbFXTV5/3.8.0.12304 Firefox/3.6.8"

Here:
_Rule: SecFilterSelective HEADER_USER_AGENT "Firefox/3"
 
#10
I tried both:
SecFilterSelective HTTP_USER_AGENT "MSIE\ 6"
SecFilterSelective HEADER_USER_AGENT "MSIE\ 6"

But nothing to effect

113.162.74.188 - - [02/Mar/2012:12:52:58 +0700] "GET /auto.html HTTP/1.1" 200 935 "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
113.162.74.188 - - [02/Mar/2012:12:52:58 +0700] "GET /hinh/ads.jpg HTTP/1.1" 200 15668 "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
 
Top