Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums.

You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today!

If you have any problems with the registration process or your account login, please contact us.

Post New Thread Reply

Register GFY Rules Calendar Mark Forums Read
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >
New Webmasters ask "How-To" questions here. This is where other fucking Webmasters help.

 
Thread Tools
Old 10-06-2017, 04:25 PM   #151
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
no in bash.sh the ; at the end of a statement is not needed
var=something (declaration)
like JavaScript
$var beneath (a declared variable)
like
echo $var

the caps are what i did they could be in lowercase too -- but bash .sh is case sensitive

in a terminal

$ dothis; sothat; is this; && dosomethindgood | (<pipe>) to the next statement
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-09-2017, 02:19 PM   #152
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
So,
I did that...
Code:
#!/bin/bash
# Shell script to backup MySql database
# To backup Nysql databases file to /backup dir and later pick up by your
# script. You can skip few databases from backup too.
# For more info please see (Installation info):
# http://www.cyberciti.biz/nixcraft/vivek/blogger/2005/01/mysql-backup-script.html
# Last updated: Aug - 2005
# --------------------------------------------------------------------
# This is a free shell script under GNU GPL version 2.0 or above
# Copyright (C) 2004, 2005 nixCraft project
# -------------------------------------------------------------------------
# This script is part of nixCraft shell script collection (NSSC)
# Visit http://bash.cyberciti.biz/ for more information.
# -------------------------------------------------------------------------
STARTTIME=date +%s

MyUSER=root     # USERNAME
MyPASS=Alfarenna79   # PASSWORD
MyHOST=localhost        # Hostname
 
# Linux bin paths, change this if it can't be autodetected via which command
MYSQL="$(which mysql)"
MYSQLDUMP="$(which mysqldump)"
CHOWN="$(which chown)"
CHMOD="$(which chmod)"
GZIP="$(which gzip)"
 
# Backup Dest directory, change this if you have someother location
DEST="/var/backup"
 
# Main directory where backup will be stored
MBD="$DEST/mysql"

#elimino vecchi backup
rm $MBD/*
 
# Get hostname
HOST="$(hostname)"
 
# Get data in dd-mm-yyyy format
NOW="$(date +"%d-%m-%Y")"
 
# File to store current backup file
FILE=""
# Store list of databases
DBS=""
 
# DO NOT BACKUP these databases
IGGY="information_schema cond_instances mysql performance_schema phpmyadmin"
 
[ ! -d $MBD ] && mkdir -p $MBD || :
 
# Only root can access it!
$CHOWN 0.0 -R $DEST
$CHMOD 0600 $DEST
 
# Get all database list first
DBS="$($MYSQL -u $MyUSER -h $MyHOST -p$MyPASS -Bse 'show databases')"
 
for db in $DBS
do
    skipdb=-1
    if [ "$IGGY" != "" ];
    then
        for i in $IGGY
        do
            [ "$db" == "$i" ] && skipdb=1 || :
        done
    fi
 
    if [ "$skipdb" == "-1" ] ; then
        #FILE="$MBD/$db.$HOST.$NOW.gz"
        #no gzip, comprimo dopo tutta la cartella
        FILE="$MBD/$db.$HOST.$NOW.sql"

        # do all inone job in pipe,
        # connect to mysql using mysqldump for select mysql database
        # and pipe it out to gz file in backup dir :)
        #$MYSQLDUMP -u $MyUSER -h $MyHOST -p$MyPASS $db | $GZIP -9 > $FILE

        #no gzip, comprimo dopo tutta la cartella
        $MYSQLDUMP -u $MyUSER -h $MyHOST -p$MyPASS $db > $FILE
    fi
done

#comprimo tutto
zip -r $DEST/mysql-backup-$HOST.zip $MBD/

#tar -zcvf $DEST/mysql-backup-$HOST.tar.gz $MBD

ENDTIME=date +%s

TOTTIME=$ENDTIME-$STARTTIME

echo Elapsed_time: $TOTTIME
But it tells me
Code:
/var/backup/mysql_backup: line 15: +%s: command not found
It seems that he does not like the +%s, but I really do not know how to solve it...
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-10-2017, 06:03 AM   #153
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
STARTTIME=(`date +%s`)

try like this and the time will be in epoch seconds

ENDTIME=(`date +%s`)
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-10-2017, 03:52 PM   #154
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
It works, but the result is kinda odd...

Code:
Elapsed_time: 1507671766-1507671705
Practically he tells me: the calculation made you
I tried to put quotes, parentheses etc etc, but he does not want to do it... can we do this last thing too?

It takes more time for this little thing than to configure all the server...
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-10-2017, 11:15 PM   #155
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
Maybe in $()
TOTTIME=$($ENDTIME-$STARTTIME)

Do the math -- the sum is in seconds

[email protected]:~$ bc <<< 1507671766-1507671705
61
seconds
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-11-2017, 03:30 PM   #156
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
Says:
Code:
/var/backup/mysql_backup: line 93: 1507756208-1507756139: command not found
But looking for "bc" (which I did not know), I found this:
Code:
TOTTIME=`expr $ENDTIME - $STARTTIME`
This seems to work

Now I'm worried about those odd quotes...
In PHP when I find those quotes it means that there was a copy paste error from the HTML and nothing works anymore.

So I have the habit of removing them as soon as I see them and replace them with a normal apex... in sh instead it seems to be fundamental... I surely have removed someone thinking they were a error...

I shouldn't have done any damage, because everything seems to work, but maybe I'm going to look for the original script and I see if there was someone...

P.S. It's strange how we can install an entire server, and then the simplest things make us crazy...
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-11-2017, 04:03 PM   #157
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
these are called backticks

bc is a terminal calculator program

apt install bc

man bc

Quote:
The backtick (``) is actually called command substitution. The purpose of command substitution is to evaluate the command which is placed inside the backtick and provide its result as an argument to the actual command. The command substitution can be done in two ways one is using $() and the other is "``" .Dec 22, 2011
look above^^

Quote:
STARTTIME=(`date +%s`)

try like this and the time will be in epoch seconds

ENDTIME=(`date +%s`)
I just habitually close an expression in () for clarity in my code -- it is probably in most cases superfluous (but benign)
Like
our @array =(<FILENAME>);
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-12-2017, 03:34 PM   #158
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
There's one last thing that scares me a lot

Load


Much of that red is due to the phase of moving sites and all the importing error of those damned databases.

Also the other server at the beginning had very red, then slowly it is normalized.

This is taking a little more...

But what sounds strange to me is that going to see the detail of the server, you do not understand why there is all that red.



The CPU rarely arrives at 90%, the memory is a bit chubby but it works, disk there is plenty, errors or special problems there are none...
The sites are running well, fast, without interruptions, or visible slowdowns...

Cpu sometimes says "stolen" even if it is working maybe at 70%, and already this is odd.

But it is the usual load to give more worries, sometimes even 4-5, I also saw 7 in the days of cronjob (they are still synchronizing many data due to the lack of cronjob in the other servers)

What does it actually indicate load?

And how much do I have to worry?

On a scale it goes from "quiet, goes all right" to "shit the server is going to explode, run away all before it's late, shit we'll die all ", where am I?
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-12-2017, 07:58 PM   #159
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
Your problem is your PHP script and the MySQL daemon (server). Software for your application; or,
If you look at the times of the peak usage and grep those times in the server access logs you may find that bing is indexing too many pages too fast -- you can place a directive in the robots.txt
User-agent: bingbot
Crawl-delay:$v
5
10

see
https://www.siteground.com/kb/how_to..._eng ine_bot/
https://www.bing.com/webmaster/help/...ntrol-55a30302

Slow bing down -- don't Disallow Bing they bring good converting traffic and the sell their PSaaS or indexed database to Yahoo and other search engines.

You may find Baidu is indexing too many pages too fast -- block them at your firewall I have had luck that way
Porn is illegal in China and you won't sell legit Chinese buyers either.
# Free IP2Location Firewall List by Search Engine
# Source: Whitelist Robots by Search Engine | IP2Location

Code:
whois -h v4.whois.cymru.com " -c -p 183.131.32.0/20"
AS      | IP               | BGP Prefix          | CC | AS Name
4134    | 183.131.32.0     | 183.128.0.0/11      | CN | CHINANET-BACKBONE No.31,Jin-rong Street, CN
 -c -p 12.1.72.32/27"
7018    | 12.1.72.32       | 12.0.0.0/9          | US | ATT-INTERNET4 - AT&T Services, Inc., US
" -c -p 104.193.88.0/22"
55967   | 104.193.88.0     | 104.193.88.0/24     | US | CNNIC-BAIDU-AP Beijing Baidu Netcom Science and Technology Co., Ltd., CN
or use Ruby gem nicinfo
https://github.com/arineng/nicinfo

that will give you full RDAP/whois information

Third way is just $ whois <ip address>

If you are generating many dynamic pages search engines may be causing this problem

Scrapers and *bad bots* may be the issue too.

This is what server logs are for to search for problems and find patterns.
A firewall is the way to go -- just do not answer -- drop the packet.
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-16-2017, 04:40 PM   #160
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
But Holy cow

I was away 2 days and the server was invaded by bots, just like you said...



Code:
51.255.65.66 - - [16/Oct/2017:22:25:31 +0000] "GET /27 HTTP/1.1" 302 3634 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
157.55.39.234 - - [16/Oct/2017:22:25:09 +0000] "GET /cimla+sexy+photos.com/ HTTP/1.1" 200 32929 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
93.105.187.11 - - [16/Oct/2017:22:25:05 +0000] "GET /search.php?q=shemale+mia+isabella+teacher+her+student+a+lesson+free+porn&sort=date&page=5 HTTP/1.1" 200 10438 "http://www.bigbigbigboobs.com/search.php?q=shemale+mia+isabella+teacher+her+student+a+lesson+free+porn&sort=date" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
216.244.66.245 - - [16/Oct/2017:22:25:30 +0000] "GET /search-amy+anderssen+photos+pk/ HTTP/1.1" 200 79767 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, [email protected])"
207.46.13.86 - - [16/Oct/2017:22:25:24 +0000] "GET /search-bigboob+s+saree+woman+photo+pk/ HTTP/1.1" 200 24426 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
::1 - - [16/Oct/2017:22:25:31 +0000] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g (internal dummy connection)"
66.249.64.3 - - [16/Oct/2017:22:25:16 +0000] "GET /love+sex+move/ HTTP/1.1" 200 34386 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
40.77.167.14 - - [16/Oct/2017:22:25:31 +0000] "GET /search-bbw+back+sid+girl+photos.com/ HTTP/1.1" 200 24797 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
66.249.70.19 - - [16/Oct/2017:22:25:32 +0000] "GET /74277 HTTP/1.1" 200 19774 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.70.30 - - [16/Oct/2017:22:25:32 +0000] "GET /savita+babhi/ HTTP/1.1" 200 20327 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
[email protected]:~# tail /var/log/apache2/access.log
8.37.233.40 - - [16/Oct/2017:22:26:54 +0000] "GET /download+video+bokep+jepang+rina+araki/ HTTP/1.1" 200 29346 "https://www.google.co.id/search?client=ucweb-b-bookmark&q=video+ngentot+rina+araki&oq=video+ngentot+rina+araki&aqs=mobile-gws-lite.." "Mozilla/5.0 (Linux; U; Android 6.0.1; en-US; SM-G532G Build/MMB29T) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 UCBrowser/11.3.5.972 U3/0.8.0 Mobile Safari/534.30"
49.34.127.70 - - [16/Oct/2017:22:26:56 +0000] "GET /xvillage+desi+8+saal+ki+bachi+ki+chudai+video/ HTTP/1.1" 200 31479 "android-app://com.google.android.googlequicksearchbox" "Mozilla/5.0 (Linux; Android 5.1.1; SM-J200G Build/LMY47X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.91 Mobile Safari/537.36"
93.105.187.11 - - [16/Oct/2017:22:26:30 +0000] "GET /page-17/search-googleweblight.comlite_url+2+mom+big+naked+milky+boobs+images.com/date/ HTTP/1.1" 200 26231 "http://www.monsterboobshardpics.com/page-14/search-googleweblight.comlite_url+2+mom+big+naked+milky+boobs+images.com/date/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
216.244.66.228 - - [16/Oct/2017:22:27:08 +0000] "GET /search-big+assas+larag+ass+masive+ass+huge+cock+large+cock+hardcore+anal+gp+download+free/ HTTP/1.1" 200 97124 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, [email protected])"
93.105.187.11 - - [16/Oct/2017:22:26:18 +0000] "GET /page-7/search-desi+bhabi+sexy+boob+press+fuck+pussy+mp+mobile+ipone/date/ HTTP/1.1" 200 24021 "http://www.monsterboobshardpics.com/search-desi+bhabi+sexy+boob+press+fuck+pussy+mp+mobile+ipone/date/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
93.105.187.11 - - [16/Oct/2017:22:26:23 +0000] "GET /page-14/search-boobs+milk+breathing+imeges/date/ HTTP/1.1" 200 24406 "http://www.monsterboobshardpics.com/page-9/search-boobs+milk+breathing+imeges/date/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
207.46.13.183 - - [16/Oct/2017:22:27:07 +0000] "GET /page-15/search-african+black+pussy/ HTTP/1.1" 200 24382 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
202.46.58.190 - - [16/Oct/2017:22:27:06 +0000] "GET /search-big+black+fatty+boom+shemale+fuck/ HTTP/1.1" 200 24940 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36"
51.255.65.27 - - [16/Oct/2017:22:27:09 +0000] "GET /36331 HTTP/1.1" 200 19518 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
93.105.187.11 - - [16/Oct/2017:22:26:08 +0000] "GET /page-12/search-tite+big+round+heavy+boobs+hd+pics/date/ HTTP/1.1" 200 25087 "http://www.monsterboobshardpics.com/page-7/search-tite+big+round+heavy+boobs+hd+pics/date/" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
[email protected]:~# tail /var/log/apache2/access.log
216.244.66.228 - - [16/Oct/2017:22:27:24 +0000] "GET /search-bbw+big+hips+mom+churidar+hot+photo+xxxin/ HTTP/1.1" 200 92852 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, [email protected])"
46.229.168.79 - - [16/Oct/2017:22:27:25 +0000] "GET /52199 HTTP/1.1" 200 19569 "-" "Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)"
93.105.187.11 - - [16/Oct/2017:22:27:06 +0000] "GET /search.php?q=leanne+crow+huge+boobs+fake&page=5 HTTP/1.1" 200 10927 "http://www.bigbigbigboobs.com/search.php?q=leanne+crow+huge+boobs+fake" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
93.105.187.11 - - [16/Oct/2017:22:27:02 +0000] "GET /search.php?q=windian+bhabi+tight+salwar+gand+penty+showing+sexy+pic&page=2 HTTP/1.1" 200 10397 "http://www.bigbigbigboobs.com/search.php?q=windian+bhabi+tight+salwar+gand+penty+showing+sexy+pic" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
::1 - - [16/Oct/2017:22:27:26 +0000] "OPTIONS * HTTP/1.0" 200 126 "-" "Apache/2.4.18 (Ubuntu) OpenSSL/1.0.2g (internal dummy connection)"
157.55.39.77 - - [16/Oct/2017:22:27:24 +0000] "GET /page-13/search-pornstar+aunty+sex+videos+downloadiporntv.net/ HTTP/1.1" 200 26726 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
216.244.66.233 - - [16/Oct/2017:22:27:24 +0000] "GET /303/ HTTP/1.1" 200 79410 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, [email protected])"
93.105.187.11 - - [16/Oct/2017:22:27:16 +0000] "GET /?q=face+book+hot+nice+aunty+xxx+back+side+imagedate/ HTTP/1.1" 200 9673 "http://www.bigbigbigboobs.com/search.php?q=face+book+hot+nice+aunty+xxx+back+side+imagedate&page=6" "Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0"
40.77.167.62 - - [16/Oct/2017:22:27:25 +0000] "GET /search-anteysex+photo.com/ HTTP/1.1" 200 23531 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
180.76.15.8 - - [16/Oct/2017:22:27:26 +0000] "GET /page-3/search-japanese+boobs+pics/random/ HTTP/1.1" 500 637 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
[email protected]:~# tail /var/log/apache2/access.log
157.55.39.238 - - [16/Oct/2017:22:27:20 +0000] "GET /page-5/search-african+aunty+without+dress+and+bra+big+boobs+sexy+photos/ HTTP/1.1" 200 24875 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
207.46.13.20 - - [16/Oct/2017:22:27:28 +0000] "GET /page-16/search-sa+tranny+nude+pics/ HTTP/1.1" 200 25587 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
207.46.13.39 - - [16/Oct/2017:22:27:21 +0000] "GET /desi+girl+in+loose+tshirt+pics/ HTTP/1.1" 200 27165 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
157.55.39.29 - - [16/Oct/2017:22:27:26 +0000] "GET /page-14/search-hot+sexy+aunty+boobs+in+saree+hd+picturescom/ HTTP/1.1" 200 25844 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
202.46.57.88 - - [16/Oct/2017:22:27:28 +0000] "GET /page-5/search-naked+pics+of+nicole+charming/ HTTP/1.1" 200 24457 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36"
157.55.39.149 - - [16/Oct/2017:22:26:51 +0000] "GET /page-2/search-big+boobs+pandora+peaks+bikini/ HTTP/1.1" 200 0 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
202.46.58.166 - - [16/Oct/2017:22:27:28 +0000] "GET /search-lesbian+sucking+boobs/random/ HTTP/1.1" 200 24133 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36"
164.132.161.3 - - [16/Oct/2017:22:27:31 +0000] "GET /7241 HTTP/1.1" 302 3638 "-" "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)"
207.46.13.152 - - [16/Oct/2017:22:27:30 +0000] "GET /search-big+boobs+tite+studant/ HTTP/1.1" 200 23030 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
46.229.168.67 - - [16/Oct/2017:22:27:27 +0000] "GET /sunnyleone%20sexbeg/ HTTP/1.1" 200 20184 "-" "Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)"
Now I see to read and understand quickly everything you wrote me

Thanks, just in time
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-16-2017, 07:11 PM   #161
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
So, I limited Bing from robots.txt on all my sites. For now I see no big differences, but maybe it takes a little because of the cache

I also found in my htaccess, these rules that should stop Yandex and China

Code:
RewriteCond %{HTTP_USER_AGENT} ^.*MJ12bot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Yandex [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Baidu [NC]
RewriteRule .*  - [L,F]

Then I went to see Ip2location, I joined and I generated the file, but I did not understand how to use the file that they gave me...

I have generated Linux iptables, they gave me a thing like this:

Code:
iptables -A INPUT -s 104.146.0.0/18 -j DROP
iptables -A INPUT -s 104.146.100.0/22 -j DROP
iptables -A INPUT -s 104.146.104.0/21 -j DROP
iptables -A INPUT -s 104.146.112.0/24 -j DROP
But on Ubuntu in Digitalocean there is UFW, how can I use the file ip2location in UFW

Do I need to install iptables? Will it still work UFW?

I have seen some sites where it says to open a configuration file of UFW and add the lines, but my lines have a different format and the files to be modified indicated in these sites are always different...

I also thought of changing manually this:
Code:
iptables -A INPUT -s 104.146.100.0/22 -j DROP
in this:
Code:
# block IP
-A ufw-before-input -s 104.146.100.0/22 -j DROP
and add the lines to the/etc/ufw/before.rules file, as shown in Ubuntu Wiki

But I'm not sure that doing this manually is a good idea

I'm not really understanding anything.

And I would not use the rules in httacces, first because here also have a different format from what I used in precedence, and then because they are so many...

Do I need to install Fail2ban?

I really need to fix this thing quickly because my server is merging, can you help me?
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-17-2017, 07:09 AM   #162
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
Code:
ufw deny from 192.187.100.58 to any;
ufw deny from 112.137.167.30 to any;
ufw deny from 82.117.194.229 to any;
ufw deny from 91.121.45.246   to any;
deny an ip in UFW

Code:
[email protected]:/home/work# ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 22                         ALLOW IN    99.30.xxx.xx/29            
[ 2] 80                         ALLOW IN    Anywhere                  
[ 3] 443                        ALLOW IN    Anywhere                  
[ 4] xxxx                       ALLOW IN    99.30.xxx.xx/29            
[ 5] 80,443/tcp                 ALLOW IN    Anywhere                  
[ 6] Nginx Full                 ALLOW IN    Anywhere                  
[ 7] Anywhere                   DENY IN     69.30.222.130             
[ 8] Anywhere                   DENY IN     155.133.82.122            
[ 9] Anywhere                   DENY IN     54.196.30.74              
[10] Anywhere                   DENY IN     66.240.205.0/26           
[11] Anywhere                   DENY IN     188.165.2.183             
[12] Anywhere                   DENY IN     71.6.146.130              
[13] Anywhere                   DENY IN     89.163.146.57             
[14] Anywhere                   DENY IN     139.162.199.176           
[15] Anywhere                   DENY IN     180.97.106.37             
[16] Anywhere                   DENY IN     104.193.252.165           
[17] Anywhere                   DENY IN     190.248.153.234           
[18] Anywhere                   DENY IN     142.54.183.226            
[19] Anywhere                   DENY IN     158.106.67.0/24           
[20] Anywhere                   DENY IN     170.210.156.91            
[21] Anywhere                   DENY IN     81.4.125.125              
[22] Anywhere                   DENY IN     66.240.192.128/26         
[23] Anywhere                   DENY IN     35.188.194.96             
[24] Anywhere                   DENY IN     149.202.207.121           
[25] Anywhere                   DENY IN     158.106.64.0/18           
[26] Anywhere                   DENY IN     142.54.161.10             
[27] Anywhere                   ALLOW IN    99.30.xx.xx/29 21         
[28] Anywhere                   DENY IN     66.240.192.0/18           
[29] Anywhere                   DENY IN     192.187.100.58            
[30] Anywhere                   DENY IN     112.137.167.30            
[31] Anywhere                   DENY IN     82.117.194.229            
[32] Anywhere                   DENY IN     91.121.45.246             
[33] 80 (v6)                    ALLOW IN    Anywhere (v6)             
[34] 443 (v6)                   ALLOW IN    Anywhere (v6)             
[35] 80,443/tcp (v6)            ALLOW IN    Anywhere (v6)             
[36] Nginx Full (v6)            ALLOW IN    Anywhere (v6)
Code:
[email protected]:/home/work# ufw delete 37
Deleting:
 allow 21/tcp
Proceed with operation (y|n)? y
Rule deleted
you have to run ufw status numbered each time you delete as the numbered order will change if you delete any number other than the last (end) number -- the rule chain will change.

Mapping the rules is a better idea but I haven't seen a good solution for ufw only for iptables and now nftables

ufw is an acronym for Uncomplicated FireWall
UFW: The Linux Uncomplicated Firewall <uncomplicated tutorial

iptables is sort of hard to understand and has been superseded by https://linux-audit.com/nftables-beg...fic-filtering/ <nftables

Baidu doesn't play by the rules regarding robits.txt
and will use IP to spider you without any user-agent sig that says 'baidu' making you .htaccess code useless.

get the ip CIDRs and block them in the ufw firewall.
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-17-2017, 05:23 PM   #163
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
So, I downloaded the CIDR of the engines that I want to block



and launched this:
Code:
while read line; do sudo ufw insert 1 deny from $line to any; done < cdir.txt
And he have added about 650 new rules in UFW

But in the access.log the ones I see most often are:
Opensiteexplorer.org/dotbot, [email protected]
semrush.com/bot.html
bing.com/bingbot.htm
ahrefs.com/robot/

Apart from Bing, the rest seem to be marketing tools, some more or less connected to Google or moz.com

I don't use them, but mostly I don't need them if they first destroyed my server...

Can I block them? Always via IP-UFW?

And in case, which IP should I block?
Their Ip in my access.log changes, eg:

Code:
46.229.168.76 - - [17/Oct/2017:23:04:53 +0000] "GET /search-busty%20mom%20loves%20to%20suck%20cock/ HTTP/1.1" 200 24789 "-" "Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)"
46.229.168.69 - - [17/Oct/2017:22:57:38 +0000] "GET /search-big%20brest%20sex%20photo/random/ HTTP/1.1" 200 21170 "-" "Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)"
46.229.168.67 - - [17/Oct/2017:22:55:13 +0000] "GET /desi%20girls%20boobs%20suckers%20bees%20photos2015/ HTTP/1.1" 200 31832 "-" "Mozilla/5.0 (compatible; SemrushBot/1.2~bl; +http://www.semrush.com/bot.html)"
How do I find all their IP?

Or is it better in this case to use robots.txt? Is there a serious list (robots.txt or IP) of bad bots to block?
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-18-2017, 06:00 PM   #164
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
So, in the meantime... I don't know if I made a crap...

But I did this thing...

I searched with grep some bots in the acces.log
Code:
grep ahrefs/var/log/apache2/access.log

Then I copied a few thousand lines and I created a PHP script that creates a file with only the IP, line by line (leaving the duplicates)
Code:
<?
$my_database_txt = 'seznambot.txt';

$array_righi = file($my_database_txt);
foreach($array_righi as $key => $capi){
    list($ip, $merda) = explode(" - - ", $capi);

    if(strpos($ip_list, $ip) === false){

        $ip_list = $ip_list.$ip.'
';

    }
}


$nome_file_index = "ip_list.txt";
//chmod($nome_file_index, 0666);
$file = fopen($nome_file_index, "w") ;
	fwrite($file, $ip_list);
	fclose($file);
?>
From all the games for now came out this file
http://porn-update.com/temp/bad-bot-cidr.txt

Then with the usual while I added the rules to UFW
Code:
while read line; do sudo ufw insert 1 deny from $line to any; done < /var/www/html/bad-bot-cidr.txt
If I did some crap I should be able to go back with this:
Code:
while read line; do sudo ufw delete deny from $line; done < /var/www/html/bad-bot-cidr.txt
Now it seems that something is happening, in the access.log I see a lot of Google and Bing, although for Bing I added crawl-delay: 1 in robots.txt and I also set to 1 the scan frequency on Bing webmaster tools for all My sites.

The CPU graph goes up and down at the moment, but that of load is slowly descending.



I wait a little, and I see what happens...
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-18-2017, 07:47 PM   #165
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
If the only tool in your toolbox is a hammer that is how you screw in a wood screw ...

try this:

Code:
$ cut -d'-' -f1 /home/work/domain.com/logs/access.log| grep -v '99\.3' |  uniq -c | sort -nr|sed 's/\([0-9]\) \([0-9]\)/\1:\2/g' |less

(returns unique hits:IP grep -v will delete your ip pattern)

   2742:173.208.249.226 
    189:158.69.229.6 
    155:160.202.163.148 
    153:78.190.44.124 
     91:82.165.75.132 
     64:178.137.82.201 
     62:46.2.77.72 
     62:201.18.18.173 
     62:201.18.18.173 
     62:185.81.155.40
to print to a file
instead of |less
>fileName.*

you don't need a hammer to tighten a screw

After checking the whois
Quote:
# The following results may also be obtained via:
# https://whois.arin.net/rest/nets;q=1...se&ext=netref2
#


# start

NetRange: 173.208.249.224 - 173.208.249.231
CIDR: 173.208.249.224/29
NetName: DS-249-225-231
NetHandle: NET-173-208-249-224-1
Parent: WII-OAK-2 (NET-173-208-128-0-1)
NetType: Reassigned
OriginAS: AS32097
Customer: Caruso, David (C06755517)
RegDate: 2017-10-18
Updated: 2017-10-18
Ref: https://whois.arin.net/rest/net/NET-173-208-249-224-1


CustName: Caruso, David
Address: 201 E. 16th st
City: North Kansas City
StateProv: MO
PostalCode: 64116
Country: US
RegDate: 2017-10-18
Updated: 2017-10-18
Ref: https://whois.arin.net/rest/customer/C06755517

OrgTechHandle: AWE13-ARIN
OrgTechName: Wendel, Aaron
OrgTechPhone: +1-816-256-3031
OrgTechEmail: [email protected]
OrgTechRef: https://whois.arin.net/rest/poc/AWE13-ARIN

OrgAbuseHandle: NETWO1111-ARIN
OrgAbuseName: Network Security
OrgAbusePhone: +1-816-256-3031
OrgAbuseEmail: [email protected]
Whoever the fuck he is ...

Code:
ufw deny from 173.208.249.224/29  to any;
Rule added
A CIDR is a group of IPs

__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-18-2017, 10:56 PM   #166
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
I thought to a fast way to get an IP CIDR

Code:
$ whois  173.208.249.226 |grep 'CIDR:'|cut -d':' -f2|sed -e 's/^/ufw deny from /g' -e 's/            / /g' -e 's/$/  to any;/g'

#returns
ufw deny from 173.208.249.224/29  to any;
ufw deny from 173.208.128.0/17  to any;
Use the smaller CIDR (narrower) of just the ip's if you dont want to block the whole datacenter (/18 to /24)

__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-19-2017, 07:04 PM   #167
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
So, something I did...

Now virtually are only the Google and Bing bots in my access.log.

But CPU and load are still at absurd levels...

The strangest thing, something has changed in both my servers....



This week the visits have not doubled (indeed, they have fallen a bit), but something is obviously changed, but I have no idea what it is...

I have not changed anything, I searched in all the logs I know, but I find nothing that can explain this sudden increase in CPU usage.

In the error logs I often find lines like this:
Code:
[Fri Oct 20 01:00:20.885615 2017] [core:error] [pid 4771] [client 37.9.113.202:36406] AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.
[Fri Oct 20 01:00:25.593807 2017] [core:error] [pid 4771] [client 37.9.118.28:37754] AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.
But I have no idea what it relates to, my sites don't make any redirect, the only redirect I've ever created is in htaccess to redirect http -> https, or vicevers https -> http, but they seem to work properly.

These days I have been trying to look also often at the sites and seem to work well.

Nixstat often says that the mysqld process is often at 130%, 150%, 180%, my sites definitely make an important use of MySQL, but having not changed anything, I do not understand why.
(It was very high when I had problems importing tables, but it had normalized after having them fixed.)

Not having increased the visits, and having eliminated many of the bots, who or what is using my CPU and my mysql?

I don't know what else to do to understand what's going on...


I'm kinda worried also because usually the visits increase on Saturday and Sunday, and I have no idea what will happen this week
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-20-2017, 06:03 AM   #168
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
https://www.google.com/search?client...utf-8&oe=utf-8

https://gist.github.com/JustThomas/141ebe0764d43188d4f2

I usually try searching the exact error to get some idea -- seems this may be some .htaccess rewrite errors ...
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-20-2017, 09:02 PM   #169
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
t's the first thing I always do, I ask here when I find nothing

I found a lot of solutions for wordpres, but my sites are not WordPress, and I understand very little url_rewriter...

I wrote these rules a long time ago, following guides, and I never saw this error before, until I started to manage my server...

My url_revriter is really simple and stupid, and the error does not give many clues to understand what creates it.

Code:
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} ^.*MJ12bot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Yandex [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Baidu [NC]
RewriteRule .*  - [L,F]


RewriteRule ^page-([^/]+)/$ index.php?page=$1 [L]

RewriteRule ^([^/]+)\.html$ video.php?title=$1 [L]

RewriteRule channels/ channels.php [L]

RewriteRule tags/ tags.php [L]

RewriteRule ^tags-([^/]+)\/$ tags.php?letter=$1 [L]

RewriteRule search/ search.php [L]

RewriteRule ^search-([^/]+)$ search.php?query=$1 [L]
RewriteRule ^search-([^/]+)\/$ search.php?query=$1 [L]
RewriteRule ^page-([^/]+)/search-([^/]+)/$ /search.php?page=$1&query=$2 [L]

RewriteRule ^search-([^/]+)\/([^/]+)\/$ search.php?query=$1&sort=$2 [L]
RewriteRule ^page-([^/]+)/search-([^/]+)/([^/]+)/$ /search.php?page=$1&query=$2&sort=$3 [L]

RewriteRule ^([0-9]+)$ out.php?id_photo=$1 [L]
RewriteRule ^([0-9]+)/$ out.php?id_photo=$1 [L]
RewriteRule ^([0-9]+)-([^/]+)$ out.php?id_photo=$1 [L]

But I also found another thing... Last week Google decided to scan my sites, all together, and a lot of pages of each site...

Now I'm thinking, I don't know if...

Wait a few days, maybe, when Google has finished this scan the server returns to a normal regime.

Or

Try to limit its consumption by Google, adding for example if-midified-since and Last-Modified in the headers of my pages

There were already, but time ago I had to comment on why they created problems with some crappy VPS
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-20-2017, 09:57 PM   #170
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
I had a problem with the new Apache2 version then changed to Nginx for other reasons.

Code:
	</IfModule>
	<Directory /home/xxxxx/xxx.xxx.com/public_html>
	   Order allow,deny
	   Allow from all
	   # New directive needed in Apache 2.4.3: 
	   Require all granted
	</Directory>
You might want to check for this in your configurations that it is there -- IDK if this is part of the issue or not.
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-21-2017, 06:11 PM   #171
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
But where?
I remember you had already shown me this thing, but for some strange reason in my .conf files there is not...

Where should I add, in the files mysitecom.conf?
In sites-enabled or sites-available? or both?

Then I have to disable and enable the sites?

Restart Apache clearly, but I have to reboot the server too?

I want to fix this, because my logs are still full of those errors,

Perhaps there is some new hope, in the last few hours something is changing, as it started, it seems perhaps to return to normal...

porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-21-2017, 07:42 PM   #172
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
Code:
cd /etc/apache2/sites-available
ls -1
cp file file.bk <making a backup copy
nano file
save
then
######
#then make symbolic link
Code:
a2ensite <file>
/etc/init.d/apache2 reload
######
or
service apache2 reload

reload as opposed to restart this just reloads the new configuration.
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-31-2017, 08:52 PM   #173
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
So, I made the changes on all my domains, then I waited some days, because here things never seem to happen right away... maybe because of the various caches but it seems to me always want a few days to see the changes actually applied.
(then I waited a few more days due to an flu and fever, balls)

The errors unfortunately did not go away, indeed, I find some very strange, like this:

Code:
[Wed Nov 01 01:28:37.585964 2017] [core:error] [pid 31598] [client 113.200.85.109:36600] AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer: https://m.baidu.com/from=1005640b/bd_page_type=1/ssid=0/uid=0/pu=cuid%40Yiv_ugul2ill82uxgaBWiguwHtY5iHue_u2b8_uh2iqMuHi3A%2Cosname%40baidubrowser%2Ccua%40_a2St4aQ2igyNm65fI2UN_aLXioouvhBmmNqA%2Ccut%4099mjqk4iv9grivh5gaXliouRL8_4h2NlgNEHjA4msqqSB%2Ccsrc%40ios_box_suggest_txt%2Cctv%402%2Ccfrom%401005640b%2Cbbi%40ga2Mijah2uz3uSf3lh2ti_O3sf0kuSNT0uvo8guESilSu2iuA%2Ccen%40cuid_cua_cut_bbi%2Csz%401320_2004%2Cta%40iphone_1_11.0_5_4.10%2Cusm%401/baiduid=242780940BA28A3DDDE364D833464EE4/w=0_10_/t=iphone/l=1/tc?ref=www_iphone&lid=11746036396691745334&order=3&fm=alop&tj=www_normal_3_0_10_title&url_mf_score=3&vit=osres&m=8&cltj=cloud_title&asres=1&title=JapanesePornUpdates%7CJapanese%2CAsian%2CExotic...&dict=32&w_qd=IlPT2AEptyoA_yiiC6SnGjEtwQ4INvD8&sec=25105&di=2c8f31196ae39669&bdenc=1&tch=124.167.24.701.1.0&nsrc=IlPT2AEptyoA_yixCFOxXnANedT62v3IEQGG_yRZAje5mFqyavvxHcFqZj0bNWjMIEb9gTCc&eqid=a3024dde9f9ee8001000000359f92319&wd=&clk_info=%7B%22srcid%22%3A%221599%22%2C%22tplname%22%3A%22www_normal%22%2C%22t%22%3A1509499715089%2C%22sig%22%3A%2241388%22%2C%22xpath%22%3A%22div-a-h3%22%7D
In the Error_log now I also find some errors PDO, however showing the queries to video errors are not there and everything works... all kinda odd... maybe I miss something obvious...

The CPUs continue to come and go, in the last 15 days, have behaved in a really odd way, it seems that my sites to work regularly need about 20%, then when they arrive the search engines would need another 10 servers... do not know if there is something that It does not work, but in case, it does not work only with search engines because the sites work very well.
Visits have not undergone major increases or losses. are more or less stable.



In the access_log I see very google and especially much Bing even though I added delay=1 in robots and reduced time and scan frequency in Bing Webmaster

At this point I'm kinda confused about what to do (maybe even because of the flu) and I have no clear if everything works great and if there is something that really does not work.

Suggestions to understand how the situation actually is and if I can do something to improve it?
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-31-2017, 09:48 PM   #174
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
1 sec try =5 (5 sec)

Run it until it croaks -- Who wrote that PHP script that is leaking (possibly)?

whois 113.200.85.109 CHINA ofc
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-01-2017, 09:01 PM   #175
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
Sorry, this time I did not really understand what I have to do, maybe I'm still kinda feels from the flu...

I wrote every single line of code of all my sites, my rule is to write code in the simple possible way, to avoid as many problems as possible.

I can't figure out what creates problems and how to fix them

Sorry, I'm not at my best these days...
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-01-2017, 11:27 PM   #176
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
Do I have to activate/set something on the server in order to use these?

PHP Code:
header('Last-Modified: '.gmdate('D, d M Y H:i:s'$last_mod_header).' GMT');
header('If-Modified-Since: '.gmdate('D, d M Y H:i:s'$last_mod_header).' GMT');
header('Expires: '.gmdate('D, d M Y H:i:s'time() + 604800).' GMT'); 
Can they be dangerous to the update frequency of search engine ?
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-02-2017, 08:01 AM   #177
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
IDK but if you make a script and run it -- then look at the browser w/ firefox live HTTP Headers add on; or,

curl your test page URL

Code:
[email protected]:~$ curl -X HEAD -i 'http://gfy.com/'
Warning: Setting custom HTTP method to HEAD with -X/--request may not work the 
Warning: way you want. Consider using -I/--head instead.
HTTP/1.1 200 OK
Date: Thu, 02 Nov 2017 13:55:53 GMT
Content-Type: text/html; charset=ISO-8859-1
Connection: keep-alive
Set-Cookie: __cfduid=dd618d22df242d865d99e4178888286b31509630953; expires=Fri, 02-Nov-18 13:55:53 GMT; path=/; domain=.gfy.com; HttpOnly
Set-Cookie: bbsessionhash=ecf9c70481cf1feb1bf1dd91ac56c2b1; path=/; HttpOnly
Set-Cookie: bblastvisit=1509630953; expires=Fri, 02-Nov-2018 13:55:53 GMT; path=/
Set-Cookie: bblastactivity=0; expires=Fri, 02-Nov-2018 13:55:53 GMT; path=/
Expires: Thu, 02 Nov 2017 13:55:52 GMT
Cache-Control: no-cache
Pragma: no-cache
X-UA-Compatible: IE=7
Server: cloudflare-nginx
CF-RAY: 3b77981133232579-ORD
Who the Fuck uses IE 7 today LOL

On the SEO Q?:
If your content is not changed then you would be gaming Googlebot and the other indexing bots ...
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-02-2017, 05:20 PM   #178
VRR
Registered User
 
VRR's Avatar
 
Industry Role:
Join Date: Jul 2017
Location: Sofia, Bulgaria
Posts: 13
Did not read the whole 4 pages, but:

- Any slow DB queries, or intensive queries that are being executed together? Enable the slow query log and check what will happen. If there are slow queries that mark the start of the overloads, are they malformed queries or are they regular ones? If they are a result of the overload and not the cause, they would pop up after the CPU spikes, so then what is PHP doing at the same time?
- Is the DB itself optimised, with indexes and everything?
- Is disk I/O out of the question or is there a significant disk wait during the time when the CPU hits 100%, or shortly before that moment?
- What does "top" show when this happens? Is PHP spiking its CPU usage or is MySQL showing on top?



My opinion is that you should hire someone and get them to profile the whole thing if that is possible, as the graphs above don't provide any clues. And I don't think that you have a problem with bots either.
VRR is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-02-2017, 09:59 PM   #179
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
If it was a hardware issue digital ocean would be aware of it. You are using a VPS cloud so you are not an isolated user with dedicated hardware.

You are having the same sort of problems you were having with your shared hosting.

Code:
[email protected]:/var/log/mysql# zcat error.log.4.gz

[email protected]:$ cat /var/log/apache2/error.log
Check your MySQL and Apache2 logs for any errors or information (if you haven't already).

check your PHP code.
Code:
$ php file.php > file_name_warnings.txt
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-02-2017, 11:25 PM   #180
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
So, I think we need a fuller view.

In the other VPS is hard to say, the sites could not stay online 24 hours below...

However, in Digitalocean I have 2 servers, one in NY and the other in Bangalore, but they practically go into crisis together.

The error that I find most often in the logs is always that of too many redirects, it is not gone and I do not know what else to do, but it is an error that there is always, not only when the server goes into crisis, but also when the CPU works at 20-30%.

The page with most visits in all my sites is the internal search engine, about 80-90% of the visits and a MySQL table that also contains about 80-90% of the content of the site, in this case about 400,000 search tags, about 5000 pages of content.

This is the structure of the table:
Code:
-- Struttura della tabella `tgp_search`
CREATE TABLE `tgp_search` (
  `id_search` int(11) NOT NULL,
  `query` varchar(255) NOT NULL,
  `views` int(11) NOT NULL DEFAULT '1',
  `insitemap` int(1) NOT NULL DEFAULT '0',
  `data_ins` varchar(255) NOT NULL DEFAULT '1388796621',
  `last_mod` varchar(255) DEFAULT NULL,
  `engine` varchar(255) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

-- Indici per le tabelle `tgp_search`
ALTER TABLE `tgp_search`
  ADD PRIMARY KEY (`id_search`),
  ADD KEY `query` (`query`);
ALTER TABLE `tgp_search` ADD FULLTEXT KEY `query_2` (`query`);

-- AUTO_INCREMENT per la tabella `tgp_search`
ALTER TABLE `tgp_search`
  MODIFY `id_search` int(11) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=409143;
These are the queries that are executed on the page, in this sequence:
Code:
SELECT query, MATCH(query) AGAINST('tits') as score_search FROM tgp_search WHERE MATCH(query) AGAINST('tits') ORDER BY views, score_search DESC LIMIT 25

UPDATE tgp_search SET views='2407', last_mod = '1509683869' WHERE query = :query

SELECT SQL_CALC_FOUND_ROWS id_photo, title, description, url, img, MATCH(title, description) AGAINST('tits') as score FROM tgp_photo WHERE MATCH(title, description) AGAINST('tits') ORDER BY score DESC LIMIT 0, 66

SELECT FOUND_ROWS() AS 'found_rows';

SELECT query FROM tgp_search WHERE MATCH(query) AGAINST('tits') ORDER BY views DESC LIMIT 25
This,
Code:
SELECT SQL_CALC_FOUND_ROWS id_photo, title, description, url, img, MATCH(title, description) AGAINST('tits') as score FROM tgp_photo WHERE MATCH(title, description) AGAINST('tits') ORDER BY score DESC LIMIT 0, 66
has always left me some doubt about its use of resources... They seem to be a fantastic and wonderful type of query for search engines, but maybe it weighs too much on the server?
Compared to a simple query like: LIKE '% boobs% '

I have made complete PDFs of the nixstat statistics of the last 15 days, including everything.
I/O is used very little, as well as the disk, the RAM is little, but we only have that, the most active are CPU and MySQL process.

ubuntu-1gb-nyc3-01-NIXStats.pdf ubuntu-2gb-blr1-14-04-3-NIXStats.pdf

Obvious that the site uses heavily MySQL, when the CPU goes to 100% the MySQL process goes to 150-200% (although I do not understand how it is possible )

I activated the log on long queries, about 4 hours ago, for now it is empty.

I almost thought I'd try to block Bing for a few days, just to see what happens, and if it can be his fault, but maybe it's a stupid idea.

I'm very undecided on whether or not to use the last-mod and if-modified-since headers, because I'm always afraid that once you set, big G you don't consider it anymore.

Anyway, now my sites are online 24 hours a day, seem fast and work without problems even when the server goes into crisis.
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-10-2017, 02:27 PM   #181
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
So, these days I tried, studied, tested, but htaccess and url_rewriting I understand very little...

This error is still there... there is always...
Code:
[Fri Nov 10 21:22:42.691073 2017] [core:error] [pid 23733] [client 95.108.129.196:58013] AH00124: Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.
After days of study and research I thought maybe, could be the folder structure?

I did not use the structure indicated in the tutorials "var/www/html/site.com/public_html

I public_html never liked it, so I ignored it, in site.com, there are files of my sites.




All my htaccess start with this line
Code:
RewriteBase /
That (theoretically) should indicate where it all begins, can it be that it is confused and bring to /html? So all the errors?

A virtually identical error occurs with WordPress multisite (which I have never used and I do not know) in Apache 2.4
The solution seem to be this couple of lines:
Code:
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
That I have absolutely no idea what they mean and whether they must be added to the htaccess of /html or in all the htaccess of my sites?

Any ideas to help?


P.S. other thing, can I run in a single site a different version of PHP, generally php7, and in a single site php5?
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-11-2017, 08:05 AM   #182
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
mod_rewrite - Apache HTTP Server Version 2.4

Quote:
DocumentRoot "/var/www/example.com"
AliasMatch "^/myapp" "/opt/myapp-1.2.3"
<Directory "/opt/myapp-1.2.3">
RewriteEngine On
RewriteBase "/myapp/"
RewriteRule "^index\.html$" "welcome.html"
</Directory>
/etc/apache2/sites-available

! -d not a directory
! -f not a file

! is a negation

= equal to

!= not equal to
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-13-2017, 02:53 PM   #183
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
I don't think it's my solution...

Of this there is nothing physically on the server, it is all url_rewriting
Code:
http://www.alternativegirlshardpics.com/search-boobs/
On the server there is this:
Code:
http://www.alternativegirlshardpics.com/search.php?query=boobs
If all that is not file or directory is resubmitted to index.php, pretty much all my site would be sent to index.php


I begin to have also some doubt that the problem is URL rewriter

On the same page, there is also this:
Code:
tail -f error_log|fgrep '[rewrite:'
I tried several times to launch it on the server in the past few weeks, but fails to give a result (think about it for hours and hours and hours and hours and hours)

Today I decided to download the log files locally, the result is 0, not found





On the weekend I also tried to block Bing via IP list downloaded from ip2location

The CPU behavior hasn't changed much, it's not him the problem
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-13-2017, 09:12 PM   #184
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
cd to the path for error_log

Code:
$ tac error_log|grep 'rewrite:'|cat -n|less
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-14-2017, 06:28 PM   #185
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
The answer is empty... nothing... only: (END)

Very strange...

there seem to be no errors with url_rewriter, but looking in the search engines the redirect error, the only solution I find refers to the correction of an error in htaccess in WordPress on Url_rewriter of the multisite...

I'm really confused
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-15-2017, 09:49 PM   #186
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
Sometimes strange things happen...

For example, I deleted a site, I disabled it, I deleted files and databases and deleted the virtual host files, but I forgot to change the DNS...

All site requests ended on the first site (in alphabetical order) on my server...

Now I have also changed the DNS, but I keep seeing errors like this:

Code:
[Thu Nov 16 04:15:35.503536 2017] [:error] [pid 4988] [client 66.249.64.128:48685] script '/var/www/html/mysite.com/search.php' not found or unable to stat
The site in question is a blog, all with code that I wrote, so I'm sure the search.php file does not exist, it exists instead in my tube and TGP sites.

But I find errors even stranger, like this:

Code:
[Thu Nov 16 04:16:00.637321 2017] [:error] [pid 4835] [client 199.59.91.34:49152] script '/var/www/html/mysite.com/status.php' not found or unable to stat
and status.php does not exist in any of my sites (I wrote all the code of all my sites so I'm sure it does not exist)

Sometimes I also often see errors related to wordpress, errors related to directors or files type wp-admin or wp-login, but there is no wordpress on my servers.

Code:
[Thu Nov 16 04:43:18.759137 2017] [:error] [pid 7967] [client 120.28.68.192:64647] script '/var/www/html/veryhardsexupdates.com/wp-login.php' not found or unable to stat
[Thu Nov 16 04:45:10.462677 2017] [:error] [pid 10408] [client 120.28.68.192:64801] script '/var/www/html/extremehardpics.com/wp-login.php' not found or unable to stat
Even Yandex sometimes looks for references to wordpress.

porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-15-2017, 11:44 PM   #187
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
Code:
[email protected]:~$ host  66.249.64.128
128.64.249.66.in-addr.arpa domain name pointer crawl-66-249-64-128.googlebot.com.
[email protected]:~$ host  199.59.91.34
34.91.59.199.in-addr.arpa domain name pointer cs2508.mojohost.com.
[email protected]:~$ host  120.28.68.192
Host 192.68.28.120.in-addr.arpa. not found: 3(NXDOMAIN)
They are just old links or old cached cached links. How links that never existed happened -- I really don't know.

So there is a 404 -- so what? You could set up Permanent Redirect

Quote:
Redirect permanent "/one" "http://example.com/two"
Redirect 303 "/three" "http://example.com/other"
https://httpd.apache.org/docs/curren...mod_alias.html
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-16-2017, 05:36 PM   #188
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
Something really weird is happening...

Today I got an AdSense alert with this URL

Quote:
Sample page where the violation occurred: http://analvideoupdates.bigbigbigboobs.com/search-bbw+pink+juggs+boob+remove+black+bra+sex+images/
This is because all that does not exist seems to end up in the first site of the server (which is not porn ).

The structure with subdomains (eg. analvideoupdates.bigbigbigboobs.com) existed in the old VPS where Bigbigbigboobs.com was the "main domain" and for each domain added was created a sub-domain to the main domain.

I have absolutely no idea why, or what they used to do, but that idiot of CPanel created them...

I have never indexed them or used them, but for some strange reason the search engines seem to know that they exist.

The thing that worries me now is that every time I changed VPS the "main domain" changed, not to put off line all the sites, it was created a new one and moved everything quietly.

Now I have no idea how many of these subdomains have been created and under what "main domains"...

For now I have temporarily solved the situation by moving a site, now even the first is a porn website.

I do not think I can do everything with Apache alias, I thought I would create a script that intercepts the refer (including domain, subdomain and query string) and redirect based on the sub-domain, in the right site.
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-16-2017, 08:17 PM   #189
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
If you are redirecting 404's DON'T

Better the links that do not exist dead end and get deindexed.
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-17-2017, 07:08 AM   #190
lakiya69
Registered User
 
Industry Role:
Join Date: Apr 2017
Posts: 37
Interesting Topic :3
__________________
Do you Guys Love To Watch HD FREE Big Booty Latina Videos?

Visit Here - Big Booty Latina Videos FREE

Also You Can Watch FREE Foursome Porn too so

Visit Here - Foursome Porn Videos FREE
lakiya69 is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-17-2017, 12:38 PM   #191
porn-update
Confirmed User
 
porn-update's Avatar
 
Industry Role:
Join Date: Apr 2014
Posts: 336
Quote:
Originally Posted by Barry-xlovecam View Post
If you are redirecting 404's DON'T
What do you mean?

I have this in all my httaccess and a 404 page in all my sites (e.g. 404 Page Not Found | Alternative Girls Hard Pics)
Code:
ErrorDocument 404 /404.php
ErrorDocument 403 /403.php

Quote:
Better the links that do not exist dead end and get deindexed.
This scares me... only in the last year I have changed 5 different vps, all with cpanel, this means hundreds of thousands of links to subdomains that no longer exist...

I thought I would retrieve something with these 2 sites (one per server, first site in alphabetical order).
Adult Hashtag Adult Sex Search
This type of website saved searches from the query string... at least recovery the keywords of the search are read by search engines on all those sub-domains.

What do you think? stupid idea?
porn-update is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 11-17-2017, 01:16 PM   #192
Barry-xlovecam
It's 42
 
Barry-xlovecam's Avatar
 
Industry Role:
Join Date: Jun 2010
Location: Global
Posts: 17,908
Your 404.php is OK it say 'page not found'
I meant 404 (302)-> index.php
__________________
Vive la résistance -- Carpe diem

"If you only knew the power of the dark side." ...Darth Vader
Sk.why.pe: barry_555_5 -- ICQ: three1896three617
Barry-xlovecam is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Post New Thread Reply
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >

Bookmarks

Tags
vps, centos, cwp, thinking, line, free, command, software, advice, direction, cpanel, memcache, guides, correct, reverse, varnish, tool, litespeed, proxy, initial, acpu, opcache, apache, web, past
Thread Tools




©2000-, AVN Media Network Inc



Powered by vBulletin
Copyright © 2000- Jelsoft Enterprises Limited.