Service that kills mirrors and scrapers - GoFuckYourself.com

TrafficTitan · 09-01-2017, 04:07 PM

Is there an affordable online company that kills people who are doing complete site copies and are scraping content to displace you from google search results?

sarettah · 09-01-2017, 04:08 PM

Quote:

Originally Posted by TrafficTitan

Is there an affordable online company that kills people who are doing complete site copies and are scraping content to displace you from google search results?

I know a guy that says he will do that. Usually wants cash in small unmarked bills though.

.

pimpmaster9000 · 09-01-2017, 04:10 PM

Affordable and kill is a bad idea IMO...

TrafficTitan · 09-01-2017, 04:15 PM

Kills the sites, not the person lol

sarettah · 09-01-2017, 05:39 PM

Quote:

Originally Posted by TrafficTitan

Kills the sites, not the person lol

Oh, that's a whole different story.

.

bronco67 · 09-01-2017, 08:18 PM

There's always going to be thieving motherfuckers and the internet just makes it 100 times easier to get away with.

Bladewire · 09-01-2017, 10:13 PM

Quote:

Originally Posted by TrafficTitan

Is there an affordable online company that kills people who are doing complete site copies and are scraping content to displace you from google search results?

You can protect your sites in .htaccess from scrapers. Are your sites on WP installs?

Do you want me to paste some code here for you?

Barry-xlovecam · 09-02-2017, 05:11 AM

It's really cat and mouse. UFW or iptables -- firewall them out -- if you have root. However, they will change IPs or AS networking so it is a never ending game.

sarettah · 09-02-2017, 06:31 AM

Cloudflare can protect against some of the scrapers out there but just like anything, they can be beaten too.

.

rowan · 09-02-2017, 06:40 AM

Quote:

Originally Posted by Barry-xlovecam

It's really cat and mouse. UFW or iptables -- firewall them out -- if you have root. However, they will change IPs or AS networking so it is a never ending game.

I have a site that's scraped to hell and back. If you exclude Googlebot and all of the scrapers, there's probably less than 2% remaining (loads by a browser).

Over the years I've added bits and pieces to log various interesting information. The big red flag that sticks out, at least for my site: scrapers use proxies, so their IPs can change without notice, but the headers they send are usually a fixed pattern that is nothing like a real browser, so they're super easy to block.

Even a simple CAPTCHA that is triggered after say 10 loads without presenting a cookie manages to block most of them. Some IPs are constantly bashing at the site, day after day, even though they are almost perpetually 403'd or firewalled.

Guess there is a market for a service like this, if one doesn't exist... but integrating it into a customer's existing site would be interesting...

Barry-xlovecam · 09-02-2017, 09:43 AM

Quote:

Originally Posted by sarettah

Cloudflare can protect against some of the scrapers out there but just like anything, they can be beaten too.

.

Any other questions

Code:

</html>barry@paragon-DS-7:/media/barry/ADATA3$ dig NS gfy.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> NS gfy.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21089
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;gfy.com.			IN	NS

;; ANSWER SECTION:
gfy.com.		86399	IN	NS	june.ns.cloudflare.com.
gfy.com.		86399	IN	NS	theo.ns.cloudflare.com.

;; Query time: 56 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Sep 02 12:42:15 EDT 2017
;; MSG SIZE  rcvd: 88
=======================

barry@paragon-DS-7:/media/barry/ADATA3$ curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 (Official Build) (64-bit)" "https://gfy.com/"
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en">
<head>
	<!-- no cache headers -->
	<meta http-equiv="Pragma" content="no-cache" />
	<meta http-equiv="Expires" content="-1" />
	<meta http-equiv="Cache-Control" content="no-cache" />
	<!-- end no cache headers -->

 	<meta name="robots" content="noarchive">
	
	<title>GFY Webmaster Board</title>

freecartoonporn · 09-02-2017, 10:36 AM

Quote:

Originally Posted by Barry-xlovecam

Any other questions

Code:

</html>barry@paragon-DS-7:/media/barry/ADATA3$ dig NS gfy.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> NS gfy.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21089
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;gfy.com.			IN	NS

;; ANSWER SECTION:
gfy.com.		86399	IN	NS	june.ns.cloudflare.com.
gfy.com.		86399	IN	NS	theo.ns.cloudflare.com.

;; Query time: 56 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Sep 02 12:42:15 EDT 2017
;; MSG SIZE  rcvd: 88
=======================

barry@paragon-DS-7:/media/barry/ADATA3$ curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 (Official Build) (64-bit)" "https://gfy.com/"
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en">
<head>
	<!-- no cache headers -->
	<meta http-equiv="Pragma" content="no-cache" />
	<meta http-equiv="Expires" content="-1" />
	<meta http-equiv="Cache-Control" content="no-cache" />
	<!-- end no cache headers -->

 	<meta name="robots" content="noarchive">
	
	<title>GFY Webmaster Board</title>

what are you trying to show ?

Barry-xlovecam · 09-02-2017, 10:39 AM

That cloudflare is a submarine with screen doors?

Bladewire · 09-02-2017, 02:46 PM

They need to tweak their cloudflare setting is all

Not sure if they changed their old IP afternoon cloudflare, as I suggested before, if not they're still wide open.

Edit: nope they didn't change it and they have 64 hostnames pointing to that same IP :/

Step 10 here - change IP after setup

Barry-xlovecam · 09-02-2017, 03:05 PM

GFY IS using a Cloudflare IP

Luckily botnets are made of stupid computer users. Maybe, Cloudflare can control ddos spikes in traffic.

Cloudflare will not stop lone wolf hackers or scrapers. I just repeated that curl successfully from a Digital Ocean server I have in Amsterdam. So, the AS block is not checked at Cloudflare either.

Bladewire · 09-02-2017, 03:12 PM

Quote:

Originally Posted by Barry-xlovecam

GFY IS using a Cloudflare IP

Luckily botnets are made of stupid computer users. Maybe, Cloudflare can control ddos spikes in traffic.

Cloudflare will not stop lone wolf hackers or scrapers. I just repeated that curl successfully from a Digital Ocean server I have in Amsterdam. So, the AS block is not checked at Cloudflare either.

You dope.

The GFY server IP is the same as when it was first attacked.

Then GFY hid behind cloudflare, with the same IP, and continued to go down.

So clearly the only reason the attack stopped was because the attacker stopped hitting the original IP.

I pay for cloudflare business , and use cloudflare. And I looked up GFY's IP.

I just PM'd it to you

freecartoonporn · 09-04-2017, 04:22 AM

Quote:

Originally Posted by Barry-xlovecam

Any other questions

Code:

</html>barry@paragon-DS-7:/media/barry/ADATA3$ dig NS gfy.com

; <<>> DiG 9.10.3-P4-Ubuntu <<>> NS gfy.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21089
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;gfy.com.			IN	NS

;; ANSWER SECTION:
gfy.com.		86399	IN	NS	june.ns.cloudflare.com.
gfy.com.		86399	IN	NS	theo.ns.cloudflare.com.

;; Query time: 56 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sat Sep 02 12:42:15 EDT 2017
;; MSG SIZE  rcvd: 88
=======================

barry@paragon-DS-7:/media/barry/ADATA3$ curl -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 (Official Build) (64-bit)" "https://gfy.com/"
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en">
<head>
	<!-- no cache headers -->
	<meta http-equiv="Pragma" content="no-cache" />
	<meta http-equiv="Expires" content="-1" />
	<meta http-equiv="Cache-Control" content="no-cache" />
	<!-- end no cache headers -->

 	<meta name="robots" content="noarchive">
	
	<title>GFY Webmaster Board</title>

i still cant figure out gfy ip from that, what am i missing ?

thanks

Barry-xlovecam · 09-04-2017, 04:47 AM

BW I don't give a fuck how ...
PM Eric and tell what DNS server that needs to have its records fixed.

09-01-2017, 04:07 PM	#1
TrafficTitan Confirmed User Industry Role: Join Date: Nov 2012 Posts: 346	Service that kills mirrors and scrapers Is there an affordable online company that kills people who are doing complete site copies and are scraping content to displace you from google search results? __________________ https://www.hentaicity.com/ \| http://www.traffictitan.com/

09-01-2017, 04:10 PM	#3
pimpmaster9000 Too lazy to set a custom title Industry Role: Join Date: Dec 2011 Posts: 26,732	Affordable and kill is a bad idea IMO... __________________ Report a suspicious cracker: Click Here

09-01-2017, 04:15 PM	#4
TrafficTitan Confirmed User Industry Role: Join Date: Nov 2012 Posts: 346	Kills the sites, not the person lol __________________ https://www.hentaicity.com/ \| http://www.traffictitan.com/

09-01-2017, 08:18 PM	#6
bronco67 Too lazy to set a custom title Join Date: Dec 2006 Posts: 29,035	There's always going to be thieving motherfuckers and the internet just makes it 100 times easier to get away with. __________________

09-02-2017, 05:11 AM	#8
Barry-xlovecam It's 42 Industry Role: Join Date: Jun 2010 Location: Global Posts: 18,083	It's really cat and mouse. UFW or iptables -- firewall them out -- if you have root. However, they will change IPs or AS networking so it is a never ending game.

09-02-2017, 06:31 AM	#9
sarettah l8r Industry Role: Join Date: Oct 2002 Posts: 13,488	Cloudflare can protect against some of the scrapers out there but just like anything, they can be beaten too. .

09-02-2017, 10:39 AM	#13
Barry-xlovecam It's 42 Industry Role: Join Date: Jun 2010 Location: Global Posts: 18,083	That cloudflare is a submarine with screen doors?

09-02-2017, 02:46 PM	#14
Bladewire StraightBro Industry Role: Join Date: Aug 2003 Location: Monarch Beach, CA USA Posts: 56,232	They need to tweak their cloudflare setting is all Not sure if they changed their old IP afternoon cloudflare, as I suggested before, if not they're still wide open. Edit: nope they didn't change it and they have 64 hostnames pointing to that same IP :/ Step 10 here - change IP after setup

09-02-2017, 03:05 PM	#15
Barry-xlovecam It's 42 Industry Role: Join Date: Jun 2010 Location: Global Posts: 18,083	GFY IS using a Cloudflare IP Luckily botnets are made of stupid computer users. Maybe, Cloudflare can control ddos spikes in traffic. Cloudflare will not stop lone wolf hackers or scrapers. I just repeated that curl successfully from a Digital Ocean server I have in Amsterdam. So, the AS block is not checked at Cloudflare either.

09-04-2017, 04:47 AM	#18
Barry-xlovecam It's 42 Industry Role: Join Date: Jun 2010 Location: Global Posts: 18,083	BW I don't give a fuck how ... PM Eric and tell what DNS server that needs to have its records fixed.