why are CJ/trading scripts inaccurate... - GoFuckYourself.com

XXXManager · 08-21-2002, 12:50 PM

Up until now, trading scripts (CJ scripts included) were (and still are) relying on the value called HTTP_REFERER (one of the parameter passed by visitors? browser to your Web-server when moving from page to page) to identify the origin of a visitor and by that, accrediting the correct trader for traffic he sent. In return for traffic a trader sends to you, you send traffic back to his site (sometimes additional factors such as ?quality? of traffic influenced the amount of traffic you send back). That is how trading relations form today: If you do not get traffic from your trader you do not send any traffic back to him.
If you've ever wondered why most CJ sites requires that the return URL will be in the same domain as the one where you send traffic from, it is because it's a way to identify that traffic is sent by you ? through a match of the Domain.
Unfortunately, this is one of the biggest apparent weaknesses of trading and CJ scripts. The reason is that a considerable percentage of traffic over the Internet lacks this HTTP_REFERER parameter - over 13%! There are many reasons for that, for example: around 4% of traffic over the Internet is generated by users who use Norton personal firewall. In addition to being a firewall, it is also a system to "protects your privacy". What Norton personal firewall does is eliminating the HTTP_REFERER parameter and replacing it with an encrypted and hashed (?unrecognizable?) HTTP_WEFERER. More reasons for the lack of HTTP_REFERER will be: other privacy protection software, HTTP_REFERER disabled browsers, visitors behind proxies and more (most don't even supply a hashed version of this parameter ? not that one can really be used) ?
As a matter of fact the percentage of HTTP_REFERER "free" traffic is constantly increasing, part of it is because of the popularity of firewall while the other part is because of the increasing concern of people with privacy. This number is predicted to increase until majority of traffic over the Internet lacks this parameter.
So what is the problem? Well... if you're trading with somebody and sending 1000 hits, (assuming he uses a CJ script or any other trading script that relies on HTTP_REFERER) your trader will see only 850 to 900 hits coming from you. If you ever wondered why your traffic counters are sometimes have a totally different value than your trader?s, it is (most probably) not because he/she cheat you (unless the difference is much bigger than that), but because of the absence of HTTP_REFERER and their inability to identify all traffic sent by you as such. Also - if you ever wondered why so many sites say they send 120% of the traffic they get, it is not because they are too generous but because it goes to compensate for the lost traffic and the lack of accuracy of the script. This 120% "trick" is often used as the ?official? way to compensate for the lost traffic. But as the percentage of traffic without HTTP_REFERER grows there will be a need to increase the 120% to 150% or even 200%. This may be sooner than you think. Some use a ?special? logic saying that ?everybody looses to everybody, so in the end we are all even ;)??
Now, the question you may want to ask yourself is: ?Is this an acceptable solution to compensate for the inaccuracy? Is it a valid way? Do I like it??? While the problem is less urgent at 13%, what would you say when it triples and gets to 40%? Ask yourself the following question: ?Would you sign up to an affiliation program, where 30% of your sales are lost???

08-21-2002, 12:52 PM

You don't really expect me to read all this, do you ?

08-21-2002, 12:54 PM

Ok, I read it all, didn't get anything new out of it, guess I'll just Walk On ..... or I ask for a reinvention of the http standard protocol.

08-21-2002, 12:56 PM

Actually - sweet shit. Soon all CJ sites will die, so will TGPs and there will be less free porn

salsbury · 08-21-2002, 12:57 PM

yep, correct.

good trading scripts should let you give out specific URLs for other sites to link to. ie

www.cjsites.com/in.cgi?from=crapola.com

i think autorank will do this, but it's not exactly a trading script.

FlyingIguana · 08-21-2002, 12:57 PM

is there a better way?

Mark · 08-21-2002, 01:02 PM

Quote:

is there a better way?

Actually there is.. by coincidence its: xxxmanager.com

We use it on startgp.com and its damn cool!!

XXXManager · 08-21-2002, 01:24 PM

Quote:

Originally posted by FlyingIguana
is there a better way?

Yes FlyingIguana, there is.
One way will be, as salsbury suggested..

Quote:

Originally posted by salsbury
good trading scripts should let you give out specific URLs for other sites to link to. ie

...give an Identification of the traffic so that when it arrives the destination its source can be recognized

Quote:

Originally posted by Equinox
..... or I ask for a reinvention of the http standard protocol.

Equinox... I wouldnt hold my breath that long ;)

BTW. thx Mark

FATPad · 08-21-2002, 01:50 PM

Specific URL's (http://www.domain.com/in.cgi?myid=FATPAD) aren't a new invention.

XXXManager · 08-21-2002, 02:00 PM

Quote:

Originally posted by FATPad
Specific URL's (http://www.domain.com/in.cgi?myid=FATPAD) aren't a new invention.

I didnt say its new - I said its a good solution that works

Thomas007 · 08-21-2002, 02:01 PM

Quote:

Originally posted by FATPad
Specific URL's (http://www.domain.com/in.cgi?myid=FATPAD) aren't a new invention.

You are right, they were more used in the past and is almost faced out because it's easier just sending to the domain.
If everybody started using specific urls again, the problem would be solved.

Bad B0y · 08-21-2002, 02:18 PM

Can I try to get on your top 30.

I tested my shitty bot on cjultra and it works like a dream, but I'd love to have a little go

Bad B0y · 08-21-2002, 02:20 PM

Don't worry no syn floods or anything dirty like that.

XXXManager · 08-21-2002, 03:33 PM

Quote:

Originally posted by Tricky007
You are right, they were more used in the past and is almost faced out because it's easier just sending to the domain.
If everybody started using specific urls again, the problem would be solved.

Yep - you are correct. There was a move to domain only redirections. But what I was explaining/suggesting is that there will soon be a move back to reference based redirection and the reason is the great and increasing inaccuracy of (other

) CJ scripts.
Standing on 13-14% today - this will very soon grow considerably. It is soon to be a thing that can not be disregarded.

Only think about what will happen if the next version of ZoneAlarm includes REFERER encrypting/disabling

Bad B0y · 08-21-2002, 03:54 PM

would it not be easier to spoof if you only rely on the username that is in the url?

XXXManager · 08-21-2002, 05:47 PM

Quote:

Originally posted by Bad B0y
would it not be easier to spoof if you only rely on the username that is in the url?

Not much difference there. Its quite easy to spoof HTTP_REFERER
Besides - all easy spoofing mechanism are easily detectable.
If someone is not smart enough to use advanced cheating - he/she will be easily detected and flushed.
If someone is smart enough to deploy advanced cheating and spoofing "tools" - its very much likely that REFERER will not be a deterrant.
In any way - the most smart cheating mechanism revolve not around IDs nor around referer.

Bad B0y · 08-21-2002, 06:39 PM

but http://yoursite.com?id=blah could come from anywhere and referer can be changed to whatever just as easy.

so what would say should be secure?

XXXManager · 08-22-2002, 07:53 AM

Quote:

Originally posted by Bad B0y
but http://yoursite.com?id=blah could come from anywhere and referer can be changed to whatever just as easy.
so what would say should be secure?

If you are concerned with security - there is not much difference - thats what I was trying to say in the previous post.
http://yoursite.com?id=blah can come from anywhere - TRUE
http://yoursite.com with the appropriate REFERER can ALSO come from anywhere - that is not hard to fake.

If you trust your security issues with the use of HTTP_REFERER (or even assist with that )you are tragically mistaken. It doesnt even help - you earn nothing in that aspect. If someone wants to cheat you - he will do so while you will still get the REFERER - trust cheaters ;)
All HTTP_REFERER gives is - an easy URL to send traffic to coupled with a big and growing inaccuracy

Its like letting grocery customers pay by presenting price tags - without showing the merchendise.. It will all work fine for the honost ones. On the other side, a bad customer can "pick" tags from cheaper products or even sneak some products he never intends to show the price tag for (some would call it stealing ;) ).

IDs are not different in that aspect at all. The difference is not security - its accuracy in tracking and counting - a thing that could not be achieved with non-ID(REFERER-based) CJ scripts.

HomeGrown · 08-22-2002, 08:00 AM

Quote:

Originally posted by Equinox
You don't really expect me to read all this, do you ?

Heh, that's what I was thinking. But I am going to read it.

08-21-2002, 12:50 PM	#1
XXXManager So Fucking Banned Join Date: Mar 2002 Location: Far out in the uncharted backwaters of the unfashionable end of the Western Spiral arm of the Galaxy Posts: 893	why are CJ/trading scripts inaccurate... Up until now, trading scripts (CJ scripts included) were (and still are) relying on the value called HTTP_REFERER (one of the parameter passed by visitors? browser to your Web-server when moving from page to page) to identify the origin of a visitor and by that, accrediting the correct trader for traffic he sent. In return for traffic a trader sends to you, you send traffic back to his site (sometimes additional factors such as ?quality? of traffic influenced the amount of traffic you send back). That is how trading relations form today: If you do not get traffic from your trader you do not send any traffic back to him. If you've ever wondered why most CJ sites requires that the return URL will be in the same domain as the one where you send traffic from, it is because it's a way to identify that traffic is sent by you ? through a match of the Domain. Unfortunately, this is one of the biggest apparent weaknesses of trading and CJ scripts. The reason is that a considerable percentage of traffic over the Internet lacks this HTTP_REFERER parameter - over 13%! There are many reasons for that, for example: around 4% of traffic over the Internet is generated by users who use Norton personal firewall. In addition to being a firewall, it is also a system to "protects your privacy". What Norton personal firewall does is eliminating the HTTP_REFERER parameter and replacing it with an encrypted and hashed (?unrecognizable?) HTTP_WEFERER. More reasons for the lack of HTTP_REFERER will be: other privacy protection software, HTTP_REFERER disabled browsers, visitors behind proxies and more (most don't even supply a hashed version of this parameter ? not that one can really be used) ? As a matter of fact the percentage of HTTP_REFERER "free" traffic is constantly increasing, part of it is because of the popularity of firewall while the other part is because of the increasing concern of people with privacy. This number is predicted to increase until majority of traffic over the Internet lacks this parameter. So what is the problem? Well... if you're trading with somebody and sending 1000 hits, (assuming he uses a CJ script or any other trading script that relies on HTTP_REFERER) your trader will see only 850 to 900 hits coming from you. If you ever wondered why your traffic counters are sometimes have a totally different value than your trader?s, it is (most probably) not because he/she cheat you (unless the difference is much bigger than that), but because of the absence of HTTP_REFERER and their inability to identify all traffic sent by you as such. Also - if you ever wondered why so many sites say they send 120% of the traffic they get, it is not because they are too generous but because it goes to compensate for the lost traffic and the lack of accuracy of the script. This 120% "trick" is often used as the ?official? way to compensate for the lost traffic. But as the percentage of traffic without HTTP_REFERER grows there will be a need to increase the 120% to 150% or even 200%. This may be sooner than you think. Some use a ?special? logic saying that ?everybody looses to everybody, so in the end we are all even ;)?? Now, the question you may want to ask yourself is: ?Is this an acceptable solution to compensate for the inaccuracy? Is it a valid way? Do I like it??? While the problem is less urgent at 13%, what would you say when it triples and gets to 40%? Ask yourself the following question: ?Would you sign up to an affiliation program, where 30% of your sales are lost???

08-21-2002, 12:57 PM	#5
salsbury Confirmed User Join Date: Feb 2002 Location: Seattle Posts: 1,070	yep, correct. good trading scripts should let you give out specific URLs for other sites to link to. ie www.cjsites.com/in.cgi?from=crapola.com i think autorank will do this, but it's not exactly a trading script. __________________

08-21-2002, 12:52 PM	#2
X37375787 Guest Posts: n/a	You don't really expect me to read all this, do you ?

08-21-2002, 12:54 PM	#3
X37375787 Guest Posts: n/a	Ok, I read it all, didn't get anything new out of it, guess I'll just Walk On ..... or I ask for a reinvention of the http standard protocol.

08-21-2002, 12:56 PM	#4
X37375787 Guest Posts: n/a	Actually - sweet shit. Soon all CJ sites will die, so will TGPs and there will be less free porn

08-21-2002, 12:57 PM	#6
FlyingIguana aspiring banker Join Date: Mar 2002 Location: toronto Posts: 10,870	is there a better way?

08-21-2002, 01:50 PM	#9
FATPad Confirmed User Join Date: Oct 2001 Posts: 6,693	Specific URL's (http://www.domain.com/in.cgi?myid=FATPAD) aren't a new invention.

08-21-2002, 02:18 PM	#12
Bad B0y Confirmed User Join Date: Aug 2002 Posts: 160	Can I try to get on your top 30. I tested my shitty bot on cjultra and it works like a dream, but I'd love to have a little go

08-21-2002, 02:20 PM	#13
Bad B0y Confirmed User Join Date: Aug 2002 Posts: 160	Don't worry no syn floods or anything dirty like that.

08-21-2002, 03:54 PM	#15
Bad B0y Confirmed User Join Date: Aug 2002 Posts: 160	would it not be easier to spoof if you only rely on the username that is in the url?

08-21-2002, 06:39 PM	#17
Bad B0y Confirmed User Join Date: Aug 2002 Posts: 160	but http://yoursite.com?id=blah could come from anywhere and referer can be changed to whatever just as easy. so what would say should be secure?