The reason why Search is down is because of the Board Leech sites.

[Lensman]

As Ray over at Jupiter said in his thread (http://www.gofuckyourself.com/showthread.php?t=470021), GFY is under a constant barrage of spiders and other data theives. It has gotten so bad, that we have a full time person to deal with the load issues. It is so bad, that that is the reason that search keeps going down, and that is why it is off right now.

GFY serves the adult community and is the most active industry forum, we would never want to charge for posting, searching, etc. All we ask is that people give as well as receive, that they contribute posts, do business, and have fun. Make no mistake. GFY does not authorize the theft of the content that resides on GFY servers. The board leech sites are unauthorized content theives, it's the same if I were to spider a dating site, and steal all of the info that was posted there. They do not ask permission, and we request that they do not do it.

Why are board leeches bad?
1. They slow the board down, every day.
2. They have crashed the board several times with their bots.
3. They keep people from contributing to the forum they read, if you can read it without visiting, you are less likely to post.
4. It's ILLEGAL content theft.

I am asking nicely for all board leeches to stop republishing GFY. If you won't do it nicely, we can do it not so nice. We have taken legal action on board leeches in the past, and we will in the future. We start with a DCMA notice, it's usually enough to lose your hosting. Then we will escalate it from there.

Some people ask: Why do smaller boards allow it? Because they are SMALLER, and they hope to mooch the traffic.

I hope this ANSWERS Notjoe's question of why don't we want him to take GFY's content and repost it elsewhere.

On another note, we are working on a more powerful search solution, and should have it back up next week.

[Ice]

Thanks for the update Lens............. enjoy your weekend

[cyber_ninja]

[JD]

kickass man! The search function is greatly missed....

[AlienQ - BANNED FOR LIFE]

And then there was light.

Most of ya folks should have known this anyways, but its always good to have someone explaine it to stupid people so they understand the deal.

Lens translation = "Content thieves Stop fucking with GFY or I will get real"

[budz]

can't wait

thx for the post

[Manowar]

[azguy]

I'm not affiliated with any of those "leeching services", but technically speaking that's not content theft, at least not directly from you. The content still belongs to the posters as per your terms and conditions. Besides, search engines are a core element of the net and GFY is just like any other site out there that gets cached by Google/Yahoo.

http://search.yahoo.com/search?p=sit...238&fl=0&x=wrt

[Crypt]

Thanks Notjoe for crashing the board.
Now we know its your fault why we cant search atm.

Tard

[Manowar]

Originally Posted by Crypt

Thanks Notjoe for crashing the board.
Now we know its your fault why we cant search atm.

Tard

and he was the one to complain

[AlienQ - BANNED FOR LIFE]

Originally Posted by azguy

I'm not affiliated with any of those "leeching services", but technically speaking that's not content theft. The content still belongs to the posters as per your terms and conditions. Besides, search engines are a core element of the net and GFY is just like any other site out there that gets cached by Google/Yahoo.

http://search.yahoo.com/search?p=sit...238&fl=0&x=wrt

Idiot.
The posters have chosen where there content is published no where else has the rights to redistribute it or duplicate it.

[D-Money]

Best of luck Lens, I'm always impressed with how your team handles these things.

[goBigtime]

Not for nothin, but I think NotJoe's solution came after GFYs search was down for several months. NotJoe probably wouldn't have even thought to have created his site if it weren't for GFY search being down so much.

[azguy]

Originally Posted by AlienQ

Idiot.
The posters have chosen where there content is published no where else has the rights to redistribute it or duplicate it.

Can't you have a serious conversation? Why the hell do you need to call someone an idiot? Grow up.

Google and Yahoo cache the entire net, what makes you think one forum is different than the rest? Those "leeching services" simply link back to GFY, it's not like they post the entire content.

[machinegunkelly]

Lens can you email me @ [email protected]
I've tried to reach you and I guess somethings a miss :P

[Crypt]

Originally Posted by Manowar

and he was the one to complain

Yeah because his bot rely on search probably

[WiredGuy]

What about sites like saerchtheboards?
The search of GFY works from there, but are they considered to be a board leech?
WG

[uno]

Can't you just make a spider trap to kill the bad bots?

[azguy]

Originally Posted by AlienQ

Idiot.
The posters have chosen where there content is published no where else has the rights to redistribute it or duplicate it.

You do realize that there are search engines out there, right? Why do you use Google if they search someone else's content? Those "leechers" did not even duplicate the content, they merely linked to the specific thread (unless there are other sites which I'm not currently aware of who do just that).

[Paraskass]

Lensman, great news.

thanks.

[Manowar]

Originally Posted by WiredGuy

What about sites like saerchtheboards?
The search of GFY works from there, but are they considered to be a board leech?
WG

i think thats the one in question

[CDSmith]

That's good enough for me. Done and done.

Now back to work.

[goBigtime]

Originally Posted by WiredGuy

What about sites like saerchtheboards?
The search of GFY works from there, but are they considered to be a board leech?
WG

hahahahahahahahahahahahahahaha = Notjoe, WG.

[azguy]

Originally Posted by Manowar

i think thats the one in question

Yes, we were talking about that one. All they're doing is link to the thread and show a short excerpt. I heard Google is doing the same thing, not sure A 60 billion dollar leecher.

[AlienQ - BANNED FOR LIFE]

Originally Posted by azguy

You do realize that there are search engines out there, right? Why do you use Google if they search someone else's content? Those "leechers" did not even duplicate the content, they merely linked to the specific thread (unless there are other sites which I'm not currently aware of who do just that).

Ya can ask Lens that however here is my basic take....

THE SEARCH ENGINES ARE NOT GOOD FOR THIS BOARD ANYWAYS.
Turn em away he does not need them and he does not need Mainstream surfers comming here.

Sure some get through but I can tell ya why there are boosts in Newbies here from time to time and they are nothing more than surfers.
Ya want more surfers here?

Mainstream SE BOTS = BAD

[digifan]

It's so sad when they try to ruin gfy.. hope it's the resolution solution!

[tranza]

Thanks for clearing that up for everybody Lens!

[azguy]

Originally Posted by AlienQ

Ya can ask Lens that however here is my basic take....

THE SEARCH ENGINES ARE NOT GOOD FOR THIS BOARD ANYWAYS.
Turn em away he does not need them and he does not need Mainstream surfers comming here.

Sure some get through but I can tell ya why there are boosts in Newbies here from time to time and they are nothing more than surfers.
Ya want more surfers here?

Mainstream SE BOTS = BAD

I understand that. All I'm saying is that technically it's not entirely illegal to do what those "leechers" do and it's common practice in this beast called the world wide web.

[Juicy D. Links]

Originally Posted by Lensman

4. It's ILLEGAL content theft.

I am content?

[wdsguy]

Originally Posted by azguy

Can't you have a serious conversation? Why the hell do you need to call someone an idiot? Grow up.

Google and Yahoo cache the entire net, what makes you think one forum is different than the rest? Those "leeching services" simply link back to GFY, it's not like they post the entire content.

Don't be so hard on him, he didn't take his meds today.

[loverboy]

slendid work Lens

[Juicy D. Links]

Originally Posted by wdsguy

Don't be so hard on him, he didn't take his meds today.

i took 2 xanax and 2 paxils today

[AlienQ - BANNED FOR LIFE]

Originally Posted by azguy

I understand that. All I'm saying is that technically it's not entirely illegal to do what those "leechers" do and it's common practice in this beast called the world wide web.

I apologise for the idiot part earlier.
anyways...
Sure it's illegal if ya think about it.
Just beacuse its common practice does not mean its fair or legal.

[Lensman]

Board republishers and search engines are two different things.

And taking content without respecting the wishes of the site spidered is content theft.

[Lensman]

Originally Posted by Juicy D. Links

I am content?

Funny to think about, eh?

[ronaldo]

I'm GUESSING what pissed a lot of people off, was just not knowing if and/or when it would be back.

One minute it was there, but the next minute it was gone without explanation. (Note-I certainly may have missed the explanation if there was one).

Your explanation along with Ray's will certainly go a long way to appease most.

[pornguy]

Thanks for the update, and let us know how we can help in any other way.

[azguy]

Originally Posted by AlienQ

I apologise for the idiot part earlier.
anyways...
Sure it's illegal if ya think about it.
Just beacuse its common practice does not mean its fair or legal.

It's not as black and white as some may think. Since in this case Lensman doesn't own the content (I didn't see anything saying otherwise in the terms & conditions) and therefore it still belongs to the author, he might have problems suing the leechers on his users' behalf. It's a VERY complex situation when message boards are involved.

[AlienQ - BANNED FOR LIFE]

Lens may not technically own it but its his right to protect it.

[pr0]

And to all you potential thieves out there....if you republish PG in ANY FORM, we expect written consent of F30 INC., Party Goat LTD. & the National Football League.

Bitches

[azguy]

Originally Posted by Lensman

Board republishers and search engines are two different things.

And taking content without respecting the wishes of the site spidered is content theft.

STBoards is a search engine in every respect. There may be other leechers who operate sites that republish entire GFY threads, but I'm assuming we're still discussing SrchTheBoards.

It may be 'content theft' (which in the case of STBoards it is clearly not), but you need to realize that you're not the owner of the users' posts on GFY, not until you clearly specify this in the terms & conditions of the forum. From my experience, the absence of a specific statement means that the copyright belongs to the author of the post.

[V_RocKs]

Originally Posted by Lensman

Board republishers and search engines are two different things.

And taking content without respecting the wishes of the site spidered is content theft.

Exactly... Google looks for a robots text file and only goes where you want it to... Board leeching spyders do not follow a robots file and usually do thier handy work by abusing the search function.

[azguy]

Originally Posted by AlienQ

Lens may not technically own it but its his right to protect it.

Nope. He can sue Google for republishing the site's logo in their cache DB, but not the content of the actual posts, not in this case. Google may comply if such DMCA notice is sent to them, but they'd win should they choose to fight it.

[Sosa]

thanks for the update

[AlienQ - BANNED FOR LIFE]

Why would lens want to own copyright of peoples posts?
To much Liability.

He is housing the statements and once its in his house he has the choice of who or what can access the information.

It's pretty cut and dry.

[Big Ray]

Originally Posted by WiredGuy

What about sites like saerchtheboards?
The search of GFY works from there, but are they considered to be a board leech?
WG

Hey WG.

You make a good point, however, their is a big difference in the google and yahoo spider than the ones that try and pull 200 threads a second all day every day. What we have to contend with are aggressive screen scrapers that feel the need to index every post the second it's made. They are highly aggressive and are not throttled. They behavior more like a DoS attack than a bot.

People ask why we don't ban them. The answer is we try, however, the majority of them use anonymous proxies and I don't mean a few.. I am talking hundreds. When we block those, they get a new batch. It's a constant battle which we fight every day.

The search feature is widely used to attempt to create search notification systems. Some companies do this, and hit the site once a day to see what people are saying about them or "keyword". Others are constantly hitting search for keywords like "need content" or "need hosting". Ever wonder how the same folks always post first to a request for product or service thread? now ya know. lol

Ray

[azguy]

Originally Posted by V_RocKs

Exactly... Google looks for a robots text file and only goes where you want it to... Board leeching spyders do not follow a robots file and usually do thier handy work by abusing the search function.

Not true. The new generation of spiders is going to ignore those. www.dipsie.com is spidering the 'Deep Web' and are not following the robots.txt anymore. They log on to sites such as GFY with a login/pass and cache everything. That I know for a fact, I've met with them a while ago.

[Babaganoosh]

Lensman, what the search really needs is an image verification. That would stop this shit immediately. It wouldn't be tough for one of your geeks to whip up.

[WiredGuy]

Originally Posted by Big Ray

Hey WG.

You make a good point, however, their is a big difference in the google and yahoo spider than the ones that try and pull 200 threads a second all day every day. What we have to contend with are aggressive screen scrapers that feel the need to index every post the second it's made. They are highly aggressive and are not throttled. They behavior more like a DoS attack than a bot.

People ask why we don't ban them. The answer is we try, however, the majority of them use anonymous proxies and I don't mean a few.. I am talking hundreds. When we block those, they get a new batch. It's a constant battle which we fight every day.

The search feature is widely used to attempt to create search notification systems. Some companies do this, and hit the site once a day to see what people are saying about them or "keyword". Others are constantly hitting search for keywords like "need content" or "need hosting". Ever wonder how the same folks always post first to a request for product or service thread? now ya know. lol

Ray

Wow, i had no idea they were pulling in data so frequently. I thought these feeds were just fetching your data daily or so. If they really are using hundreds of changeable proxies like this, I'd get ICS' attorneys involved at that point, a C&D should probably suffice. They obviously are not respecting your servers limits and circumventing your techniques to prevent their abuse. I really had no idea they were pulling data that frequently, that seems like an overkill where once per day suffices just as well.

WG

[AlienQ - BANNED FOR LIFE]

Is it OK to walk into a Library and say hey..

"Ya mind if I use your paper and your ink and start copying all the books in this place?
OK great, go ahead and start with shelf "A-B" move all the people away from that area and get all the books for that section. I dont care if people want to read this shit I am copying it and archiving it"

Your going to get one pissed off librarian and the cops will be on your ass for copying books that do not belong to you.