Anybody know some good screen scraper software? - GoFuckYourself.com

Mike Semen · 10-05-2008, 06:23 AM

Spam me!

grumpy · 10-05-2008, 07:14 AM

you mean like this?

http://www.screen-scraper.com/

Mike Semen · 10-05-2008, 08:25 AM

Yes thanks, but I more after recommendations...

gornyhuy · 10-05-2008, 08:36 AM

I usually write my own custom scraper scripts for mainstream data acquisition purposes, but for once I'm up against a system that has good IP activity detection and it is defeating all my tricks.

Does anybody have a reliable proxy service that actually has all working proxies, that I can access from within a script and that I can hit a few times a second without getting banned?

PS: this has nothing to do with spam, email slurping, or anything shady. Its totally legit, but the publishers of the data are total pricks so I have to jump through these ridiculous hoops.

gornyhuy · 10-05-2008, 11:25 AM

*bump*

Screen-Scraper is $2500.... thats a bit steep.

Mike Semen · 10-05-2008, 01:16 PM

Quote:

Originally Posted by gornyhuy

*bump*

Screen-Scraper is $2500.... thats a bit steep.

Indeed. Last company I worked at one of our techies wrote a custom one on a couple of hours. If someone wants to do it for a reasonable price I might consider it.

Rorschach · 10-05-2008, 01:21 PM

Quote:

Originally Posted by gornyhuy

I usually write my own custom scraper scripts for mainstream data acquisition purposes, but for once I'm up against a system that has good IP activity detection and it is defeating all my tricks.

Does anybody have a reliable proxy service that actually has all working proxies, that I can access from within a script and that I can hit a few times a second without getting banned?

PS: this has nothing to do with spam, email slurping, or anything shady. Its totally legit, but the publishers of the data are total pricks so I have to jump through these ridiculous hoops.

Use Tor

678

gornyhuy · 10-05-2008, 03:43 PM

Quote:

Originally Posted by Rorschach

Use Tor

678

Sweet! Looks promising. I was thinking tor was a browser-only type of system, but it looks like I can get it going for my scripts.

Slow as all hell, but I guess thats the price you pay.
Thanks!

10-05-2008, 06:23 AM	#1
Mike Semen Confirmed User Join Date: Dec 2001 Location: London Town Posts: 2,924	Anybody know some good screen scraper software? Spam me! __________________ ICQ 1454 81 522 \|

10-05-2008, 07:14 AM	#2
grumpy Too lazy to set a custom title Join Date: Jan 2002 Location: Holland Posts: 9,870	you mean like this? http://www.screen-scraper.com/ __________________ Don't let greediness blur your vision \| You gotta let some shit slide icq - 441-456-888

10-05-2008, 08:25 AM	#3
Mike Semen Confirmed User Join Date: Dec 2001 Location: London Town Posts: 2,924	Yes thanks, but I more after recommendations... __________________ ICQ 1454 81 522 \|

10-05-2008, 08:36 AM	#4
gornyhuy Chafed. Join Date: May 2002 Location: Face Down in Pussy Posts: 18,041	I usually write my own custom scraper scripts for mainstream data acquisition purposes, but for once I'm up against a system that has good IP activity detection and it is defeating all my tricks. Does anybody have a reliable proxy service that actually has all working proxies, that I can access from within a script and that I can hit a few times a second without getting banned? PS: this has nothing to do with spam, email slurping, or anything shady. Its totally legit, but the publishers of the data are total pricks so I have to jump through these ridiculous hoops. __________________ icq:159548293 Last edited by gornyhuy; 10-05-2008 at 08:38 AM..

10-05-2008, 11:25 AM	#5
gornyhuy Chafed. Join Date: May 2002 Location: Face Down in Pussy Posts: 18,041	bump Screen-Scraper is $2500.... thats a bit steep. __________________ icq:159548293