Curl or Wget - GoFuckYourself.com

Vapid - BANNED FOR LIFE · 12-06-2012, 06:46 AM

What do you prefer

EddyTheDog · 12-06-2012, 06:48 AM

Curl....

Vapid - BANNED FOR LIFE · 12-06-2012, 06:49 AM

#1 curl ^

HomerSimpson · 12-06-2012, 08:07 AM

cURL

fris · 12-06-2012, 08:25 AM

curl from php/ruby/python,etc

use curl and wget from console

bns666 · 12-06-2012, 08:56 AM

wget

, using it from shell only.

ottopottomouse · 12-06-2012, 08:57 AM

curl + php

V_RocKs · 12-06-2012, 09:02 AM

depends on what I am doing...

AndrewX · 12-06-2012, 09:10 AM

Quote:

Originally Posted by V_RocKs

depends on what I am doing...

From console wget is easier to use and is available even when curl is not installed (yet).

seeandsee · 12-06-2012, 09:13 AM

Curl i guess :D

Stann · 12-06-2012, 09:14 AM

wget from shell

Killswitch · 12-06-2012, 09:14 AM

Quote:

Originally Posted by fris

curl from php/ruby/python,etc

use curl and wget from console

JamesM · 12-06-2012, 09:16 AM

for downloading wget from shell.,

for doing automation curl and php.

shake · 12-06-2012, 11:29 AM

As most others have said, curl for programming, wget is a nice program to use on it's own from the command line.

idolbucks · 12-07-2012, 03:46 PM

wget -i list.txt is nice for dl'ing a huge list of things

moeloubani · 12-07-2012, 03:51 PM

im actually in the early stages of planning something here and im thinking of using curl but still trying to figure out how to do what i need to do with it

basically i need to use a login form on one site to log into another site - easy
then i need to display what was on that other site onto the new site - easy

the tricky part is i want to change the links on the new site and on the page gathered from the old one so that when someone clicks on them they stay on the new site instead of navigating out

any ideas? maybe set up some sort of iframe after the initial login? i have access to both sites

AutumnBH · 12-07-2012, 04:00 PM

Same as everyone else, curl + php or wget from the command line...

However the other day I was scraping an ebook archive. I used curl + php to scrape the html pages and get the actual pdf urls. Then in the next stage I downloaded all the pdfs using a shell script. However I found some hosts would issue a 206 error (partial content) when using wget. I ended up having to use curl from the command line to get the goods.

MrCain · 12-07-2012, 09:42 PM

Wget from the shell. CURL in PHP.

pornsprite · 12-09-2012, 11:22 AM

wget with perl and the command line curl with php

fris · 12-09-2012, 11:25 AM

Here is an example i did recently with wget

Code:

#!/bin/bash

# grab images from fishki.net blog posts

if [ $# -ne 1 ]
then
	echo "fishki.net url to grab images needed."
	exit
fi

if [[ "$1" == http://fishki.net/comment.php?id=* ]]
then
	echo "downloading..."
	dir=`date "+%Y-%m-%d_%H:%M"`;
	wget -r -p -q -nd -e robots=off -P images_$dir -A '*.jpg' -R 'tn.jpg' -H -D 'ru.fishki.net' $1
	echo -n "images saved: " ; ls images_$dir/ | grep ".jpg" -c
else
	echo "invalid url: must contain http://fishki.net/comment.php?id=xxxxx"
fi
exit 64
;;
esac

;)

mafia_man · 12-09-2012, 12:10 PM

http://aria2.sourceforge.net/

fris · 12-09-2012, 12:35 PM

get the latest 20 popular images from 500px.com

Code:

for line in `wget --quiet -O- http://500px.com/popular | grep "from=popular" | sed -n 's/.*<img src="\([^"]*\)".*/\1/p' | sed s/"3.jpg"/"4.jpg"/ | sed s/"?t".*$//`; do wget -O $RANDOM.jpg --quiet "$line"; done

VenusBlogger · 12-09-2012, 05:37 PM

I'm suprised so much people here are programmers...

so where are the people that never used CURL+WGET?

You dont wonder or ask why they use it? Are you not curious at all?

Maybe its something that could be helpful and help in your webmaster tasks.. but nobody that never used it jumps into this thread, im very surprised... No curiosity? NOBODY?... That's how humans discover things, with curiosity...

I personally used WGET in SSH before, but only a few times.

Miguel T · 12-09-2012, 06:36 PM

cURL all the way!

CYF · 12-09-2012, 06:44 PM

wget for command line, curl for scripts and stuff.

edgeprod · 12-09-2012, 06:52 PM

I don't think I've ever used curl OUTSIDE of a script, or wget INSIDE of one, so I see them as environment-based tools. I've been coding cURL stuff all weekend, so I'd definitely say I use cURL a lot more FREQUENTLY, if that's the question.

The answer of "this or that" is usually "the right tool for the job" .. so what's the job?

sandman! · 12-09-2012, 06:58 PM

wget.....

12-06-2012, 08:07 AM	#4
HomerSimpson Too lazy to set a custom title Industry Role: Join Date: Sep 2005 Location: Springfield Posts: 13,826	cURL __________________ Make a bank with Chaturbate - the best selling webcam program Ads that can't be block with AdBlockers !!! /// Best paying popup program (Bitcoin payouts) !!! PHP, MySql, Smarty, CodeIgniter, Laravel, WordPress, NATS... fixing stuff, server migrations & optimizations... My ICQ: 27429884 \| Email:

12-06-2012, 08:25 AM	#5
fris Too lazy to set a custom title Industry Role: Join Date: Aug 2002 Posts: 55,372	curl from php/ruby/python,etc use curl and wget from console __________________ Since 1999: 69 Adult Industry awards for Best Hosting Company and professional excellence. WP Stuff

12-06-2012, 08:56 AM	#6
bns666 Confirmed Fetishist Industry Role: Join Date: Mar 2005 Location: Fetishland Posts: 11,539	wget , using it from shell only. __________________ ⭐ CAM SODA ⭐ STRIPCHAT ⭐ ⭐ CHATURBATE ⭐ X LOVE CAM ⭐

12-06-2012, 08:57 AM	#7
ottopottomouse She is ugly, bad luck. Industry Role: Join Date: Jan 2010 Posts: 13,177	curl + php __________________ ↑ see post ↑ 13101

12-06-2012, 09:13 AM	#10
seeandsee Check SIG! Industry Role: Join Date: Mar 2006 Location: Europe (Skype: gojkoas) Posts: 50,945	Curl i guess :D __________________ BUY MY SIG - 50$/Year Contact here

12-06-2012, 06:46 AM	#1
Vapid - BANNED FOR LIFE Barterer Industry Role: Join Date: Aug 2004 Posts: 4,864	Curl or Wget What do you prefer Last edited by Vapid - BANNED FOR LIFE; 12-06-2012 at 06:47 AM..

12-06-2012, 06:48 AM	#2
EddyTheDog Just Doing My Own Thing Industry Role: Join Date: Jan 2011 Location: London, Spain, New Zealand, GFY - Not Croydon... Posts: 25,248	Curl....

12-06-2012, 06:49 AM	#3
Vapid - BANNED FOR LIFE Barterer Industry Role: Join Date: Aug 2004 Posts: 4,864	#1 curl ^

12-06-2012, 09:02 AM	#8
V_RocKs Damn Right I Kiss Ass! Industry Role: Join Date: Dec 2003 Location: Cowtown, USA Posts: 32,422	depends on what I am doing...

12-06-2012, 09:14 AM	#11
Stann Registered User Industry Role: Join Date: Dec 2012 Posts: 11	wget from shell __________________

12-06-2012, 09:16 AM	#13
JamesM Confirmed User Industry Role: Join Date: Nov 2012 Posts: 732	for downloading wget from shell., for doing automation curl and php. __________________ Ex GF Films \| Grab Dollars Up To 80% Rev-Share \| 255 Day Cookie \| Legal Content \| Variety of Promo Tools \| CCBill Program \| GF Niche james[at]grabdollars[dot]com \| ICQ::611-99-zero-zero-20

12-06-2012, 11:29 AM	#14
shake frc Industry Role: Join Date: Jul 2003 Location: Bitcoin wallet Posts: 4,663	As most others have said, curl for programming, wget is a nice program to use on it's own from the command line. __________________ Crazy fast VPS for $10 a month. Try with $20 free credit

12-07-2012, 03:46 PM	#15
idolbucks Confirmed User Join Date: Dec 2008 Posts: 914	wget -i list.txt is nice for dl'ing a huge list of things __________________ Idol Bucks - always more...

12-07-2012, 03:51 PM	#16
moeloubani Confirmed User Industry Role: Join Date: Dec 2007 Location: Ontario Posts: 4,235	im actually in the early stages of planning something here and im thinking of using curl but still trying to figure out how to do what i need to do with it basically i need to use a login form on one site to log into another site - easy then i need to display what was on that other site onto the new site - easy the tricky part is i want to change the links on the new site and on the page gathered from the old one so that when someone clicks on them they stay on the new site instead of navigating out any ideas? maybe set up some sort of iframe after the initial login? i have access to both sites

12-07-2012, 04:00 PM	#17
AutumnBH Confirmed User Industry Role: Join Date: Oct 2012 Location: Spamville Posts: 294	Same as everyone else, curl + php or wget from the command line... However the other day I was scraping an ebook archive. I used curl + php to scrape the html pages and get the actual pdf urls. Then in the next stage I downloaded all the pdfs using a shell script. However I found some hosts would issue a 206 error (partial content) when using wget. I ended up having to use curl from the command line to get the goods.

12-07-2012, 09:42 PM	#18
MrCain Confirmed User Join Date: Jun 2006 Posts: 3,332	Wget from the shell. CURL in PHP. __________________ Sigmund

12-09-2012, 11:22 AM	#19
pornsprite Confirmed User Industry Role: Join Date: Dec 2009 Location: Texas Posts: 1,643	wget with perl and the command line curl with php __________________ Go Fuck Yourself

12-09-2012, 12:10 PM	#21
mafia_man Confirmed User Industry Role: Join Date: Jul 2005 Location: icq#: 639544261 Posts: 1,965	http://aria2.sourceforge.net/

12-09-2012, 12:35 PM	#22
fris Too lazy to set a custom title Industry Role: Join Date: Aug 2002 Posts: 55,372	get the latest 20 popular images from 500px.com Code: for line in `wget --quiet -O- http://500px.com/popular \| grep "from=popular" \| sed -n 's/.<img src="\([^"]\)"./\1/p' \| sed s/"3.jpg"/"4.jpg"/ \| sed s/"?t".$//`; do wget -O $RANDOM.jpg --quiet "$line"; done __________________ Since 1999: 69 Adult Industry awards for Best Hosting Company and professional excellence. WP Stuff

12-09-2012, 05:37 PM	#23
VenusBlogger So Fucking Banned Industry Role: Join Date: Nov 2011 Posts: 1,540	I'm suprised so much people here are programmers... so where are the people that never used CURL+WGET? You dont wonder or ask why they use it? Are you not curious at all? Maybe its something that could be helpful and help in your webmaster tasks.. but nobody that never used it jumps into this thread, im very surprised... No curiosity? NOBODY?... That's how humans discover things, with curiosity... I personally used WGET in SSH before, but only a few times. Last edited by VenusBlogger; 12-09-2012 at 05:40 PM..

12-09-2012, 06:36 PM	#24
Miguel T ♦ Web Developer ♦ Industry Role: Join Date: May 2005 Location: Full-Stack Developer Posts: 12,472	cURL all the way! __________________ Full Stack Webdeveloper: HTML5/CSS3, jQuery, AJAX, ElevatedX, NATS, MechBunny, Wordpress

12-09-2012, 06:44 PM	#25
CYF Coupon Guru Industry Role: Join Date: Mar 2009 Location: Minneapolis Posts: 10,973	wget for command line, curl for scripts and stuff. __________________ Webmaster Coupons Coupons and discounts for hosting, domains, SSL Certs, and more! AmeriNOC Coupons \| Certified Hosting Coupons \| Hosting Coupons \| Domain Name Coupons

12-09-2012, 06:52 PM	#26
edgeprod Permanently Gone Industry Role: Join Date: Mar 2004 Posts: 10,019	I don't think I've ever used curl OUTSIDE of a script, or wget INSIDE of one, so I see them as environment-based tools. I've been coding cURL stuff all weekend, so I'd definitely say I use cURL a lot more FREQUENTLY, if that's the question. The answer of "this or that" is usually "the right tool for the job" .. so what's the job?

12-09-2012, 06:58 PM	#27
sandman! Icq: 14420613 Industry Role: Join Date: Mar 2001 Location: chicago Posts: 15,432	wget..... __________________ Need WebHosting ? Email me for some great deals [email protected]