|
|
|
|
||||
|
Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums. You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today! If you have any problems with the registration process or your account login, please contact us. |
 |
|
|||||||
| Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed. |
|
|
Thread Tools |
07-15-2004, 01:30 PM
|
#1 |
|
Join The Royal Family
Join Date: Apr 2002
Posts: 25,463
|
I have a huge list of urls, need app to remove duplicates
Any good free apps that will help remove duplicate urls? I have a list of thousands of urls.
__________________
 Looking for a KICK ASS TEEN SPONSOR? Check out ROYAL CASH - THE KING OF TEEN!
Incredible webmaster tools FHGs, Morphing Blog and RSS Feeds, Embedded FLV & WMV Videos. With TOP RATIO Sites like ATMovs.com | iTeenVideo.com | TeenSexMovs.com | TeenSexMania.com |
|
|
|
07-15-2004, 01:32 PM
|
#2 |
|
Confirmed User
Join Date: May 2004
Location: South Florida
Posts: 4,134
|
import them into a database with a index field then you can export to a clean list.
|
|
|
|
|
07-15-2004, 01:40 PM
|
#3 |
|
Confirmed User
Join Date: Feb 2003
Location: Orlando, Florida
Posts: 3,521
|
I'm sure somebody here could program that for real cheap for ya
|
|
|
|
|
07-15-2004, 01:42 PM
|
#4 |
|
Pounding Googlebot
Industry Role:
Join Date: Aug 2002
Location: Canada
Posts: 34,486
|
Import it into MS Access, keep distinct rows and you're done.
WG
__________________
I play with Google. |
|
|
|
|
07-15-2004, 01:45 PM
|
#5 |
|
Confirmed User
Join Date: May 2004
Location: San Diego, CA
Posts: 1,007
|
or if you are in some sort of Unix environment (or Linux)...
cat urls.txt | sort | uniq > uniqueurls.txt
__________________
I like ducks. |
|
|
|
|
07-15-2004, 01:47 PM
|
#6 | |
|
Too lazy to set a custom title
Join Date: Mar 2002
Location: Australia
Posts: 17,393
|
Quote:
 sort -u urls.txt > uniqueurls.txt |
|
|
|
|
|
07-15-2004, 01:48 PM
|
#7 | |
|
Confirmed User
Join Date: Sep 2002
Location: Oakville, Canada
Posts: 9,134
|
Quote:
__________________
Free agent |
|
|
|
|
|
07-15-2004, 01:49 PM
|
#8 |
|
Confirmed User
Join Date: Oct 2003
Location: Apollo Beach, FL
Posts: 128
|
Free Bookmark (favorites) manager
found on google in approx 23 secs... I'm starting to think GFY stands for "Googled For You" 
__________________
 Programming/System Admin/Security ICQ 2 2 2 1 9 2 2 1 0 |
|
|
|
|
07-15-2004, 01:55 PM
|
#9 |
|
Join The Royal Family
Join Date: Apr 2002
Posts: 25,463
|
Thanks guys, found a free program called raptor and worked great
__________________
Looking for a KICK ASS TEEN SPONSOR? Check out ROYAL CASH - THE KING OF TEEN!
Incredible webmaster tools FHGs, Morphing Blog and RSS Feeds, Embedded FLV & WMV Videos. With TOP RATIO Sites like ATMovs.com | iTeenVideo.com | TeenSexMovs.com | TeenSexMania.com |
|
|
|
|
07-15-2004, 02:01 PM
|
#10 |
|
Confirmed User
Industry Role:
Join Date: Mar 2004
Location: Location: Location:
Posts: 1,245
|
learn to use a shell. its an amazing tool.
|
|
|
|
|
07-15-2004, 02:03 PM
|
#11 |
|
Disruptive Innovator
Industry Role:
Join Date: Sep 2003
Location: Vegas
Posts: 4,230
|
$sliced = explode("\n", $pizza);
$new_array = array_unique($sliced); echo implode("\n",$new_array);
__________________
C:\Code\ C:\Code\Run\ |
|
|
|
