GoFuckYourself.com - Adult Webmaster Forum - I have 2 text files from which i need to remove duplicates

GoFuckYourself.com - Adult Webmaster Forum (https://gfy.com/index.php)

- Fucking Around & Business Discussion (https://gfy.com/forumdisplay.php?f=26)

- - I have 2 text files from which i need to remove duplicates (https://gfy.com/showthread.php?t=793949)

Dirty F

12-21-2007 10:38 AM

I have 2 text files from which i need to remove duplicates

I have 2 text files with names.
One is a big list with xxxx amount of names and the text file is a small list with xxx amount of names. The smaller list names also appear on the bigger list. Now i need a way to compare both list and remove ALL the names that appear on both lists.

Know what i mean?

Lace	12-21-2007 10:55 AM

get a list manager...

Voodoo

12-21-2007 10:56 AM

http://www.flashpeak.com/slimlist/slimlist.htm

Dirty F

12-21-2007 10:56 AM

I have list managers but they can only remove the duplicates and not the original name as well. I want any duplicate name to be gone. So it doesnt appear at all on the list anymore.

Voodoo

12-21-2007 10:58 AM

SlimList will remove duplicate entries, but only the duplicate.

Are you looking to do something like this:
LIST A: name1 name2 name3 name4
LIST B: name1 name5 name6 name7

ACTION: remove name1 from both lists.

Is that what you're talking about? Or simply removing duplicates?

ne0	12-21-2007 10:59 AM

Quote:

Originally Posted by Dirty F (Post 13546773)

if the names are one per line, and they appear the same way on the first and the second list, you can do this using a *nix server:
cat list1.txt list2.txt |sort |uniq > list3.txt
the list3.txt will contain both lists, ordered and without the dupes.

Lace	12-21-2007 11:00 AM

Or you could throw it in a XLS, sort and remove by hand.

Dirty F

12-21-2007 11:01 AM

Quote:

Originally Posted by Voodoo (Post 13546838)

http://www.flashpeak.com/slimlist/slimlist.htm

I dont see how this can do what i need.

Dirty F

12-21-2007 11:03 AM

Quote:

Originally Posted by Lace (Post 13546864)

Or you could throw it in a XLS, sort and remove by hand.

If i would want to do it by hand i wouldnt ask for a solution here :)

moogle

12-21-2007 11:14 AM

http://www.download.com/Duplicate-Te...html?tag=lst-1

:thumbsup

Dirty F

12-21-2007 11:16 AM

Quote:

Originally Posted by moogle (Post 13546930)

http://www.download.com/Duplicate-Te...html?tag=lst-1

:thumbsup

Its the same as others posted. A duplicate content remover. Its not what i need.

Zoose

12-21-2007 11:28 AM

Code:

<?php



$list1 = file( "list1.txt" );

$list2 = file( "list2.txt" );



for( $i=0; $i<count($list2); $i++ ){



if( in_array( $list2[$i], $list1 ) ){ /* do nothing */ } else { $new_array[] = $list2[$i]; }



}



echo "<pre>";

print_r( $new_array );

echo "</pre>";



?>

Dirty F

12-21-2007 11:41 AM

Quote:

Originally Posted by Zoose (Post 13547015)

Code:

<?php



$list1 = file( "list1.txt" );

$list2 = file( "list2.txt" );



for( $i=0; $i<count($list2); $i++ ){



if( in_array( $list2[$i], $list1 ) ){ /* do nothing */ } else { $new_array[] = $list2[$i]; }



}



echo "<pre>";

print_r( $new_array );

echo "</pre>";



?>

Totally appreciate it. Even more if it was 2 mins earlier before i paid someone 25 bucks for the same thing :thumbsup

UniqueD

12-21-2007 12:02 PM

paste into excel and run a macro to remove duplicates, do a google search for excel remove dupes.

All times are GMT -7. The time now is 07:51 AM.