I have 2 text files from which i need to remove duplicates - GoFuckYourself.com

Dirty F · 12-21-2007, 10:38 AM

I have 2 text files with names.
One is a big list with xxxx amount of names and the text file is a small list with xxx amount of names. The smaller list names also appear on the bigger list. Now i need a way to compare both list and remove ALL the names that appear on both lists.

Know what i mean?

Lace · 12-21-2007, 10:55 AM

get a list manager...

Voodoo · 12-21-2007, 10:56 AM

http://www.flashpeak.com/slimlist/slimlist.htm

Dirty F · 12-21-2007, 10:56 AM

I have list managers but they can only remove the duplicates and not the original name as well. I want any duplicate name to be gone. So it doesnt appear at all on the list anymore.

Voodoo · 12-21-2007, 10:58 AM

SlimList will remove duplicate entries, but only the duplicate.

Are you looking to do something like this:
LIST A: name1 name2 name3 name4
LIST B: name1 name5 name6 name7

ACTION: remove name1 from both lists.

Is that what you're talking about? Or simply removing duplicates?

ne0 · 12-21-2007, 10:59 AM

Quote:

Originally Posted by Dirty F

I have 2 text files with names.
One is a big list with xxxx amount of names and the text file is a small list with xxx amount of names. The smaller list names also appear on the bigger list. Now i need a way to compare both list and remove ALL the names that appear on both lists.

Know what i mean?

if the names are one per line, and they appear the same way on the first and the second list, you can do this using a *nix server:
cat list1.txt list2.txt |sort |uniq > list3.txt
the list3.txt will contain both lists, ordered and without the dupes.

Lace · 12-21-2007, 11:00 AM

Or you could throw it in a XLS, sort and remove by hand.

Dirty F · 12-21-2007, 11:01 AM

Quote:

Originally Posted by Voodoo

http://www.flashpeak.com/slimlist/slimlist.htm

I dont see how this can do what i need.

Dirty F · 12-21-2007, 11:03 AM

Quote:

Originally Posted by Lace

Or you could throw it in a XLS, sort and remove by hand.

If i would want to do it by hand i wouldnt ask for a solution here

moogle · 12-21-2007, 11:14 AM

http://www.download.com/Duplicate-Te...html?tag=lst-1

Dirty F · 12-21-2007, 11:16 AM

Quote:

Originally Posted by moogle

http://www.download.com/Duplicate-Te...html?tag=lst-1

Its the same as others posted. A duplicate content remover. Its not what i need.

Zoose · 12-21-2007, 11:28 AM

Code:

<?php

$list1 = file( "list1.txt" );
$list2 = file( "list2.txt" );

for( $i=0; $i<count($list2); $i++ ){

if( in_array( $list2[$i], $list1 ) ){ /* do nothing */ } else { $new_array[] = $list2[$i]; }

}

echo "<pre>";
print_r( $new_array );
echo "</pre>";

?>

Dirty F · 12-21-2007, 11:41 AM

Quote:

Originally Posted by Zoose

Code:

<?php

$list1 = file( "list1.txt" );
$list2 = file( "list2.txt" );

for( $i=0; $i<count($list2); $i++ ){

if( in_array( $list2[$i], $list1 ) ){ /* do nothing */ } else { $new_array[] = $list2[$i]; }

}

echo "<pre>";
print_r( $new_array );
echo "</pre>";

?>

Totally appreciate it. Even more if it was 2 mins earlier before i paid someone 25 bucks for the same thing

UniqueD · 12-21-2007, 12:02 PM

paste into excel and run a macro to remove duplicates, do a google search for excel remove dupes.

12-21-2007, 10:38 AM	#1
Dirty F Too lazy to set a custom title Industry Role: Join Date: Jul 2001 Posts: 59,204	I have 2 text files from which i need to remove duplicates I have 2 text files with names. One is a big list with xxxx amount of names and the text file is a small list with xxx amount of names. The smaller list names also appear on the bigger list. Now i need a way to compare both list and remove ALL the names that appear on both lists. Know what i mean?

12-21-2007, 10:55 AM	#2
Lace Too lazy to set a custom title Industry Role: Join Date: Mar 2004 Posts: 16,116	get a list manager... __________________ Your Paysite Partner Strength In Numbers! StickyDollars \| RadicalCash \| KennysPennies \| HomegrownCash

12-21-2007, 10:56 AM	#3
Voodoo ♥ ♦ ♣ ♠ Industry Role: Join Date: Sep 2002 Posts: 10,592	http://www.flashpeak.com/slimlist/slimlist.htm __________________ "I'm selflessly supporting the common good, but only coincidentally looking out for No.1."

12-21-2007, 10:58 AM	#5
Voodoo ♥ ♦ ♣ ♠ Industry Role: Join Date: Sep 2002 Posts: 10,592	SlimList will remove duplicate entries, but only the duplicate. Are you looking to do something like this: LIST A: name1 name2 name3 name4 LIST B: name1 name5 name6 name7 ACTION: remove name1 from both lists. Is that what you're talking about? Or simply removing duplicates? __________________ "I'm selflessly supporting the common good, but only coincidentally looking out for No.1."

12-21-2007, 11:00 AM	#7
Lace Too lazy to set a custom title Industry Role: Join Date: Mar 2004 Posts: 16,116	Or you could throw it in a XLS, sort and remove by hand. __________________ Your Paysite Partner Strength In Numbers! StickyDollars \| RadicalCash \| KennysPennies \| HomegrownCash

12-21-2007, 11:14 AM	#10
moogle Confirmed User Join Date: Sep 2007 Location: Somewhere over the rainbow.... ICQ-474167022 Posts: 314	http://www.download.com/Duplicate-Te...html?tag=lst-1 __________________

12-21-2007, 12:02 PM	#14
UniqueD Confirmed User Industry Role: Join Date: Aug 2004 Location: Barcelona Posts: 1,022	paste into excel and run a macro to remove duplicates, do a google search for excel remove dupes.