mailing list cleaner to rid duplicate emails **no pics inside**
 

Help a girl out!! :winkwink:

Anyone heard of a software or script that would go through tons of txt or csv files I have with email addresses and combine them into one that takes out duplicates?

I have over 30,000 emails that I've collected over the years from different sites of mine and I want to combine them all into a new email list software that I have.

Any help is appreciated. :thumbsup

Excel can do it. Just import your data, sort, remove dupes.

You're likely stuck at a max of 65,000 records per sheet, though.

how can I remove dups?

I dont see anything to do that in the menus.

ok I found this of the Microcrap site:

1. Select all the rows, including the column headers, in the list you want to filter.

ShowTip
Click the top left cell of the range, and then drag to the bottom right cell.
2. On the Data menu, point to Filter, and then click Advanced Filter.
3. In the Advanced Filter dialog box, click Filter the list, in place.
4. Select the Unique records only check box, and then click OK.

The filtered list is displayed and the duplicate rows are hidden.
5. On the Edit menu, click Office Clipboard.

The Clipboard task pane is displayed.
6. Make sure the filtered list is still selected, and then click Copy Copy button.

The filtered list is highlighted with bounding outlines and the selection appears as an item at the top of the Clipboard.
7. On the Data menu, point to Filter, and then click Show All.

The original list is re-displayed.
8. Press the DELETE key.

The original list is deleted.
9. In the Clipboard, click on the filtered list item.

The filtered list appears in the same location as the original list.

Put them all into one file and then in Linux or Cygwin:

cat file.txt | sort | uniq > new_file.txt

:)

(this would only work if you have an e-mail per line and nothing else, so if you have names, opt-in data, etc then it wouldn't work as easily)

GrouchyAdmin hey thx

Linguist whoa whoa now that's crazy. Who runs linux on the desktop.

now that's white and nerdy!

http://cygwin.com ;)

No problem.

Then you've got bigger issues. Some native Cygwin ports do/don't automatically distinct case between CR (unix) and CR+LF (dos). Not a good plan. You'd be better off using Perl than even awk in this case. :Oh crap

Just tried making a list of e-mails in Notepad out of curiosity and it works here *shrug*

I'd expect that utilities whose job is to work with lines would be ported properly (like sort, cat, etc)

Linux and VMware on a desktop - great combination!

now is there a way to combine a bunch of csv files into one file and then I can run the filter tool.

eListPro .. oldschool :thumbsup

I'll check that out....

later that night...

Wow you were right. It combines all my lists into 1 file, cleans them of malformed emails, filters out domains I don't want and then sorts by domain or randomizes.

13k clean emails in 1 list!!

Who's buying?? :)