need a program to remove duplicate text - GoFuckYourself.com

galleryseek · 12-17-2003, 01:46 AM

and no, its not for email lists.

have a bunch of abbreviations n' such that are probably duplicates

thx

lagwagon · 12-17-2003, 01:48 AM

find and replace in dreamweaver

fsfaz · 12-17-2003, 01:50 AM

sed 's/was/isnow/g' file1

Shooter · 12-17-2003, 01:50 AM

Bk Replace Em

johnbosh · 12-17-2003, 01:50 AM

find and replace in homesite:D

galleryseek · 12-17-2003, 01:53 AM

Quote:

Originally posted by lagwagon
find and replace in dreamweaver

it needs to be able to search for duplicates on it's own without me specifying text.

fsfaz · 12-17-2003, 02:03 AM

Maybe give this program a try if your on Windows ...

I just did a search and found it ... haven't used it myself.

http://www.crystalsoftware.com.au/te.../standard.html

galleryseek · 12-17-2003, 02:05 AM

Quote:

Originally posted by fsfaz
Maybe give this program a try if your on Windows ...

I just did a search and found it ... haven't used it myself.

http://www.crystalsoftware.com.au/te.../standard.html

installed and errored

OzMan84 · 12-17-2003, 02:12 AM

i need something aswell as i dnt like having the same gallery descrption twice

OzMan84 · 12-17-2003, 05:48 AM

bump, any1 help?

xxxoutsourcing · 12-17-2003, 06:08 AM

Quote:

Originally posted by lagwagon
find and replace in dreamweaver

Coyote · 12-17-2003, 07:05 AM

From your descriptions, galleryseek and ozman84, it soumds like you want an application to automatically remove same words and/or phrases from two non-contiguous locations within a block of text.

Although any word processing app's spell check would remove (or flag for removal) two same contiguous words, the problem arises when you expect the app to recognize a threshold that signals a duplicate. For example,

test phrase one goes here.
...some more text goes here.
test phrase two goes here.

The 'test phrase' and 'goes here' are duplicates, but so is 'test' and 'phrase' and 'goes' and 'here'. Coding intelligence into an application to automatically recognize this threshold, plus which was intentional, would be a major undertaking. You're better off actually reading it yourself and making your own decision.

jimthefiend · 12-17-2003, 07:07 AM

http://www.abacre.com/afr/index.htm

OzMan84 · 12-17-2003, 07:17 AM

mate i cant be fucked reading 700+ gallerys thats y i wanted a program, im testing out the program that jim posted 4 me now

Coyote · 12-17-2003, 07:24 AM

Quote:

Originally posted by galleryseek

it needs to be able to search for duplicates on it's own without me specifying text.

Sorry Oz, I was addressing the automation. Most apps, like ultra edit and abacre, will replace known values across multiple files.

crash · 12-17-2003, 08:09 AM

Ultraedit

file > sort > advanced options > check the 'remove duplicates' box

12-17-2003, 01:46 AM	#1
galleryseek Confirmed User Industry Role: Join Date: Mar 2002 Posts: 8,234	need a program to remove duplicate text and no, its not for email lists. have a bunch of abbreviations n' such that are probably duplicates thx

12-17-2003, 01:48 AM	#2
lagwagon Confirmed User Join Date: Jul 2001 Location: az Posts: 8,464	find and replace in dreamweaver __________________ FTVGirls - FTVMilfs - DanielleFTV

12-17-2003, 02:12 AM	#9
OzMan84 Confirmed User Join Date: Nov 2003 Location: Australia Posts: 4,740	i need something aswell as i dnt like having the same gallery descrption twice __________________ XferClick - 300,000 UV daily and growing quickly. Majority US, EU, CA and AU traffic. Contact me for more information ICQ: 304568058

12-17-2003, 05:48 AM	#10
OzMan84 Confirmed User Join Date: Nov 2003 Location: Australia Posts: 4,740	bump, any1 help? __________________ XferClick - 300,000 UV daily and growing quickly. Majority US, EU, CA and AU traffic. Contact me for more information ICQ: 304568058

12-17-2003, 07:17 AM	#14
OzMan84 Confirmed User Join Date: Nov 2003 Location: Australia Posts: 4,740	mate i cant be fucked reading 700+ gallerys thats y i wanted a program, im testing out the program that jim posted 4 me now __________________ XferClick - 300,000 UV daily and growing quickly. Majority US, EU, CA and AU traffic. Contact me for more information ICQ: 304568058

12-17-2003, 01:50 AM	#3
fsfaz Confirmed User Join Date: Apr 2003 Location: Hollyweird, CA Posts: 747	sed 's/was/isnow/g' file1

12-17-2003, 01:50 AM	#4
Shooter Confirmed User Join Date: Apr 2001 Posts: 2,842	Bk Replace Em

12-17-2003, 01:50 AM	#5
johnbosh Confirmed User Join Date: Aug 2002 Location: The Netherlands, Rotterdam Posts: 8,965	find and replace in homesite:D

12-17-2003, 02:03 AM	#7
fsfaz Confirmed User Join Date: Apr 2003 Location: Hollyweird, CA Posts: 747	Maybe give this program a try if your on Windows ... I just did a search and found it ... haven't used it myself. http://www.crystalsoftware.com.au/te.../standard.html

12-17-2003, 07:05 AM	#12
Coyote Drinker of Scotch Industry Role: Join Date: May 2003 Location: Texas Posts: 242	From your descriptions, galleryseek and ozman84, it soumds like you want an application to automatically remove same words and/or phrases from two non-contiguous locations within a block of text. Although any word processing app's spell check would remove (or flag for removal) two same contiguous words, the problem arises when you expect the app to recognize a threshold that signals a duplicate. For example, test phrase one goes here. ...some more text goes here. test phrase two goes here. The 'test phrase' and 'goes here' are duplicates, but so is 'test' and 'phrase' and 'goes' and 'here'. Coding intelligence into an application to automatically recognize this threshold, plus which was intentional, would be a major undertaking. You're better off actually reading it yourself and making your own decision.

12-17-2003, 07:07 AM	#13
jimthefiend So Fucking Banned Join Date: Oct 2003 Location: icq: 121189 Posts: 18,889	http://www.abacre.com/afr/index.htm

12-17-2003, 08:09 AM	#16
crash Registered User Join Date: Jul 2002 Location: Sydney, Australia Posts: 97	Ultraedit file > sort > advanced options > check the 'remove duplicates' box __________________ My Adult Banners The Webs Leading Adult Banner Exchange Network