Originally Posted by rickholio

I'd suggest that the easiest way to do it is through a temporary table... do a SELECT DISTINCT into the temp table to eliminate the duplicates.

Alternately, a cute way to do a de-facto culling is to ALTER TABLE and make your url column UNIQUE, ignoring (and therefore dropping) duplicates:

ALTER IGNORE TABLE sometable ADD UNIQUE (url)

... which would serve the additional purpose of permanently removing the possibility of duplicate URLs in the future, as well.

Obviously you'll want to back up everything before you try any of this.