2 txt files, how to remove doubles?

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • justsexxx
    Too lazy to set a custom title
    • Aug 2001
    • 13723

    #1

    2 txt files, how to remove doubles?

    I have 2 huge txt files with galleries. One with valid url's, and one with url's which don't work anymore, cheated etc.

    SInce they contain over 1000 url's each, it is a lot of work to find what are double etc.

    Is there an easy way to find which url's are double. They should be removed from the "good" list.

    so good.txt bad.txt and url's listed on bad.txt should be removed from good.txt in case they are listed there.

    How todo this quick and easy?

    Andre
    Questions?

    ICQ: 125184542
  • Basic_man
    Programming King Pin
    • Oct 2003
    • 27360

    #2
    Any basic programmers out there can do this very quick!
    UUGallery Builder - automated photo/video gallery plugin for Wordpress!
    Stop looking! Checkout Naked Hosting, online since 1999 !

    Comment

    • justsexxx
      Too lazy to set a custom title
      • Aug 2001
      • 13723

      #3
      I need one that is ready to use. Money is not an issue
      Questions?

      ICQ: 125184542

      Comment

      • Dirty F
        Too lazy to set a custom title
        • Jul 2001
        • 59204

        #4
        I'll help ya, check icq.

        Comment

        • justsexxx
          Too lazy to set a custom title
          • Aug 2001
          • 13723

          #5
          Originally posted by Battuss
          I'll help ya, check icq.
          Thanks for your help buddy

          Andre
          Questions?

          ICQ: 125184542

          Comment

          • JulianSosa
            Confirmed User
            • Aug 2003
            • 3042

            #6
            Originally posted by justsexxx
            I have 2 huge txt files with galleries. One with valid url's, and one with url's which don't work anymore, cheated etc.

            SInce they contain over 1000 url's each, it is a lot of work to find what are double etc.

            Is there an easy way to find which url's are double. They should be removed from the "good" list.

            so good.txt bad.txt and url's listed on bad.txt should be removed from good.txt in case they are listed there.

            How todo this quick and easy?

            Andre

            I think this works

            cat old.txt | uniq > new.txt

            Comment

            • Trixxxia
              Confirmed User
              • Aug 2004
              • 5600

              #7
              Originally posted by justsexxx
              Thanks for your help buddy

              Andre
              Justsexxx - let me know if you got what you need.

              I have a macro in excel to do that for me.

              Comment

              • rowan
                Too lazy to set a custom title
                • Mar 2002
                • 17393

                #8
                To remove the entries in bad.txt from good.txt:

                cat good.txt | fgrep -vf bad.txt > newgood.txt

                I think that should do the trick.

                Comment

                Working...