Any perl people in the house?

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • gezzed
    Confirmed User
    • Mar 2002
    • 317
    #1

    Any perl people in the house?

    I'm trying to develop a script that searches through a simple .txt file and returns all the records matching a search query.

    This is the general format of the .txt file

    xxx----somecategory----123Mb
    yyy----someothercategory----456Mb
    zzz----onemorecategory---789Mb

    Then, someone specifies a query through my site and I need to check whether any of the records in the .txt file match the query by scanning through the xxx, yyy, zzz parts.

    Here's the snag, I have 50,000 entries in the .txt file so a simple foreach loop takes way too long.

    Any ideas?
    Tags: None
    • mortenb
      Confirmed User
      • Jul 2004
      • 2203
      #2
      The simplest solution would be to stop using a txt file and throw the data in a db..

        Comment

        • gezzed
          Confirmed User
          • Mar 2002
          • 317
          #3
          Originally posted by mortenb
          The simplest solution would be to stop using a txt file and throw the data in a db..
          I've always been a fan of minimalism. I suppose that if there is no other way I'll look into using SQL. But I'd like believe there is a way around this problem that doesn't complicate matters further.

            Comment

            • NetRodent
              Confirmed User
              • Jan 2002
              • 3985
              #4
              [QUOTE=gezzed]Here's the snag, I have 50,000 entries in the .txt file so a simple foreach loop takes way too long./QUOTE]

              Its not the foreach that's taking the time. Its the pattern matching on 50,000 entries.

              The easiest way would be with a database, properly indexed.

              If you're stuck on storing your data in text files, you could create a new set of textfiles to index what's in the main text file.
              "Every normal man must be tempted, at times, to spit on his hands, hoist the black flag, and begin slitting throats."
              --H.L. Mencken

                Comment

                • Tom_PM
                  Porn Meister
                  • Feb 2005
                  • 16443
                  #5
                  Probably with grep. It's inherently line-based and fast with regex. I dont have a sample, but that should get you on the right path I think.
                  43-922-863 Shut up and play your guitar.

                    Comment

                    • gezzed
                      Confirmed User
                      • Mar 2002
                      • 317
                      #6
                      Originally posted by PR_Tom
                      Probably with grep. It's inherently line-based and fast with regex. I dont have a sample, but that should get you on the right path I think.
                      Thanks Tom, I'll have a look into it.

                        Comment

                        • gezzed
                          Confirmed User
                          • Mar 2002
                          • 317
                          #7
                          Looks like grep is the answer. Thanks again Tom.

                            Comment

                            • sonofsam
                              Too lazy to set a custom title
                              • Dec 2004
                              • 18647
                              #8


                              sorry
                              I like turtles.

                                Comment

                                • swedguy
                                  Confirmed User
                                  • Jan 2002
                                  • 7981
                                  #9
                                  Is the xxx,yyy,zzz a set length (like always 3 chars long)?

                                    Comment

                                    • MrJackMeHoff
                                      Confirmed User
                                      • Mar 2004
                                      • 4569
                                      #10
                                      Using a %hash you could search 50k text file easy

                                        Comment

                                        • teksonline
                                          So Fucking Banned
                                          • Jan 2005
                                          • 2904
                                          #11
                                          use regular expressions

                                          hit me up if you want on aim

                                          freebsdteks

                                            Comment

                                            Previous template Next
                                            Working...