making sentances into words...

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • qwe
    Confirmed User
    • Jul 2003
    • 2109

    #1

    making sentances into words...

    is there a way to let's say load up a .txt file with a bunch of lines of text and extract all the words as a list 1 word per line ? and maybe eliminate any words larger then 2-3 characters?
    Last edited by qwe; 05-03-2009, 08:04 PM.
  • baddog
    So Fucking Banned
    • Apr 2001
    • 107089

    #2
    and maybe eliminate any words larger then 2-3 characters? That is going to make it a pretty short list.

    Comment

    • qwe
      Confirmed User
      • Jul 2003
      • 2109

      #3
      Originally posted by baddog
      and maybe eliminate any words larger then 2-3 characters? That is going to make it a pretty short list.
      not when you load up 1mb .txt file.... any know how can that be done?

      Comment

      • d-null
        . . .
        • Apr 2007
        • 13724

        #4
        this kind of problem is first year computer science stuff, shouldn't be too hard to find a script example out there

        __________________

        Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad!
        Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs
        Check out the #1 WordPress SEO Plugin: CyberSEO Suite

        Comment

        • Iron Fist
          Too lazy to set a custom title
          • Dec 2006
          • 23400

          #5
          Good luck to you
          i like waffles

          Comment

          • baddog
            So Fucking Banned
            • Apr 2001
            • 107089

            #6
            Originally posted by qwe
            not when you load up 1mb .txt file.... any know how can that be done?
            Maybe I am just burned out. Are you saying you want to get 1mb .txt file of words like

            and
            but
            or
            if
            may
            yes
            to
            too
            she
            he


            and randomize them to make sentences?

            Comment

            • qwe
              Confirmed User
              • Jul 2003
              • 2109

              #7
              Originally posted by d-null
              this kind of problem is first year computer science stuff, shouldn't be too hard to find a script example out there
              lol i know, but don't know jack about programming

              Comment

              • qwe
                Confirmed User
                • Jul 2003
                • 2109

                #8
                Originally posted by baddog
                Maybe I am just burned out. Are you saying you want to get 1mb .txt file of words like

                and
                but
                or
                if
                may
                yes
                to
                too
                she
                he


                and randomize them to make sentences?
                no, other way around.... you have a bunch of sentences and you want to make them into 1 word per line

                Comment

                • d-null
                  . . .
                  • Apr 2007
                  • 13724

                  #9
                  Originally posted by qwe
                  no, other way around.... you have a bunch of sentences and you want to make them into 1 word per line
                  so no count necessary, just extract all unique words with 1, 2, or 3 characters and list them in a text file one word per line?

                  __________________

                  Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad!
                  Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs
                  Check out the #1 WordPress SEO Plugin: CyberSEO Suite

                  Comment

                  • qwe
                    Confirmed User
                    • Jul 2003
                    • 2109

                    #10
                    Originally posted by d-null
                    so no count necessary, just extract all unique words with 1, 2, or 3 characters and list them in a text file one word per line?
                    yes but all unique words EXCEPT 1/2/3 characters
                    Last edited by qwe; 05-03-2009, 08:22 PM.

                    Comment

                    • ProG
                      Confirmed User
                      • Apr 2009
                      • 1319

                      #11
                      PHP Code:
                      <?php
                      if ( ( $file = file( "file.txt" ) ) !== false )
                      {
                          foreach( $file as $line )
                          {
                              $words = explode( " ", trim( $line ) );
                              foreach( $words as $word )
                              {
                                  if ( strlen( $word ) > 3 )
                                  {
                                      echo $word . "\n";
                                  }
                              }
                          }
                      }
                      ?>
                      History will be kind to me for I intend to write it.

                      Comment

                      • qwe
                        Confirmed User
                        • Jul 2003
                        • 2109

                        #12
                        Originally posted by ProG
                        PHP Code:
                        <?php
                        if ( ( $file = file( "file.txt" ) ) !== false )
                        {
                            foreach( $file as $line )
                            {
                                $words = explode( " ", trim( $line ) );
                                foreach( $words as $word )
                                {
                                    if ( strlen( $word ) > 3 )
                                    {
                                        echo $word . "\n";
                                    }
                                }
                            }
                        }
                        ?>
                        thanks, is there anyway I can run it in windows ? sorry for stupid questions

                        Comment

                        • ProG
                          Confirmed User
                          • Apr 2009
                          • 1319

                          #13
                          Originally posted by qwe
                          thanks, is there anyway I can run it in windows ? sorry for stupid questions
                          WampServer.com
                          History will be kind to me for I intend to write it.

                          Comment

                          • qwe
                            Confirmed User
                            • Jul 2003
                            • 2109

                            #14
                            Originally posted by ProG
                            WampServer.com
                            sorry, i installed it and what do I do next ? i put that test.php into C:\wamp\www and go to http://localhost/phpmyadmin/test.php and page is blank.... i'm using windows7 64bit maybe thats an issue?

                            Comment

                            • ProG
                              Confirmed User
                              • Apr 2009
                              • 1319

                              #15
                              /www is the root directory which means it would be localhost/test.php
                              History will be kind to me for I intend to write it.

                              Comment

                              • TidalWave
                                Confirmed User
                                • Sep 2007
                                • 2706

                                #16
                                edit the filename to the one of the txt you already have.
                                make sure the file is in the same directory as your php script.
                                www.SwiftNode.com

                                Comment

                                • qwe
                                  Confirmed User
                                  • Jul 2003
                                  • 2109

                                  #17
                                  Originally posted by ProG
                                  /www is the root directory which means it would be localhost/test.php
                                  yah I tried that... i put test.php and test.txt into www folder under C:\wamp\www and when i go to http://localhost/test.php nothing happens just a blank white page in firefox, in explorer says page can't be found... says wampserver is online, it also has index.php in the www folder as well (it came with it) and that page doesn't load either hrmmmm

                                  Comment

                                  • qwe
                                    Confirmed User
                                    • Jul 2003
                                    • 2109

                                    #18
                                    Originally posted by TidalWave
                                    edit the filename to the one of the txt you already have.
                                    make sure the file is in the same directory as your php script.
                                    yap did that, edited it with test.txt inside test.php

                                    Comment

                                    • ProG
                                      Confirmed User
                                      • Apr 2009
                                      • 1319

                                      #19
                                      Is the wampserver icon in the system tray all white? You only need Apache running. Click the icon, goto Apache, Restart Service
                                      History will be kind to me for I intend to write it.

                                      Comment

                                      • qwe
                                        Confirmed User
                                        • Jul 2003
                                        • 2109

                                        #20
                                        Originally posted by ProG
                                        Is the wampserver icon in the system tray all white?
                                        no, it's yellow but if i move mouse over it, it says server online

                                        Comment

                                        • qwe
                                          Confirmed User
                                          • Jul 2003
                                          • 2109

                                          #21
                                          Originally posted by ProG
                                          Is the wampserver icon in the system tray all white? You only need Apache running. Click the icon, goto Apache, Restart Service
                                          ok figured it out, skype was using port 80, ok apache started and looks like it tried to execute your code, and came up with an error "Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 35 bytes) in C:\wamp\www\test.php on line 2"

                                          Comment

                                          • ProG
                                            Confirmed User
                                            • Apr 2009
                                            • 1319

                                            #22
                                            Click wampserver icon, goto Config Files, goto php.ini

                                            Search for "memory_limit" and change the value from 8M to 128M

                                            (You may have to restart the Apache service again)
                                            History will be kind to me for I intend to write it.

                                            Comment

                                            • qwe
                                              Confirmed User
                                              • Jul 2003
                                              • 2109

                                              #23
                                              Originally posted by ProG
                                              Click wampserver icon, goto Config Files, goto php.ini

                                              Search for "memory_limit" and change the value from 8M to 128M

                                              (You may have to restart the Apache service again)
                                              ok it was already at 128mb, anyways i made file smaller and it worked but all the words are on the same line, not 1 word per line... also they include dots, comas and " ' anyway to get rid of them? here's what it displays now:

                                              morning, stay same room keep watch shifts, never more than three asleep time. Hirschel asked. believe. doesn't have windows. angry Cyr's put-down, Jubal said, windows? What does that matter? killer said. still ruling other possibilities. Besides looking each you, don't want have guard windows. Hirschel said,

                                              suppose to be :

                                              morning
                                              stay
                                              same
                                              room
                                              etc
                                              etc
                                              Last edited by qwe; 05-03-2009, 09:10 PM.

                                              Comment

                                              • ProG
                                                Confirmed User
                                                • Apr 2009
                                                • 1319

                                                #24
                                                If you right click and View Source it will not be on one line. Otherwise, replace "\n" with "<br>" for html output.

                                                Revised code to remove extra chars.

                                                PHP Code:
                                                <?php
                                                if ( ( $file = file( "file.txt" ) ) !== false )
                                                {
                                                    foreach( $file as $line )
                                                    {
                                                        $words = explode( " ", trim( $line ) );
                                                        foreach( $words as $word )
                                                        {
                                                            $word = ereg_replace( "[^A-Za-z0-9]", "", $word );
                                                            if ( strlen( $word ) > 3 )
                                                            {
                                                                echo $word . "\n";
                                                            }
                                                        }
                                                    }
                                                }
                                                ?>
                                                History will be kind to me for I intend to write it.

                                                Comment

                                                • qwe
                                                  Confirmed User
                                                  • Jul 2003
                                                  • 2109

                                                  #25
                                                  everything worked.... thanks for your help man you rock

                                                  Comment

                                                  • qwe
                                                    Confirmed User
                                                    • Jul 2003
                                                    • 2109

                                                    #26
                                                    btw do you have icq/skype? i might need more small scripts (next one i wont be asking for free )

                                                    Comment

                                                    • ProG
                                                      Confirmed User
                                                      • Apr 2009
                                                      • 1319

                                                      #27
                                                      np.

                                                      Just post your problems on GFY, it gives some of us something to do besides look at boobies on an "adult" message board.
                                                      History will be kind to me for I intend to write it.

                                                      Comment

                                                      • ProG
                                                        Confirmed User
                                                        • Apr 2009
                                                        • 1319

                                                        #28
                                                        One more revision that will give you a listing of unique words (no duplicates).

                                                        PHP Code:
                                                        <?php
                                                        $unique_words = array( );
                                                        if ( ( $file = file( "file.txt" ) ) !== false )
                                                        {
                                                            foreach( $file as $line )
                                                            {
                                                                $words = explode( " ", trim( $line ) );
                                                                foreach( $words as $word )
                                                                {
                                                                    $word = ereg_replace( "[^A-Za-z0-9]", "", $word );
                                                                    if ( strlen( $word ) > 3 )
                                                                    {
                                                                        if ( !in_array( $word, $unique_words ) )
                                                                        {
                                                                            $unique_words[] = $word;
                                                                        }
                                                                    }
                                                                }
                                                            }
                                                        }
                                                        foreach( $unique_words as $word )
                                                        {
                                                            echo $word . "\n";
                                                        }
                                                        ?>
                                                        Enjoy. Now that you have WampServer, head over to PHP.net, read the documentation and learn something
                                                        History will be kind to me for I intend to write it.

                                                        Comment

                                                        • qwe
                                                          Confirmed User
                                                          • Jul 2003
                                                          • 2109

                                                          #29
                                                          Originally posted by ProG
                                                          np.

                                                          Just post your problems on GFY, it gives some of us something to do besides look at boobies on an "adult" message board.
                                                          hehe yah you have a point there damn this i7 with 6gb of memory is killing huge txt files

                                                          Comment

                                                          • qwe
                                                            Confirmed User
                                                            • Jul 2003
                                                            • 2109

                                                            #30
                                                            btw that latest code you gave gives me this error "Fatal error: Maximum execution time of 30 seconds exceeded in C:\wamp\www\test3.php on line 13" maybe file too big ?

                                                            Comment

                                                            • qwe
                                                              Confirmed User
                                                              • Jul 2003
                                                              • 2109

                                                              #31
                                                              yah, i down sized file like 10x smaller, and it worked....

                                                              Comment

                                                              • ProG
                                                                Confirmed User
                                                                • Apr 2009
                                                                • 1319

                                                                #32
                                                                You know, if you have 6gb of RAM you can use it all. Change the memory_limit to 512MB or 1024MB.
                                                                History will be kind to me for I intend to write it.

                                                                Comment

                                                                • qwe
                                                                  Confirmed User
                                                                  • Jul 2003
                                                                  • 2109

                                                                  #33
                                                                  Originally posted by ProG
                                                                  You know, if you have 6gb of RAM you can use it all. Change the memory_limit to 512MB or 1024MB.
                                                                  cool, i also switched 30seconds to 200 seconds for script execution ;>

                                                                  Comment

                                                                  • ProG
                                                                    Confirmed User
                                                                    • Apr 2009
                                                                    • 1319

                                                                    #34
                                                                    Yes, that too. I usually set my localhost to 300 seconds (5 minutes).
                                                                    History will be kind to me for I intend to write it.

                                                                    Comment

                                                                    • qwe
                                                                      Confirmed User
                                                                      • Jul 2003
                                                                      • 2109

                                                                      #35
                                                                      Originally posted by ProG
                                                                      Yes, that too. I usually set my localhost to 300 seconds (5 minutes).
                                                                      sweet thanks again

                                                                      Comment

                                                                      • qwe
                                                                        Confirmed User
                                                                        • Jul 2003
                                                                        • 2109

                                                                        #36
                                                                        one more thing, in php.ini should it be

                                                                        memory_limit = 1024M OR memory_limit = 1024

                                                                        does it need to have M at the end ? thx

                                                                        Comment

                                                                        • ProG
                                                                          Confirmed User
                                                                          • Apr 2009
                                                                          • 1319

                                                                          #37
                                                                          I am not 100% sure but I believe it works on a system like "100K", "100M", "100G", so the "M" would be necessary. Sorry I'm a programmer not a system admin.
                                                                          History will be kind to me for I intend to write it.

                                                                          Comment

                                                                          • qwe
                                                                            Confirmed User
                                                                            • Jul 2003
                                                                            • 2109

                                                                            #38
                                                                            Originally posted by ProG
                                                                            I am not 100% sure but I believe it works on a system like "100K", "100M", "100G", so the "M" would be necessary. Sorry I'm a programmer not a system admin.
                                                                            gotchya, thanks

                                                                            Comment

                                                                            • qwe
                                                                              Confirmed User
                                                                              • Jul 2003
                                                                              • 2109

                                                                              #39
                                                                              hey ProG, anyway you can make me another simple script? to check for any empty lines and lines that start with 0-9 or some weird symbols (such as -,---,===,%---, etc, etc) and remove those lines ? basically to remove any empty lines and lines that start with anything other then a valid character (a-z or A-Z)
                                                                              Last edited by qwe; 05-03-2009, 10:52 PM.

                                                                              Comment

                                                                              • d-null
                                                                                . . .
                                                                                • Apr 2007
                                                                                • 13724

                                                                                #40
                                                                                Originally posted by qwe
                                                                                hey ProG, anyway you can make me another simple script? to check for any empty lines and lines that start with 0-9 or some weird symbols (such as -,---,===,%---, etc, etc) and remove those lines ? basically to remove any empty lines and lines that start with anything other then a valid character (a-z or A-Z)

                                                                                if I understand what you are asking for, this will do it nicely:

                                                                                Code:
                                                                                <?php 
                                                                                if ( ( $file = file( "file.txt" ) ) !== false ) 
                                                                                { 
                                                                                    foreach( $file as $line ) 
                                                                                    { 
                                                                                        if (ctype_alpha($line{0}))
                                                                                		
                                                                                        { 
                                                                                              echo $line . "<br>"; 
                                                                                             
                                                                                        } 
                                                                                    } 
                                                                                } 
                                                                                ?>

                                                                                __________________

                                                                                Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad!
                                                                                Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs
                                                                                Check out the #1 WordPress SEO Plugin: CyberSEO Suite

                                                                                Comment

                                                                                • d-null
                                                                                  . . .
                                                                                  • Apr 2007
                                                                                  • 13724

                                                                                  #41
                                                                                  also note that I used <br> for html output instead of \n so if you need it for your text file use then you should change that part

                                                                                  __________________

                                                                                  Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad!
                                                                                  Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs
                                                                                  Check out the #1 WordPress SEO Plugin: CyberSEO Suite

                                                                                  Comment

                                                                                  • ExLust
                                                                                    Confirmed User
                                                                                    • Aug 2008
                                                                                    • 3223

                                                                                    #42
                                                                                    Good luck!

                                                                                    BE A PARTNER

                                                                                    Comment

                                                                                    • voa
                                                                                      Too lazy to set a custom title
                                                                                      • Nov 2006
                                                                                      • 16532

                                                                                      #43
                                                                                      im not sure that something like that is exist

                                                                                      Comment

                                                                                      • d-null
                                                                                        . . .
                                                                                        • Apr 2007
                                                                                        • 13724

                                                                                        #44
                                                                                        Originally posted by voa
                                                                                        im not sure that something like that is exist

                                                                                        __________________

                                                                                        Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad!
                                                                                        Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs
                                                                                        Check out the #1 WordPress SEO Plugin: CyberSEO Suite

                                                                                        Comment

                                                                                        • Scootermuze
                                                                                          Confirmed User
                                                                                          • Dec 2001
                                                                                          • 4513

                                                                                          #45
                                                                                          Sentances?

                                                                                          Might wanna incorporate a spell checker into this thing too..

                                                                                          Sorry.. had to be said..

                                                                                          Comment

                                                                                          • d-null
                                                                                            . . .
                                                                                            • Apr 2007
                                                                                            • 13724

                                                                                            #46
                                                                                            Originally posted by Scootermuze
                                                                                            Sentances?
                                                                                            Might wanna incorporate a spell checker into this thing too..

                                                                                            __________________

                                                                                            Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad!
                                                                                            Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs
                                                                                            Check out the #1 WordPress SEO Plugin: CyberSEO Suite

                                                                                            Comment

                                                                                            • seeandsee
                                                                                              Check SIG!
                                                                                              • Mar 2006
                                                                                              • 50945

                                                                                              #47
                                                                                              learn word and will be easy
                                                                                              BUY MY SIG - 50$/Year

                                                                                              Contact here

                                                                                              Comment

                                                                                              • Killswitch - BANNED FOR LIFE

                                                                                                #48
                                                                                                Originally posted by voa
                                                                                                im not sure that something like that is exist
                                                                                                Did you read the thread, or just post and leave?

                                                                                                Comment

                                                                                                • d-null
                                                                                                  . . .
                                                                                                  • Apr 2007
                                                                                                  • 13724

                                                                                                  #49
                                                                                                  Originally posted by Killswitch
                                                                                                  Did you read the thread, or just post and leave?
                                                                                                  gfy should have some kind of penalty system set up for stuff like that, like he should lose 25% of his post count for second offence or something like that

                                                                                                  __________________

                                                                                                  Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad!
                                                                                                  Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs
                                                                                                  Check out the #1 WordPress SEO Plugin: CyberSEO Suite

                                                                                                  Comment

                                                                                                  • Cyber Fucker
                                                                                                    Hmm
                                                                                                    • Sep 2005
                                                                                                    • 12642

                                                                                                    #50
                                                                                                    Originally posted by Killswitch
                                                                                                    Did you read the thread, or just post and leave?
                                                                                                    ...or maybe you are bot

                                                                                                    Comment

                                                                                                    Working...