making sentances into words... - GoFuckYourself.com

qwe · 05-03-2009, 09:03 PM

is there a way to let's say load up a .txt file with a bunch of lines of text and extract all the words as a list 1 word per line ? and maybe eliminate any words larger then 2-3 characters?

baddog · 05-03-2009, 09:07 PM

and maybe eliminate any words larger then 2-3 characters? That is going to make it a pretty short list.

qwe · 05-03-2009, 09:11 PM

Quote:

Originally Posted by baddog

and maybe eliminate any words larger then 2-3 characters? That is going to make it a pretty short list.

not when you load up 1mb .txt file.... any know how can that be done?

d-null · 05-03-2009, 09:13 PM

this kind of problem is first year computer science stuff, shouldn't be too hard to find a script example out there

Iron Fist · 05-03-2009, 09:14 PM

Good luck to you

baddog · 05-03-2009, 09:15 PM

Quote:

Originally Posted by qwe

not when you load up 1mb .txt file.... any know how can that be done?

Maybe I am just burned out. Are you saying you want to get 1mb .txt file of words like

and
but
or
if
may
yes
to
too
she
he

and randomize them to make sentences?

qwe · 05-03-2009, 09:15 PM

Quote:

Originally Posted by d-null

this kind of problem is first year computer science stuff, shouldn't be too hard to find a script example out there

lol i know, but don't know jack about programming

qwe · 05-03-2009, 09:16 PM

Quote:

Originally Posted by baddog

Maybe I am just burned out. Are you saying you want to get 1mb .txt file of words like

and
but
or
if
may
yes
to
too
she
he

and randomize them to make sentences?

no, other way around.... you have a bunch of sentences and you want to make them into 1 word per line

d-null · 05-03-2009, 09:18 PM

Quote:

Originally Posted by qwe

no, other way around.... you have a bunch of sentences and you want to make them into 1 word per line

so no count necessary, just extract all unique words with 1, 2, or 3 characters and list them in a text file one word per line?

qwe · 05-03-2009, 09:20 PM

Quote:

Originally Posted by d-null

so no count necessary, just extract all unique words with 1, 2, or 3 characters and list them in a text file one word per line?

yes but all unique words EXCEPT 1/2/3 characters

ProG · 05-03-2009, 09:26 PM

PHP Code:


			
<?php

if ( ( $file = file( "file.txt" ) ) !== false )

{

    foreach( $file as $line )

    {

        $words = explode( " ", trim( $line ) );

        foreach( $words as $word )

        {

            if ( strlen( $word ) > 3 )

            {

                echo $word . "\n";

            }

        }

    }

}

?>

qwe · 05-03-2009, 09:31 PM

Quote:

Originally Posted by ProG

PHP Code:


			
<?php

if ( ( $file = file( "file.txt" ) ) !== false )

{

    foreach( $file as $line )

    {

        $words = explode( " ", trim( $line ) );

        foreach( $words as $word )

        {

            if ( strlen( $word ) > 3 )

            {

                echo $word . "\n";

            }

        }

    }

}

?>

thanks, is there anyway I can run it in windows ? sorry for stupid questions

ProG · 05-03-2009, 09:32 PM

Quote:

Originally Posted by qwe

thanks, is there anyway I can run it in windows ? sorry for stupid questions

WampServer.com

qwe · 05-03-2009, 09:45 PM

Quote:

Originally Posted by ProG

WampServer.com

sorry, i installed it and what do I do next ? i put that test.php into C:\wamp\www and go to http://localhost/phpmyadmin/test.php and page is blank....

i'm using windows7 64bit maybe thats an issue?

ProG · 05-03-2009, 09:49 PM

/www is the root directory which means it would be localhost/test.php

TidalWave · 05-03-2009, 09:51 PM

edit the filename to the one of the txt you already have.
make sure the file is in the same directory as your php script.

qwe · 05-03-2009, 09:57 PM

Quote:

Originally Posted by ProG

/www is the root directory which means it would be localhost/test.php

yah I tried that... i put test.php and test.txt into www folder under C:\wamp\www and when i go to http://localhost/test.php nothing happens just a blank white page in firefox, in explorer says page can't be found... says wampserver is online, it also has index.php in the www folder as well (it came with it) and that page doesn't load either hrmmmm

qwe · 05-03-2009, 09:58 PM

Quote:

Originally Posted by TidalWave

edit the filename to the one of the txt you already have.
make sure the file is in the same directory as your php script.

yap did that, edited it with test.txt inside test.php

ProG · 05-03-2009, 10:00 PM

Is the wampserver icon in the system tray all white? You only need Apache running. Click the icon, goto Apache, Restart Service

qwe · 05-03-2009, 10:01 PM

Quote:

Originally Posted by ProG

Is the wampserver icon in the system tray all white?

no, it's yellow but if i move mouse over it, it says server online

qwe · 05-03-2009, 10:04 PM

Quote:

Originally Posted by ProG

Is the wampserver icon in the system tray all white? You only need Apache running. Click the icon, goto Apache, Restart Service

ok figured it out, skype was using port 80, ok apache started and looks like it tried to execute your code, and came up with an error "Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 35 bytes) in C:\wamp\www\test.php on line 2"

ProG · 05-03-2009, 10:06 PM

Click wampserver icon, goto Config Files, goto php.ini

Search for "memory_limit" and change the value from 8M to 128M

(You may have to restart the Apache service again)

qwe · 05-03-2009, 10:09 PM

Quote:

Originally Posted by ProG

Click wampserver icon, goto Config Files, goto php.ini

Search for "memory_limit" and change the value from 8M to 128M

(You may have to restart the Apache service again)

ok it was already at 128mb, anyways i made file smaller and it worked but all the words are on the same line, not 1 word per line... also they include dots, comas and " ' anyway to get rid of them? here's what it displays now:

morning, stay same room keep watch shifts, never more than three asleep time. Hirschel asked. believe. doesn't have windows. angry Cyr's put-down, Jubal said, windows? What does that matter? killer said. still ruling other possibilities. Besides looking each you, don't want have guard windows. Hirschel said,

suppose to be :

morning
stay
same
room
etc
etc

ProG · 05-03-2009, 10:15 PM

If you right click and View Source it will not be on one line. Otherwise, replace "\n" with "<br>" for html output.

Revised code to remove extra chars.

PHP Code:


			
<?php

if ( ( $file = file( "file.txt" ) ) !== false )

{

    foreach( $file as $line )

    {

        $words = explode( " ", trim( $line ) );

        foreach( $words as $word )

        {

            $word = ereg_replace( "[^A-Za-z0-9]", "", $word );

            if ( strlen( $word ) > 3 )

            {

                echo $word . "\n";

            }

        }

    }

}

?>

qwe · 05-03-2009, 10:18 PM

everything worked.... thanks for your help man you rock

qwe · 05-03-2009, 10:22 PM

btw do you have icq/skype? i might need more small scripts (next one i wont be asking for free

)

ProG · 05-03-2009, 10:25 PM

np.

Just post your problems on GFY, it gives some of us something to do besides look at boobies on an "adult" message board.

ProG · 05-03-2009, 10:32 PM

One more revision that will give you a listing of unique words (no duplicates).

PHP Code:


			
<?php
$unique_words = array( );
if ( ( $file = file( "file.txt" ) ) !== false )
{
    foreach( $file as $line )
    {
        $words = explode( " ", trim( $line ) );
        foreach( $words as $word )
        {
            $word = ereg_replace( "[^A-Za-z0-9]", "", $word );
            if ( strlen( $word ) > 3 )
            {
                if ( !in_array( $word, $unique_words ) )
                {
                    $unique_words[] = $word;
                }
            }
        }
    }
}
foreach( $unique_words as $word )
{
    echo $word . "\n";
}
?>

Enjoy. Now that you have WampServer, head over to PHP.net, read the documentation and learn something

qwe · 05-03-2009, 10:33 PM

Quote:

Originally Posted by ProG

np.

Just post your problems on GFY, it gives some of us something to do besides look at boobies on an "adult" message board.

hehe yah you have a point there

damn this i7 with 6gb of memory is killing huge txt files

qwe · 05-03-2009, 10:37 PM

btw that latest code you gave gives me this error "Fatal error: Maximum execution time of 30 seconds exceeded in C:\wamp\www\test3.php on line 13" maybe file too big ?

qwe · 05-03-2009, 10:41 PM

yah, i down sized file like 10x smaller, and it worked....

ProG · 05-03-2009, 10:46 PM

You know, if you have 6gb of RAM you can use it all. Change the memory_limit to 512MB or 1024MB.

qwe · 05-03-2009, 10:48 PM

Quote:

Originally Posted by ProG

You know, if you have 6gb of RAM you can use it all. Change the memory_limit to 512MB or 1024MB.

cool, i also switched 30seconds to 200 seconds for script execution ;>

ProG · 05-03-2009, 10:49 PM

Yes, that too. I usually set my localhost to 300 seconds (5 minutes).

qwe · 05-03-2009, 10:53 PM

Quote:

Originally Posted by ProG

Yes, that too. I usually set my localhost to 300 seconds (5 minutes).

sweet thanks again

qwe · 05-03-2009, 11:02 PM

one more thing, in php.ini should it be

memory_limit = 1024M OR memory_limit = 1024

does it need to have M at the end ? thx

ProG · 05-03-2009, 11:08 PM

I am not 100% sure but I believe it works on a system like "100K", "100M", "100G", so the "M" would be necessary. Sorry I'm a programmer not a system admin.

qwe · 05-03-2009, 11:10 PM

Quote:

Originally Posted by ProG

I am not 100% sure but I believe it works on a system like "100K", "100M", "100G", so the "M" would be necessary. Sorry I'm a programmer not a system admin.

gotchya, thanks

qwe · 05-03-2009, 11:49 PM

hey ProG, anyway you can make me another simple script? to check for any empty lines and lines that start with 0-9 or some weird symbols (such as -,---,===,%---, etc, etc) and remove those lines ?

basically to remove any empty lines and lines that start with anything other then a valid character (a-z or A-Z)

d-null · 05-04-2009, 12:58 AM

Quote:

Originally Posted by qwe

hey ProG, anyway you can make me another simple script? to check for any empty lines and lines that start with 0-9 or some weird symbols (such as -,---,===,%---, etc, etc) and remove those lines ?

basically to remove any empty lines and lines that start with anything other then a valid character (a-z or A-Z)

if I understand what you are asking for, this will do it nicely:

Code:

<?php 
if ( ( $file = file( "file.txt" ) ) !== false ) 
{ 
    foreach( $file as $line ) 
    { 
        if (ctype_alpha($line{0}))
		
        { 
              echo $line . "<br>"; 
             
        } 
    } 
} 
?>

d-null · 05-04-2009, 01:08 AM

also note that I used <br> for html output instead of \n so if you need it for your text file use then you should change that part

ExLust · 05-04-2009, 01:40 AM

Good luck!

voa · 05-04-2009, 02:52 AM

im not sure that something like that is exist

d-null · 05-04-2009, 03:26 AM

Quote:

Originally Posted by voa

im not sure that something like that is exist

Scootermuze · 05-04-2009, 07:28 AM

Sentances?

Might wanna incorporate a spell checker into this thing too..

Sorry.. had to be said..

d-null · 05-04-2009, 11:09 AM

Quote:

Originally Posted by Scootermuze

Sentances?
Might wanna incorporate a spell checker into this thing too..

seeandsee · 05-04-2009, 11:12 AM

learn word and will be easy

05-04-2009, 11:19 AM

Quote:

Originally Posted by voa

im not sure that something like that is exist

Did you read the thread, or just post and leave?

d-null · 05-04-2009, 02:34 PM

Quote:

Originally Posted by Killswitch

Did you read the thread, or just post and leave?

gfy should have some kind of penalty system set up for stuff like that, like he should lose 25% of his post count for second offence or something like that

Cyber Fucker · 05-04-2009, 02:53 PM

Quote:

Originally Posted by Killswitch

Did you read the thread, or just post and leave?

...or maybe you are bot

05-03-2009, 09:03 PM	#1
qwe Confirmed User Join Date: Jul 2003 Posts: 2,109	making sentances into words... is there a way to let's say load up a .txt file with a bunch of lines of text and extract all the words as a list 1 word per line ? and maybe eliminate any words larger then 2-3 characters? Last edited by qwe; 05-03-2009 at 09:04 PM..

05-03-2009, 09:13 PM	#4
d-null . . . Industry Role: Join Date: Apr 2007 Location: NY Posts: 13,724	this kind of problem is first year computer science stuff, shouldn't be too hard to find a script example out there __________________ __________________ Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad! Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs Check out the #1 WordPress SEO Plugin: CyberSEO Suite

05-03-2009, 09:14 PM	#5
Iron Fist Too lazy to set a custom title Join Date: Dec 2006 Posts: 23,400	Good luck to you __________________ i like waffles

05-03-2009, 09:26 PM	#11
ProG Confirmed User Join Date: Apr 2009 Posts: 1,319	PHP Code: `<?php if ( ( $file = file( "file.txt" ) ) !== false ) { foreach( $file as $line ) { $words = explode( " ", trim( $line ) ); foreach( $words as $word ) { if ( strlen( $word ) > 3 ) { echo $word . "\n"; } } } } ?>`

05-03-2009, 09:51 PM	#16
TidalWave Confirmed User Industry Role: Join Date: Sep 2007 Location: Los Angeles Posts: 2,706	edit the filename to the one of the txt you already have. make sure the file is in the same directory as your php script. __________________ www.SwiftNode.com

05-03-2009, 09:07 PM	#2
baddog So Fucking Banned Industry Role: Join Date: Apr 2001 Location: the beach, SoCal Posts: 107,089	and maybe eliminate any words larger then 2-3 characters? That is going to make it a pretty short list.

05-03-2009, 09:49 PM	#15
ProG Confirmed User Join Date: Apr 2009 Posts: 1,319	/www is the root directory which means it would be localhost/test.php

05-03-2009, 10:00 PM	#19
ProG Confirmed User Join Date: Apr 2009 Posts: 1,319	Is the wampserver icon in the system tray all white? You only need Apache running. Click the icon, goto Apache, Restart Service

05-03-2009, 10:06 PM	#22
ProG Confirmed User Join Date: Apr 2009 Posts: 1,319	Click wampserver icon, goto Config Files, goto php.ini Search for "memory_limit" and change the value from 8M to 128M (You may have to restart the Apache service again)

05-03-2009, 10:15 PM	#24
ProG Confirmed User Join Date: Apr 2009 Posts: 1,319	If you right click and View Source it will not be on one line. Otherwise, replace "\n" with "<br>" for html output. Revised code to remove extra chars. PHP Code: <?php if ( ( $file = file( "file.txt" ) ) !== false ) { foreach( $file as $line ) { $words = explode( " ", trim( $line ) ); foreach( $words as $word ) { $word = ereg_replace( "[^A-Za-z0-9]", "", $word ); if ( strlen( $word ) > 3 ) { echo $word . "\n"; } } } } ?>

05-03-2009, 10:18 PM	#25
qwe Confirmed User Join Date: Jul 2003 Posts: 2,109	everything worked.... thanks for your help man you rock

05-03-2009, 10:22 PM	#26
qwe Confirmed User Join Date: Jul 2003 Posts: 2,109	btw do you have icq/skype? i might need more small scripts (next one i wont be asking for free )

05-03-2009, 10:25 PM	#27
ProG Confirmed User Join Date: Apr 2009 Posts: 1,319	np. Just post your problems on GFY, it gives some of us something to do besides look at boobies on an "adult" message board.

05-03-2009, 10:32 PM	#28
ProG Confirmed User Join Date: Apr 2009 Posts: 1,319	One more revision that will give you a listing of unique words (no duplicates). PHP Code: <?php $unique_words = array( ); if ( ( $file = file( "file.txt" ) ) !== false ) { foreach( $file as $line ) { $words = explode( " ", trim( $line ) ); foreach( $words as $word ) { $word = ereg_replace( "[^A-Za-z0-9]", "", $word ); if ( strlen( $word ) > 3 ) { if ( !in_array( $word, $unique_words ) ) { $unique_words[] = $word; } } } } } foreach( $unique_words as $word ) { echo $word . "\n"; } ?> Enjoy. Now that you have WampServer, head over to PHP.net, read the documentation and learn something

05-03-2009, 10:37 PM	#30
qwe Confirmed User Join Date: Jul 2003 Posts: 2,109	btw that latest code you gave gives me this error "Fatal error: Maximum execution time of 30 seconds exceeded in C:\wamp\www\test3.php on line 13" maybe file too big ?

05-03-2009, 10:41 PM	#31
qwe Confirmed User Join Date: Jul 2003 Posts: 2,109	yah, i down sized file like 10x smaller, and it worked....

05-03-2009, 10:46 PM	#32
ProG Confirmed User Join Date: Apr 2009 Posts: 1,319	You know, if you have 6gb of RAM you can use it all. Change the memory_limit to 512MB or 1024MB.

05-03-2009, 10:49 PM	#34
ProG Confirmed User Join Date: Apr 2009 Posts: 1,319	Yes, that too. I usually set my localhost to 300 seconds (5 minutes).

05-03-2009, 11:02 PM	#36
qwe Confirmed User Join Date: Jul 2003 Posts: 2,109	one more thing, in php.ini should it be memory_limit = 1024M OR memory_limit = 1024 does it need to have M at the end ? thx

05-03-2009, 11:08 PM	#37
ProG Confirmed User Join Date: Apr 2009 Posts: 1,319	I am not 100% sure but I believe it works on a system like "100K", "100M", "100G", so the "M" would be necessary. Sorry I'm a programmer not a system admin.

05-03-2009, 11:49 PM	#39
qwe Confirmed User Join Date: Jul 2003 Posts: 2,109	hey ProG, anyway you can make me another simple script? to check for any empty lines and lines that start with 0-9 or some weird symbols (such as -,---,===,%---, etc, etc) and remove those lines ? basically to remove any empty lines and lines that start with anything other then a valid character (a-z or A-Z) Last edited by qwe; 05-03-2009 at 11:52 PM..

05-04-2009, 01:08 AM	#41
d-null . . . Industry Role: Join Date: Apr 2007 Location: NY Posts: 13,724	also note that I used <br> for html output instead of \n so if you need it for your text file use then you should change that part __________________ __________________ Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad! Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs Check out the #1 WordPress SEO Plugin: CyberSEO Suite

05-04-2009, 01:40 AM	#42
ExLust Confirmed User Join Date: Aug 2008 Posts: 3,223	Good luck! __________________ BE A PARTNER

05-04-2009, 02:52 AM	#43
voa Too lazy to set a custom title Join Date: Nov 2006 Posts: 16,532	im not sure that something like that is exist

05-04-2009, 07:28 AM	#45
Scootermuze Confirmed User Join Date: Dec 2001 Posts: 4,513	Sentances? Might wanna incorporate a spell checker into this thing too.. Sorry.. had to be said..

05-04-2009, 11:12 AM	#47
seeandsee Check SIG! Industry Role: Join Date: Mar 2006 Location: Europe (Skype: gojkoas) Posts: 50,945	learn word and will be easy __________________ BUY MY SIG - 50$/Year Contact here