Shell wget/sed script help needed

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • acctman
    Confirmed User
    • Oct 2003
    • 2840

    #1

    Shell wget/sed script help needed

    I need help using sed to parse html codiing, this is what i'm trying to do...

    1. wget to http://site.com/xap/wp7?p=1
    2. view the html extract all the ProductName's from in between title="Free Shipping ProductName"> ... ex: title="Free Shipping HD7-Case001"> , HD7-Case001 is extracted.
    3. processes up to page 50

    Code:
    #! /bin/sh 
    
    for ((  i = 1 ;  i <= 50;  i++  ))
    do
    	wget -q -O- "http://site.com/xap/wp7?p=$i" | 
    	sed ... Need the parsing part
    
    done < "products.txt"
  • fris
    Too lazy to set a custom title
    • Aug 2002
    • 55679

    #2
    your answer has been answered

    http://stackoverflow.com/questions/4...g-wget-and-sed
    Since 1999: 69 Adult Industry awards for Best Hosting Company and professional excellence.

    Comment

    • fris
      Too lazy to set a custom title
      • Aug 2002
      • 55679

      #3
      Code:
      [chris@jumbo ~]$ cat test.html
      <a href="http://www.domain.com" title="Free Shipping HD7-Case001">
      link</a>
      <a href="http://www.domain.com" title="Free Shipping HD2-Case001">link</a>
      <a href="http://www.domain.com" title="Free Shipping HD3-Case001">link</a>
      <a href="http://www.domain.com" title="Free Shipping HD7-Case009">link</a>
      <a href="http://www.domain.com" title="Free Shipping HD7-Case002">link</a>
      Code:
      [chris@jumbo ~]$ cat test.html | tr '"' '\n' | grep "^Free Shipping " | cut -d ' ' -f 3
      HD7-Case001
      HD2-Case001
      HD3-Case001
      HD7-Case009
      HD7-Case002
      Since 1999: 69 Adult Industry awards for Best Hosting Company and professional excellence.

      Comment

      • acctman
        Confirmed User
        • Oct 2003
        • 2840

        #4
        Originally posted by fris
        Code:
        [chris@jumbo ~]$ cat test.html
        <a href="http://www.domain.com" title="Free Shipping HD7-Case001">
        link</a>
        <a href="http://www.domain.com" title="Free Shipping HD2-Case001">link</a>
        <a href="http://www.domain.com" title="Free Shipping HD3-Case001">link</a>
        <a href="http://www.domain.com" title="Free Shipping HD7-Case009">link</a>
        <a href="http://www.domain.com" title="Free Shipping HD7-Case002">link</a>
        Code:
        [chris@jumbo ~]$ cat test.html | tr '"' '\n' | grep "^Free Shipping " | cut -d ' ' -f 3
        HD7-Case001
        HD2-Case001
        HD3-Case001
        HD7-Case009
        HD7-Case002
        thanks... everything worked out this morning i had an extra character that I missed typed.

        Comment

        Working...