Quote:
Originally Posted by critical
Check to make sure the domain you are querying is actually returning results to
you. A smart admin blocks queries from wget to db/query servers to avoid certain ddos attacks while a smart coder sets the client settings in wget to match that of mozilla or another popular web browser so it does not look automated. Set wget to look like a browser and see if you get better results. Code looks straight.
:-)
|
weird cause I used a similar code to get the product names
Code:
for page in {1..50}
do
wget -q -U Mozilla "http://www.domain.com/catalog_search/cat?p=$page" -O - \
| tr '"' '\n' | grep "^Product photo for " | cut -d ' ' -f 4 >> catNames.txt
sleep 15
done