advanced code snippet search

Posted By

noah on 11/02/07


Tagged

http download commandline pipe web perl lwp shortcut tools productivity textonly grabit numbered sequential scraping one-liners


Versions (?)

Who likes this?

3 people have marked this snippet as a favorite

basicmagic
vali29
kfaulk1


Download linked JPEGs from a Web page, on the command line

 / Published in: Bash
 

The following command will download all the files with a JPG extension that are linked from http://flickr.com.

Requires the LWP and HTML::Tree Perl modules. You must also have Wget installed on your system for this to work.

Expand | Embed | Plain Text 
  1. lwp-request -o links http://flickr.com/ | grep jpg | perl -pe "chomp; s/.*?(\S+jpg)/$1 /;" | xargs wget
  2.  
  3. #or
  4.  
  5. lwp-request -o links http://flickr.com | grep jpg | perl -pe "chomp; s/IMG\s*(.*jpg)/$1 /;" | xargs wget
  6.  
  7.  
  8. #The following will download the numbered images site.com/gallery/01.jpg through site.com/gallery/16.jpg
  9.  
  10. perl -e "$i=0;while($i<16){open(WGET,qq/|xargs wget/);printf WGET qq{http://site.com/gallery/%02d.jpg},++$i}"

Report this snippet  

You need to login to post a comment.