Return to Snippet

Revision: 26402
at April 27, 2010 09:56 by alemani


Initial Code
wget -m -k -K -E http://url/of/web/site

Initial URL

                                

Initial Description
If this command seems to run forever, it's because there may be parts of the site that generate an infinite series of different URLs. You can solve this in many ways, the simplest being to use the -l option to specify how many links “away” from the home page wget should travel. For instance, -l 3 will refuse to download pages more than three clicks away from the home page. You’ll have to experiment with different values for -l. Consult man wget for additional workarounds.

Initial Title
Mirror a website using wget

Initial Tags
web

Initial Language
Bash