Build a Better Process

Linux wget Command Summary with Examples

Linux wget allows you to download files non-interactively.
  1. Purpose - Learn what wget is for and how to find help.
  2. Options - Review a few common options and arguments.
  3. Examples - Walk through code examples with wget.
  4. A tip - Finish off with one more insight.
face pic by Paul Alan Davis, CFA
Updated: February 24, 2021
In this tutorial on Linux wget, 85 of 100, below find a 3-4 minute introductory video, a text-based tutorial and all of the code examples from the video.

Outline Back Tip Next

/ factorpad.com / tech / linux-essentials / wget-command.html

An ad-free and cookie-free website.

Examples of the Linux wget Command


Learn to download files at the Linux command line.

Video Tutorial

Videos can also be accessed from the Linux Essentials Playlist on YouTube.

Linux wget Command Summary with Examples (3:40)

Video Script

The Command and Why You Need It

Our eighty-fifth word, or command to memorize is wget from our category Communication.

wget allows you to download files non-interactively.

Common Linux wget Options
-options description
-h, --help Print help screen
-b, --background Operate in the background
-a, --append-output=FILE Append messages to FILE

Recall from the last video (tutorial) on ftp we manually logged in to a remote site, and now we'll download files non-interactively.

Before we start, it helps to think of commands as mini programs and most follow this structure: command -option(s) argument(s).

The wget command has over 130 options, and yes, I counted. (Please review man wget.) The argument is the URL, or Uniform Resource Locator, which is that long field at the top of your browser pointing to a file using a variety of Protocols, normally http, but could also be https, ftp and others.

Like most commands, help is available with double-dash --help. The -b option will kick time-intensive operations into the background, and -a will append messages to a specified log file.

So why is wget an important command? Well, organizations, like search engines, need an automated way to scrape and process information from the Internet. And now you know how to do that.


Okay, the best way to embed this in your memory is by typing in your own terminal window.

Find this on your Mac using a program called Terminal. On Linux use Terminal or Konsole, and currently Microsoft is adding this functionality to Windows.

Here we go. So wget can get very complex, but let's try it for something simple, and something I've been curious about. Let's use less to see if that simple-looking Google homepage is simple.

$ wget google.com -O video85a.txt; less video85a.txt --2016-11-12 20:18:02-- http://google.com/ Resolving google.com (google.com)...,,, ... Connecting to google.com (google.com)||:80... connected. HTTP request sent, awaiting response.. 301 Moved Permanently Location: http://www.google.com/ [following] --2016-11-12 20:18:02-- http://www.google.com/ Resolving www.google.com (www.google.com)...,,, ... Connecting to www.google.com (www.google.com)||:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: 'video85a.txt' video85a.txt [ <=> ] 10.05K --.-KB/s in 0s 2016-11-12 20:18:02 (165 MB/s) - 'video85a.txt' saved [10288]

(The output below is a subset of the html file downloaded from the google.com home page.)

<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en"><head><meta content="Search t he world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for." name="description"><meta content="noodp name="robots"><meta conten t="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_stan dard_color_128dp.png" itemprop="image"><title><Google</title><script>(function(){window.google={kEI:'euknWL2AOo SpjWTLu5_ADw',kEXPI:'1351827,1351903,1352084,3700323,402641,4029815,4031109,4032678,4036527,4038012,4039268,4 043492,4045841,4048347,4052304,4053261,4055745,4065787,4066688,4068550,4069838,4069840,4070138,4072364,4072602 ,4072705,4072773,4073111,4073405,4073418,4073757,4073915,4073959,4079597,4079626,4080167,4080383,4080535,408 0639,4080640,4080815,4081037,4081038,4081128,4081264,4081470,4081913,4082309,4082939,4082942,4 (many lines trimmed, you get the point)

Okay, well looking at it, it doesn't seem as complex as you might think (sarcasm intended), although the formatting is kind of strange here (no sarcasm intended here). ;)

And next, there's another public file I know something about, it's a data set I use for teaching an investment modeling boot camp on YouTube, the file is at factorpad.com/returns.txt.

$ wget factorpad.com/returns.txt -O video85b.txt --2016-11-12 20:18:30-- http://factorpad.com/returns.txt Resolving factorpad.com (factorpad.com)... Connecting to factorpad.com (factorpad.com)||:80... connected. HTTP request sent, awaiting response... 200 OK Length: 4080 (4.0K) [text/plain] Saving to: 'video85b.txt' video85b.txt 100%[==========================================>] 3.98K --.-KB/s in 0s 2016-11-12 20:18:30 (548 MB/s) - 'video85b.txt' saved [4080/4080]

And then last, let's review it using head.

$ head video85b.txt Date MSFT EBAY ABT MRK Market Tbill 4/30/2003 0.056175 0.089087 0.083536 0.062066 0.072716 0.001000 5/30/2003 -0.037544 0.094070 0.096480 -0.044689 0.027079 0.000900 6/30/2003 0.041853 0.023119 -0.017733 0.096319 0.035890 0.001000 7/31/2003 0.030031 0.031731 -0.100672 -0.087036 -0.031487 0.000700 8/29/2003 0.004165 0.032805 0.026752 -0.039496 0.006057 0.000700 9/30/2003 0.048265 -0.031944 0.055831 0.013306 0.021365 0.000800 10/31/2003 -0.054537 0.042692 0.004456 -0.125840 -0.033307 0.000700 11/28/2003 -0.016450 -0.000715 0.037072 -0.082486 -0.015645 0.000700 12/31/2003 0.064566 0.156021 0.054299 0.147581 0.105617 0.000800

It has returns on four stocks, an Index and Treasury bills for 5-years and we use that to geek out on Statistics in Finance (Quant 101). Check that out if you are interested.

A Final Tip

Okay now you know how to use wget. And you know the syntax for commands, options and arguments.

One last tip about the wget command. So wget can also take a long list of files as input, it can follow links recursively and it'll even eat cookies.

Okay, thanks for visiting today. I hope this was a helpful introduction to the wget command.

Learn More About The Series

For an overview of the 100 videos, the 8 quizzes, a cheat sheet, the categories and a Q&A section, visit:

What's Next?

Free learning content here is growing. Connect so you don't miss something relevant for your progress.

Outline Back Tip Next

/ factorpad.com / tech / linux-essentials / wget-command.html

linux wget command
wget command
linux wget examples
wget example
wget https
wget ssl
download entire website
wget remote files
wget download
download full website
man wget
wget command not found
wget recursive
wget directory
wget command in linux
web scraping

A newly-updated free resource. Connect and refer a friend today.