Wget command in Linux is used to download files from the internet and locally. wget is a command line utility used for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies. Wget is non-interactive, meaning that it can work in the background, while the user is not logged on. When we do the minimal installation of Linux distributions then wget does not get installed automatically. So to install wget on Linux, run
$ sudo yum install -y wget // CentOS 7 / RHEL 7 $ sudo dnf install -y wget // CentOS 8/ RHEL 8 $ sudo apt install -y wget // Ubuntu / Debian
In this article, we will discuss 12 useful wget command examples in linux.
Example 1) Downloading a single file with wget
To Download a file using wget, simply type wget command followed by the absolute path of file,
# wget http://isoredirect.centos.org/centos/8/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.iso
This command will download the CentOS 8 ISO file in the user’s current working directory.
Example 2) Resume Partial Downloaded File (-c)
There are some scenarios where we start downloading a large file but in the middle Internet got disconnected , so using the option ‘-c’ in wget command we can resume our download from where it got disconnected.
# wget -c http://mirror.nbrc.ac.in/centos/7.0.1406/isos/x86_64/CentOS-7.0-1406-x86_64-DVD.iso
Example 3) Download Files in the background (-b)
Use ‘-b’ option in wget command to download files in the background
[email protected]:~$ wget -b http://isoredirect.centos.org/centos/8/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.iso Continuing in background, pid 4505. Output will be written to ‘wget-log’.
As we can see above that downloading progress is capture in ‘wget-log’ file in user’s current directory.
[email protected]:~$ tail -f wget-log 2300K .......... .......... .......... .......... .......... 0% 48.1K 18h5m 2350K .......... .......... .......... .......... .......... 0% 53.7K 18h9m 2400K .......... .......... .......... .......... .......... 0% 52.1K 18h13m 2450K .......... .......... .......... .......... .......... 0% 58.3K 18h14m 2500K .......... .......... .......... .......... .......... 0% 63.6K 18h14m 2550K .......... .......... .......... .......... .......... 0% 63.4K 18h13m 2600K .......... .......... .......... .......... .......... 0% 72.8K 18h10m 2650K .......... .......... .......... .......... .......... 0% 59.8K 18h11m 2700K .......... .......... .......... .......... .......... 0% 52.8K 18h14m 2750K .......... .......... .......... .......... .......... 0% 58.4K 18h15m 2800K .......... .......... .......... .......... .......... 0% 58.2K 18h16m 2850K .......... .......... .......... .......... .......... 0% 52.2K 18h20m
Example 4) Limiting Download Speed in wget
By default wget command try to use full bandwidth , but there may be a case that you are using shared internet , so if you try to download huge file using wget , this may slow down Internet of other users. This situation can be avoided if you limit the download speed using ‘–limit-rate’ option.
# wget --limit-rate=100k http://isoredirect.centos.org/centos/8/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.iso
Example 5) Download multiple files with wget
If you want to download multiple files using wget command , then first create a text file and add all URLs in that text file. Example is shown below:
# cat download-list.txt url1 url2 url3 url4
Now run below command,
# wget -i download-list.txt
Example 6) Increase Retry Attempts in wget
We can increase the retry attempts using ‘–tries’ option in wget. By default wget command retries 20 times to make the download successful.
This option becomes very useful when you have internet connection problem and you are downloading a large file , then there is a chance of failures in the download.
# wget --tries=75 http://mirror.nbrc.ac.in/centos/7.0.1406/isos/x86_64/CentOS-7.0-1406-x86_64-DVD.iso
Example 7) Redirect wget logs to a log File
We can redirect the wget command logs to a log file using ‘-o’ option.
#wget -o download.log http://isoredirect.centos.org/centos/8/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.iso
download.log file will be created in the user’s current directory.
Example 8) Download full website for local viewing
# wget --mirror -p --convert-links -P ./<Local-Folder> website-url
Whereas
- –mirror : turn on options suitable for mirroring.
- -p : download all files that are necessary to properly display a given HTML page.
- –convert-links : after the download, convert the links in document for local viewing.
- -P ./Local-Folder : save all the files and directories to the specified directory.
Example 9) Reject file types while downloading
When you are planning to download full website , then we can force wget command not to download images using ‘–reject‘ option .
# wget --reject=png <Website-To-Be-Downloaded>
Example 10) Setting Download quota in wget command
We can force wget command to quit downloading when download size exceeds certain size. Use ‘-Q‘ option in wget command to set download quota.
# wget -Q10m -i download-list.txt
Note that quota will never affect downloading a single file. So if you specify wget -Q10m ftp://wuarchive.wustl.edu/ls-lR.gz, all of the ls-lR.gz will be downloaded. The same goes even when several URLs are specified on the command-line. However, quota is respected when retrieving either recursively, or from an input file. Thus you may safely type ‘wget -Q10m -i download-list.txt‘ download will be aborted when the quota is exceeded.
Example 11) Downloading file from password protected site
# wget --ftp-user=<user-name> --ftp-password=<password> Download-URL or # wget --user <user_name> --password <passwd> http://<url-path>/file_to_be_downloaded
Another way to specify username and password is in the URL itself.
Either method reveals your password to anyone who bothers to run “ps“. To prevent the passwords from being seen, store them in .wgetrc or .netrc, and make sure to protect those files from other users with “chmod“. If the passwords are really important, do not leave them lying in those files either edit the files and delete them after wget has started the download.
Example 12) Downloading file from https URL and skip certificate checks
There are some scenarios where we want to download file from https ports and want to skip certificate checks, so it can accomplished by using option “–no-check-certificate” in wget command,
# wget https://about.gitlab.com/ --no-check-certificate
Downloading Oracle Java using wget command from https portal and skip certificate checks
# wget --header "Cookie: oraclelicense=accept-securebackup-cookie" https://download.oracle.com/otn-pub/java/jdk/12.0.1+12/69cfe15208a647278a19ef0990eea691/jdk-12.0.1_linux-x64_bin.rpm --no-check-certificate
Note: Downloading Files via Proxy
There are some situations where our system is running behind the proxy server, so in such situations first we need to set proxy and then we can use wget command to download file from the internet. To set proxy on the command line use the following variables and export command
# export http_proxy=http://<Your-Proxy-IP>:<Proxy-Port> # export https_proxy=http://<Your-Proxy-IP>:<Proxy-Port> # export ftp_proxy=http://<Your-Proxy-IP>:<Proxy-Port>
In Case User Name and password is required for proxy to work then use the followings
# export http_proxy=http://<user-name>:<password>@<Your-Proxy-IP>:<Proxy-Port> # export https_proxy=http://<user-name>:<password>@<Your-Proxy-IP>:<Proxy-Port> # export ftp_proxy=http://<user-name>:<password>@<Your-Proxy-IP>:<Proxy-Port>
Additional Example
Download and Extract tar file with single wget command
Let’s suppose we want to download latest version of WordPress tar file and want to extract it under specific folder like /var/www/html, so to accomplish this task using below wget command,
# wget -q -O - http://wordpress.org/latest.tar.gz | tar -xzf - --strip-components=1 -C /var/www/html
That’s all from this article, i hope these wget command examples are informative to you, Please do share your valuable feedback and comments in the comments section below.
Also Read : 20 ps Command Examples to Monitor Linux Processes
Another useful example – download and extract archive to specific directory
wget -q -O – ‘http://wordpress.org/latest.tar.gz’ | tar -xzf – –strip 1 -C /path/to
Pradeep should include this in his examples, as this was the exact one I was looking for. I noticed something was running on our system under wget -qO- -O and I couldn’t figure out if this was a typo or what, so thanks for clearing it up for me. Took a lot from this, just wish it was more expansive!