No matter what you use your computer for, there has probably been at least one occasion where you needed to download something, and opening a browser felt like overkill. This would be a great use case for cURL.
As the name suggests, cURL is a command-line tool for transferring data with URLs. One of the simplest uses is to download a file via the command line. This is deceptive, however, as cURL is an incredibly powerful tool depending on how you use it. Even if you’re somewhat familiar with the command, you probably aren’t using it to its full potential.
Basic cURL Functionality
One of the most basic things you can do with cURL is download a webpage or file. To do this you just use the
curl command followed by a URL. Here’s an example:
Most of the time, using the command this way will get you a terminal full of raw HTML at best and a wash of unrecognizable characters at worst. If you would rather save it to a file, you can use standard Unix-style redirects to do so.
Follow HTTP Headers
Your browser often fixes this for you, but the Internet is very specific. When you type in a URL, there’s a good chance that you may be redirected one or more times before you reach your destination.
Say for example that you’re trying to reach the Make Tech Easier website. Typing the following will simply get you a redirect notice:
You can follow these HTTP location headers by using the
-L flag like so:
This won’t look great in your terminal, but it’s a good option to know.
Save cURL Results to a File
There are a few ways to save a URL’s contents to a file. The
-o option lets you decide the filename, while the
-O option uses the filename in the URL to save. To choose your own file, use the following option:
More often than not, you’ll want to save a file to the same name it uses on the server. For that, use the
Download Multiple Files at Once
If you need to download a few files at the same time, cURL makes that easy to do. You’ll typically want to use this with the
When you download this way, cURL will try to re-use the connection instead of making new connections each time.
Continue a Stopped Download
It’s never fun when a download stops halfway through. Fortunately, cURL makes it easy to resume a download without having to start over. The syntax is somewhat strange, as you need to add
-C - to your command.
Say you started a download with the following:
Then you stopped it by pressing Ctrl + C. You can resume it with the following command:
Use Basic HTTP Authentication
This won’t work for everything that requires a username and password, but if a server uses basic HTTP authentication, cURL can work with it. To download a file with username/password authentication, you can use the following:
This also works with FTP servers, as cURL supports a ton of different protocols.