Using Xargs to Download Files

One of my favorite ways to batch download files on the CLI is using xargs, cat and cURL. I know, I know… there are more efficient ways of doing this - usually involving parallel, but I’ve found this to be the most reliable way when moving between Linux distros.

First, you need a file(files.txt) with a list of files to download. I like to name my files when they are downloaded, so my list also has a name for each file.

files.txt contents:

"filename.mp4" https://example.com/path/to/filename.mp4?foo=bar
"filename2.mp4" https://example.com/path/to/filename2.mp4?baz=qux

Then to download these files we just run the following:

cat files.txt | xargs -n 2 curl -L -o

That’s it, the files will download one at a time with the names in the file.

© 2014 - 2021 · Home ·
3E2BdqbtEy2SkEEbXW8Bv5snVEMPYxjvn3