Using technology in order to access certain information is an action that we carry out daily from our home or even on the street through mobile devices and that has become something everyday over the last few years. The downloading of information and elements is something we do constantly for personal, labor, administrative or support purposes..
In this case, this tutorial is focused when our goal is to download programs or applications that will be used in different Linux distributions .
In the case of terminal-based distributions, especially server distributions, the download task may be somewhat more complex due to the use of commands . So that this is not a nightmare, there is the curl command, which will make the download process something much simpler but complete..
It is for this reason that today in this tutorial TechnoWikis will explain how we can make use of cURL in Linux to carry out download tasks in a comprehensive way step by step.
What is cURL?
cURL has been developed as a command line and library tool whose purpose is to transfer data through a URL.
Curl is an integral command since it is capable of supporting protocols such as DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB , SMBS, SMTP, SMTPS, Telnet and TFTP..
Similarly, cURL supports SSL, HTTP POST, HTTP PUT certificates, FTP uploads, HTTP uploads, proxies, HTTP / 2, cookies, user authentication and password (Basic, Plain, Digest, CRAM-MD5, NTLM, Negotiate and Kerberos ), file transfer, proxy tunneling and more.
cURL can also be used in command lines or scripts with the task of transferring data, but we can also implement it in devices such as televisions, routers, printers, audio equipment, mobile phones, media players since curl is imposed as a backbone network Internet transfer for software applications that can be used in this type of elements or devices.
CURL features
- Supports cookies, forms and SSL.
- Compatible with configuration files.
- It allows multiple loading with a single command.
- Progress bar, speed limitation and integrated download time details
Next, we will see how to use cURL in Linux and thus have a practical utility for downloading network elements.
1. How to view the version of cURL on Linux
With the use of the -V or - -version parameters, we will access both the curl version and the protocols and features supported in that version:
curl –version
2. How to download a simple file with cURL on Linux
It will be possible to obtain content from the URL and display its output in STDOUT with cURL as follows:
curl http://www.centos.org
We can also use curl with the -O and -o options, with the first option the file will be saved in the current working directory with the same name as in the remote location, while with the second parameter it is possible to specify a file name and / or different location:
3. How to view HTTP Query headers with Linux cURL
HTTP headers allow the remote web server to perform the process of sending additional information about itself along with the actual request, this is used to provide the customer with details on how the current request is being managed.
To check HTTP headers from a website, we execute the following:
curl -I www.solvetic.com
4. How to store website cookies with cURL Linux
It is possible through the cURL command to download cookies from a website and then view them, for this, we will use the following command in order to save them in a .txt file. and then it will be possible to use the cat command to see the generated file:
curl --cookie-jar solveticcookies.txt https://www.solvetic.com/index.html -O cat solveticcookies.txt
5. How to limit download rate with cURL Linux
In order to prevent cURL from misusing the bandwidth, it is possible to limit the download speed to the required speed as well as the following example:
curl --limit-rate 1K http://domain.com/archive.tar.gz -O
In this case it is limited to 1 Kilobyte.
6. How to track HTTP headers location with cURL Linux
By default, cURL does not follow the headers of the HTTP location of the visited sites, so when a requested web page is moved to another place, an HTTP location header will be sent in response and the location where the web page is located will be obtained real, for this we execute:
curl http://www.solvetic.com
At the bottom we see that this heading was moved.
7. How to list or download using ranges with Linux cURL
The cURL command utility supports ranges hosted in the URL, with these ranges, files that match within that range will be downloaded, it is useful for downloading packages from FTP replica sites for example:
curl ftp://ftp.uk.debian.org/debian/pool/main/[az[/
8. Get more website information with Linux cURL command
When using the -v parameter in the cURL command, the detailed mode is enabled and will print the details of the site in a complete way:
curl -v http://www.solvetic.com
9. Other actions with the cURL command in Linux
How to restart a slow download with curl on Linux
In the event that a download has been interrupted, it will be possible in a simple way using the -C parameter - which tells curl to resume the start of the download where it was left:
curl -C - -O http://domain.com/archive.tar.gz
Download various files with cURL on Linux
Use a proxy with or without authentication with Linux curl
In the case of using a proxy server that listens on port 8080 in our domain, we must use the following syntax to use curl:
curl -x proxy.domain.com:8080 -U user: password -O http://domain.com/yourfile.tar.gz
Download files from an FTP server with Linux cURL
We can use the following command to download files through an FTP server:
curl -u username: password -O ftp: //serverFTP/file.tar.gz
Upload files to an FTP server with Linux cURL
In the case of wishing to upload an FTP file, we execute the following:
curl -u username: password -T file.tar.gz ftp: // serverFTP
How to send cookies from a website with Linux curl
It will be possible to use the cookies stored towards subsequent requests to the same site, for this we execute the following:
curl --cookie solveticcookies.txt https://www.solvetic.com
How to download a file if it was only modified in a date range with Linux cURL
With cURL it is possible to access files that have been modified after a particular time using the -z option like this:
curl -z 25-Aug-17 http://www.site.com/yy.html
Thus, the cURL command gives us a series of useful options for everything related to downloading and managing items on the network.
