curl: my own cheatsheet
curl is a project providing a library and command-line tool for transferring data using various protocols.
The command line tool can be used inside shell scripts with ease and it has around 120 command line options for various tasks.
Is a really useful tool, in both application debug and web penetration testing activities.
Today i'd like to share my own cheatsheet, built during day by day activities.
GET Requests
These curl recipes show you how to send GET requests with curl.
The GET method is the default method for making requests in curl so you
don't have to specify any arguments.
Send a GET Request and Print the Response to Screen
$ curl https://www.andreafortuna.org
You don't need to specify any arguments to make a GET request: the command makes a GET request to https://www.andreafortuna.org and prints the returned content to screen.
Send a GET Request and Save the Response to a File
$ curl -o response.txt https://www.andreafortuna.org
With this command, curl makes a GET request to https://www.andreafortuna.org and saves the body of the HTTP response to response.txt file.
POST requests
By default, curl sends GET requests: to make it send POST requests, use the -X POST command line argument and use the -d argument to add POST data to the request.
To change the Content-Type header that tells the web server what type of data you're sending, use the -H argument.
Send an Empty POST Request
curl -X POST https://www.andreafortuna.org
This recipe uses the -X POST option that makes curl send a POST request to https://www.andreafortuna.org. Once it receives a response from the server, it prints the response body to the screen. It doesn't send any POST data.
Send a POST request with form data
curl -d 'login=emma&password=123' -X POST https://site.com/login
This recipe sends a POST request to https://site.com/login with login=emma&password=123 data in request's body. When using the -d argument, curl also sets the Content-Type header to application/x-www-form-urlencoded. Additionally, when the -d option is set, the -X POST argument can be skipped as curl will automatically set the request's type to POST.
Send a POST request and follow a redirect
curl -L -d 'tweet=hi' https://api.twitter.com/tweet
By default, curl doesn't follow the redirects .
The -L command line option tells curl to follow any possible redirects that it may encounter in the way. (the-d argument forces curl to make a POST request)
Send a POST request with JSON data
curl -d '{"login": "andrea", "password": "verystrongpassword"}' -H 'Content-Type: application/json' https://site.com/login
This request can be accomplished passing JSON to the -d option and also using the -H option that sets the Content-Type header to application/json (necessary because otherwise, the web server won't know what type of data this is).
Send a POST Request with XML Data
curl -d '<user><login>andrea</login><password>verystrongpassword</password></user>' -H 'Content-Type: text/xml' https://site.com/login
Just like the form data and JSON data, XML data is specified in the -d argument, and the Content-Type header is changed to text/xml via the -H argument.
Send a POST request with plain text data
curl -d 'hello, world!' -H 'Content-Type: text/plain' https://site.com/login
Just sends a POST request with a plain text string hello, world! in request's body. The Content-Type header is filled with text/plain to tell web server that it's just plain text coming in.
Send a POST Request with Data from a File
curl -d '@data.txt' https://site.com/login
The POST data are loaded from a file called data.txt.
The extra @ symbol before the filename tells curl that data.txt is a file and not just a string.
URL-encode POST Data
curl --data-urlencode 'comment=hello, world!' https://site.com/login
The -d argument assumes that data are already URL-encoded.
If it is not, just replace -d with --data-urlencode: it works exactly the same way as -d, except the data gets URL-encoded by curl before the request.
POST a binary file
curl -F 'file=@photo.png' https://site.com/upload
The -F argument forces curl to make a multipart form data POST request and also sets the Content-Type header to multipart/form-data.
POST a binary file and set its MIME type
curl -F 'file=@photo.png;type=image/png' https://site.com/upload
Similar to the previous example, the -F argument is used to upload a binary file via a multipart POST request, but now is also specified the MIME type of this file as image/png.
If no type is specified, then curl sets it to application/octet-stream.
POST a binary file and change its filename
curl -F 'file=@photo.png;filename=selfie.png' https://site.com/upload
This example uses the -F argument to upload a photo.png via a POST request and set the filename sent to the web server to selfie.png.
HTTP Headers manipulation
Add a single header
curl -H 'Accept-Language: en-US' https://www.andreafortuna.org
The -H argument is used to add Accept-Language: en-US header to a GET request to https://www.andreafortuna.org: this header tells site to serve the English version of the page.
Add two or more headers
curl -H 'Accept-Language: en-US' -H 'Other header: xyz' -H 'Final header: zzz' https://www.andreafortuna.org
Using multiple -H argoments is possible to send multiple headers to the request.
Add an empty header
curl -H 'Nothing;' https://google.com
If you pass Nothing; (or other text, with a semicolon at the end) as an argument to the -H option, curl converts it to an empty header Nothing: with no value.
User Agent manipulation
Change the User Agent to Firefox
curl -A 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0' https://www.andreafortuna.org
The -A argument is used sets the user agent to Firefox 60 (Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0).
Change the User Agent to Chrome
curl -A 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36' https://google.com
Similar to the previous example.
Change the User Agent to Google Bot
curl -A 'Googlebot/2.1 (+http://www.google.com/bot.html)' https://www.andreafortuna.org
Really useful to skip paywalls: often, websites will serve the content to a Google bot (so that they are indexed) but when a real browser requests them, they'll show a paywall.
Remove the User Agent
curl -A '' https://www.andreafortuna.org
The empty string passed to the -A command line option tells curl to remove the user agent header and not to send User-Agent HTTP header.
Working with cookies
Add a cookie
curl -b 'session=xyz' https://www.andreafortuna.org
The -b name=value argument sets a cookie with the name session to the value xyz in a GET request to https://www.andreafortuna.org.
Add two or more cookies
curl -b 'session=xyz' -b 'logged=1' https://www.andreafortuna.org
The same syntax of previous example can be used to set two o more cookies.
Add an empty cookie
curl -b 'session=' https://www.andreafortuna.org
Using only the first half of -b name=value option, curl can send an empty cookie that in the HTTP header looks like this Cookie: session=.
Save cookies to a file
curl -c cookies.txt https://www.andreafortuna.org
The -c cookies.txt option saves the response cookies to cookies.txt file. When curl makes a GET request to https://www.andreafortuna.org, the web server responds with one or more Set-Cookie: name=value header values.
Curl takes them and saves them to a file.
Load cookies from a file
curl -b cookies.txt https://www.andreafortuna.org
The cookies saved in the previous example can be loaded with the -b cookies.txt option.
Referrer manipulation
Add a referrer
curl -e 'https://google.com?q=devops' https://www.andreafortuna.org
This -e argument is used to set the HTTP referrer to https://google.com?q=devops and makes a GET request to https://www.andreafortuna.org. In the access.log file of andreafortuna.org's webserver, it will appear that someone came from Google while searching for devops.
Send an empty referrer
curl -e '' https://www.andreafortuna.org
Using the -e option leaving the argument empty '' clears the value part of Referer: HTTP header and sends an empty HTTP referer.
Use the Basic HTTP authentication
Set the username and password via the -u argument
curl -u 'andrea:strongpassword' https://greatapp.andreafortuna.org/login
This example uses the -u 'andrea:strongpassword' argument to set the username to andreaand password to strongpassword.
The username and password is then base64-encoded by curl and sent to the server at https://greatapp.andreafortuna.org/login in the Authorization HTTP header.
Set the username and password in the URL
curl https://andrea:strongpassword@greatapp.andreafortuna.org/login
If you put the username and password in the URL , curl is smart enough to figure it out and it does exactly the same as in the previous example, performing a basic HTTP authentication.
Make curl prompt for the password
curl -u 'andrea' https://greatapp.andreafortuna.org/login
If you skip the password part, curl will asks for the password on the command line.
This is useful if you don't want the password to be saved in the shell history.
Working with proxies
Use socks proxies
We can use the -x argument to set the proxy protocol, for examplesocks5:
curl -x socks5://andrea:strongpassword@proxy.com:8080 https://www.andreafortuna.org
In this example, the proxy username is set to andrea, proxy password to strongpassword, proxy hostname to proxy.com, and proxy port to 8080. After connecting to the proxy, curl then makes a GET request to https://www.andreafortuna.org.
The same syntax can be used for socks4 proxies:
curl -x socks4://andrea:strongpassword@proxy.com:8080 https://www.andreafortuna.org
Use an HTTP proxy
curl -x andrea:strongpassword@proxy.com:8080 https://www.andreafortuna.org
If you avoid to specify the protocol:// part of the proxy address, curl will assumes that an HTTP proxy is used.
Debugging requests
Set verbosity
curl -v https://www.andreafortuna.org
The -v argument asks curl to print detailed information about the request and the response.
Lines prefixed by > is the data sent to the server, lines prefixed by < is the data received from the server, and lines starting with * is misc information, such as connection information, SSL handshake information, and protocol information.
Detailed trace
curl --trace - https://www.andreafortuna.org
The --trace - argument enables a full trace hexdump of all incoming and outgoing data.
Adding --trace-time argument to the previous example makes curl add timestamps to full trace output
curl --trace - --trace-time https://www.andreafortuna.org
Print only specific debug messages
curl -i https://www.andreafortuna.org
By default, curl prints the response body to the screen: adding the -i argument makes it also print response headers.
If you need, you can also print only the response headers:
curl -s -o /dev/null -D - https://www.andreafortuna.org
The -s argument makes curl silent and hides errors and progress bar, then -o /dev/null makes curl ignore the response body, and -D - prints response headers to stdout.
Further, you can also print only the request headers:
curl -v -s -o /dev/null --stderr - https://www.andreafortuna.org | grep '^>'
A little more tricky: the -v argument enables the verbose output, the -s argument makes curl silent, then the -o /dev/null argument makes curl ignore the output from the server , the --stderr - argument redirects stderr to stdout , and finally print all lines that begin with > (that contain request headers) using grep.
And finally, you can print only the response code:
curl -w '%{response_code}' -s -o /dev/null https://www.andreafortuna.org
The -w argument makes curl print the extra information %{response_code} ( the response code of the request ) after the request has completed.
To make curl only print the code and not the content or other information, we also use -s to silence curl and -o /dev/null that ignores the response output.