An intro to cURL: The basics of the transfer tool

cURL is a transfer tool used to transfer data from or to a server. It supports various internet protocols of transfer, including:

• DICT
• FILE
• FTP, FTPS
• GOPHER
• HTTP, HTTPS
• IMAP, IMAPS
• LDAP, LDAPS
• POP3, POP3S
• RTMP, RTSP
• SCP, SFTP
• SMB, SMBS
• SMTP, SMTPS
• TELNET and TFTP

We can perform useful tricks with cURL like proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer resume, Metalink, and more.

Important: cURL is powered by libcurl for all transfer-related features.

cURL is used from the command-line. It’s mostly commonly used to quickly test out APIs during software development. I personally use cURL when I want to test out all my APIs in my Node.js server. It is a really handy tool for developers.

Postman is cool, but cURL is super-cool. – Chidume Nnamdi

In this post, we will go through basic cURL functions and options. We will also learn how we can use cURL to perform GET, POST requests on an API endpoint.

cURL options

--request or -X

--request and -X specify a custom request method you can use when communicating with the HTTP server. The specified request method will be used instead of the method otherwise used (which defaults to GET).

To perform a POST request:

curl --request POST

To perform a GET request:

curl --request GET

If we don’t specify the request via --request, it will default to GET.

--url</code

This specifies the URL that we will be fetching or transferring data to.

From the cURL docs:

Specify a URL to fetch. This option is mostly handy when you want to specify URL(s) in a config file.

If the given URL is missing a scheme name (such as “http://” or “ftp://”, etc) then cURL will make a guess based on the host.

If the outermost subdomain name matches DICT, FTP, IMAP, LDAP, POP3, or SMTP, then that protocol will be used. Otherwise, HTTP will be used.

For example, if you want to perform a GET request on your local server at localhost:3000, you’ll want to set the --url to be localhost:3000:

curl --request GET \
    --url http://localhost:3000

To perform a POST at the same URL:

curl --request POST \
    --url http://localhost:3000

Note: The backslash \ is used to separate options in cURL.

For an external API, it’s still the same thing.

Let’s say you want to fetch a list of movies from https://moviesdb.com/movies/all.

curl --request GET \
    --url https://moviesdb.com/movies/all

The list of all movies in moviesdb will be fetched and printed.

--header or -H

This option is used to set headers to requests.

From the cURL docs:

Extra header to include in the request when sending HTTP to a server. You may specify any number of extra headers. Note that if you should add a custom header that has the same name as one of the internal ones curl would use, your externally set
header will be used instead of the internal one.

This mirrors what we do in normal programming, especially in JavaScript using XMLHttpRequest:

const xhttp = new XMLHttpRequest()

xhttp.setHeader("Content-Type", "application/json")

Headers are used to communicate to the web server the type of data coming in, or the type of data to expect. The data type sent should be of the same type specified in the headers.

We can use headers for CORS permission, and even for permission on certain types of request methods. There is a plethora of stuff we can do with headers.

So in cURL, we can set headers using the --header option:

curl --request POST \
  --url http://localhost:5000/api/user \
  --header 'content-type: application/json'

Here, we are making a POST request to http://localhost:5000/api/user endpoint and we are telling the server the data we are sending is a JSON data type via the --header content-type: application/json'.

--data or -d

This option is used to send data to an HTTP server. This is mostly used in POST requests because we send data to the server we want to add to the datastore. So in cURL, we have the option of specifying our data to POST by setting the --data option.

From the cURL manual:

Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button.

Here’s an example:

curl --request POST \
  --url http://localhost:5000 \
  --header 'content-type: application/json' \
  --data '{"name":"Arrow","description":"bad movie","rating":"7.0","image":"michaeljackson.png"}'

Here, we are performing a POST request to http://localhost:5000 endpoint.

We set the data to send to the server in the --data option, which is: '{"name":"Arrow","description":"bad movie","rating":"7.0","image":"michaeljackson.png"}.

This is a movie object we want to add to the http://localhost:5000 datastore.

Using cURL with Node.js/Express.js

Let’s see how we can set up a Node.js/Express.js server and use cURL to test the endpoints:

// server.js

/** require dependencies */
const express = require("express")
const cors = require('cors')
const bodyParser = require('body-parser')
const helmet = require('helmet')

const app = express()

let port = 5000 || process.env.PORT

/** set up middlewares */
app.use(cors())
app.use(bodyParser.json({limit: '50mb'}))
app.use(helmet())

let users = []
let currId  Date.now()

app.get("api/user/:id", (req, res, next) => {
        const userId = req.params.id
        const users = users.filter(user => user.id === userId)
        if(users.length > 0)
            res.send({ user: users[0] })
        else
            res.send({mesg: "No user found."})
        next()
})

app.get("api/users", (req, res, next) => {
    res.send({ users })
    next()
})

app.post("api/user", (req, res, next) => {
        let bdy = req.body
        bdy = { id: ++currId, ...bdy }
        users.push(bdy)
        res.send({ user: bdy })
        next()    
})

/** start server */
app.listen(port, () => {
    console.log(`Server started at port: ${port}`);
});

We have small API endpoints here:

• GEt api/users
• GET api/user/:id
• POST api/user

It fetches all users, a specific user by its id, and can add a new user.

Fire up the server:

node server.js

Server started at port: 5000

Now, to perform a POST request at api/user, we will set the --url to http://localhost:5000/api/user, and --request to POST, the --header to content-type: application/json.

We want to add the user:

name: "Chidume",
age: 28

So the data will be '{"name": "Chidume", "age": "28"}'.

So the cURL command will be:

curl --request POST \
  --url http://localhost:5000/api/user \
  --header 'content-type: application/json' \
  --data '{"name": "Chidume", "age": "28"}'

{ id: 5122435464, name: "Chidume", age: 28 }

We see the result from the server: the user is added to the datastore with its id.

Let’s add another user:

curl --request POST \
  --url http://localhost:5000/api/user \
  --header 'content-type: application/json' \
  --data '{"name": "Nnamdi", "age": "21"}'

{ id: 5122435465, name: "Nnamdi", age: 21 }

We added another user, “Nnamdi”, and we can see the result from the server with its id.

Now, let’s query for “Chidume” via the api/user/:id. Here, :id will be the id 5122435464. So the cURL will be:

curl --request GET \
  --url http://localhost:5000/api/user/5122435464

{ user: { id: 5122435464, name: "Chidume", age: 28 } }

The result from the server returns the user with id 5122435464 which is “Chidume”.

To query for “Nnamdi”:

curl --request GET \
  --url http://localhost:5000/api/user/5122435465

{ user: { id: 5122435465, name: "Nnamdi", age: 21 } }

Ta-da!!

Now let’s query for a non-existent user:

curl --request GET \
  --url http://localhost:5000/api/user/2000000000


{ mesg: "No user found." }

That’s the result set on the server to return when we hit a non-existent user.

Now, let’s query ALL users:

curl --request GET \
  --url http://localhost:5000/api/users


{ users: [ { id: 5122435465, name: "Nnamdi", age: 21 }, { id: 5122435464, name: "Chidume", age: 28 } ] }

As you can see, all the users in the server are returned.

Conclusion

In this tutorial, we won’t over the basic options of cURL, with examples on how to use each of them, We then we developed a Node/Express server with three endpoints to demonstrate how we can use the cURL options addressed here to perform GET/POST request with cURL to the server.

There are many options to use with cURL. What we addressed here are the basic options you can use to quickly test your API endpoints.

If you have any questions regarding this or anything I should add, correct, or remove, feel free to comment, email, or DM me.