Making HTTP requests in Rust with Reqwest

It feels like Rust gets more powerful by the day, thanks to its strong open source community. Rust has already become a viable option for full-stack applications (Rocket is an interesting example).

But with that boundless application comes the need for boundless data connections. So, how do we request data from somewhere?

It’s no fun keeping data dumps and CLI inputs on our machines. Let’s learn how Rust’s Reqwest library can:

• Make basic GET and POST requests
• Authenticate and request content types using headers
• Serialize JSON to usable, type-safe structs
• Help us build a bare-bones Spotify search client

What is Rust?

If you’re new to Rust, it’s an incredibly welcoming language for any programming background. I see it as the modern equivalent of C. But to make strict types and memory management more manageable, it adds niceties for human-readable compile-time errors and functional programming features like “match” expressions.

What’s more, Rust offers a thriving community of contributors and a plethora of excellent tutorials.

What is Reqwest?

The Reqwest library is built for fetching resources using the HTTP protocol. It offers both a simplified API to make get and post requests to a given URL, along with a fully featured Client module for applying headers, cookies, redirect policies, etc.

Reqwest follows Rust’s async protocol using “futures.” If you’re unfamiliar with Rust’s asynchronous programming story, it comes with two features:

• Flags to denote our asynchronous code blocks (the await keyword). If you’ve used the await keyword in JavaScript, it works similarly in Rust
• A runtime that recognizes these flags and executes our asynchronous program. Rust has adopted the async keyword for marking functions and block scopes that we should await responses from. However, Reqwest also relies on the Tokio runtime for queuing up async events efficiently

To learn more, you can also check Rust’s quaint asynchronous programming book.

Building an app with Reqwest

Disclaimer: this demo uses the Spotify API as documented in October 2021. Their API + authentication protocols could have changed since then! But don’t worry, the learnings from this demo should still apply.

For our project, we’ll use the Spotify API to build a simple search client to quickly grab song links to share with friends. We’re skipping the official authentication flow for simplicity, so if you want to follow along, head here to grab a sample API token you can use. Accounts are free to create if you need one.

Creating our first get Reqwest

Let’s jump into some basic get and post requests. First, create a new project with Cargo and add some dependencies to the cargo.toml:

[dependencies]
reqwest = { version = "0.11", features = ["json"] } # reqwest with JSON parsing support
futures = "0.3" # for our async / await blocks
tokio = { version = "1.12.0", features = ["full"] } # for our async runtime

Now, let’s give one of Spotify’s API endpoints a call:

use reqwest;

// tokio let's us use "async" on our main function
#[tokio::main]
async fn main() {
    // chaining .await will yield our query result
    let result = reqwest::get("https://api.spotify.com/v1/search").await
}

Pretty simple! Let’s see what a print gives us:

println!("{:?}", result)

// 👉 Ok(Response { url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("api.spotify.com")), port: None, path: "/v1/search", query: None, fragment: None }, status: 401, headers: {"www-authenticate": "Bearer realm=\"spotify\"", "access-control-allow-origin": "*", "access-control-allow-headers": "Accept, App-Platform, Authorization...

Hm, we received more data than we were expecting. How come? This is because the initial response of any request will be the server’s direct Response object.

If you’ve ever used the inspector tool in your browser to watch network activity, you’ve probably seen some of these fields before. Status is a useful field for matching on error conditions, for example, which brings us to the next section.

Matching on StatusCode

You’ll likely want to branch off to different behavior depending on the response. We can match on the request’s StatusCode for this:

match response.status() {
    reqwest::StatusCode::OK => {
        println!("Success! {:?}");
    },
    reqwest::StatusCode::UNAUTHORIZED => {
        println!("Need to grab a new token");
    },
    _ => {
        panic!("Uh oh! Something unexpected happened.");
    },
};

This will prove especially useful once we start deserializing valid responses later on, as errors often have a different response body compared to valid output.

Parsing the response body

If we want the actual body of the response, we’ll need to chain a second function that specifies what we’re parsing. Let’s try the .text() field first and process it as plaintext:

#[tokio::main]
async fn main() {
    let response = reqwest::get("https://api.spotify.com/v1/search")
        .await
        // each response is wrapped in a `Result` type
        // we'll unwrap here for simplicity
        .unwrap()
        .text()
        .await;
    println!("{:?}", response);
}
// 👉 Ok("{\n  \"error\": {\n    \"status\": 401,\n    \"message\": \"No token provided\"\n  }\n}")

Not much to see here! Because we haven’t provided an authentication token, we’re hit with a 401 for our search query, which should fall through to the reqwest::StatusCode::UNAUTHORIZED case in our match from earlier.

So how do we get a 200 status response? Let’s talk headers.

Specifying content types as headers

Let’s add a few headers to our query:

• content_type and accept to confirm we want a response as JSON
• authorization to pass our Spotify account token

We can’t pass these headers to reqwest::get directly. Instead, we’ll need to reach for the client module to chain our headers together. Let’s refactor our previous query to use client first:

#[tokio::main]
async fn main() {
    let client = reqwest::Client::new();
    let response = client
        .get("https://api.spotify.com/v1/search")
        // confirm the request using send()
        .send()
        .await
        // the rest is the same!
        .unwrap()
        .text()
        .await;
    println!("{:?}", response);
}

Now that we have a client, we can add our header config, like so:

let response = client
    .get("https://api.spotify.com/v1/search")
    .header(AUTHORIZATION, "Bearer [AUTH_TOKEN]")
    .header(CONTENT_TYPE, "application/json")
    .header(ACCEPT, "application/json")
    .send()
    ...

[AUTH_TOKEN] is your account’s OAuth token (grab one here). If we log our response now, we should see… a different error status!

Instead of a 401 authorization error, we should get an error 400 bad request. This is because we haven’t specified what we’re searching for yet, so let’s do that:

let url = format!(
    "https://api.spotify.com/v1/search?q={query}&type=track,artist",
    // go check out her latest album. It's 🔥
    query = "Little Simz"
);
// the rest is the same as before!
let client = reqwest::Client::new();
let response = client
    .get(url)
    .header(AUTHORIZATION, "Bearer [AUTH_TOKEN]")
    .header(CONTENT_TYPE, "application/json")
    .header(ACCEPT, "application/json")
    .send()
    .await
    .unwrap();
println!("Success! {:?}", response)
// 👉 Ok("{\n  \"artists\" : {\n    \"href\" : \"https://api.spotify.com/v1/search?query=Lil+Simz&type=artist&offset=0&limit=20\",\n    \"items\" : [ {\n      \"external_urls\"...

Deserializing to JSON

Now we’ve received our large blob of search results. How can we turn this into a useful data structure?

Well, let’s model the data we want to receive using nested structs. Structures vary wildly depending on your use case, but here’s the model we’ll need for Spotify’s search results:

use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug)]
struct ExternalUrls {
    spotify: String,
}
#[derive(Serialize, Deserialize, Debug)]
struct Artist {
    name: String,
    external_urls: ExternalUrls,
}
#[derive(Serialize, Deserialize, Debug)]
struct Album {
    name: String,
    artists: Vec<Artist>,
    external_urls: ExternalUrls,
}
#[derive(Serialize, Deserialize, Debug)]
struct Track {
    name: String,
    href: String,
    popularity: u32,
    album: Album,
    external_urls: ExternalUrls,
}
#[derive(Serialize, Deserialize, Debug)]
struct Items<T> {
    items: Vec<T>,
}
#[derive(Serialize, Deserialize, Debug)]
struct APIResponse {
    tracks: Items<Track>,
}

Note: Rust doesn’t support nested structs at the time of this writing, so we’ll break out each level into a separately named, serializable struct.

You’ll also notice Serialize derivations, which allow Reqwest to convert the raw API response into Rust-friendly types via Serde. Be sure to add Serde as a project dependency, like so:

[dependencies]
...
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"

Now we’re ready to parse our response. We should only attempt to parse when the API responds with a 200 status. Otherwise, we’ll hit issues parsing an error message to our APIResponse struct.

Let’s bring back that match syntax from earlier based on status code:

let response = client
    .get(url)
    ...
match response.status() {
    reqwest::StatusCode::OK => {
        // on success, parse our JSON to an APIResponse
        match response.json::<APIResponse>().await {
            Ok(parsed) => println!("Success! {:?}", parsed),
            Err(_) => println!("Hm, the response didn't match the shape we expected."),
        };
    }
    reqwest::StatusCode::UNAUTHORIZED => {
        println!("Need to grab a new token");
    }
    other => {
        panic!("Uh oh! Something unexpected happened: {:?}", other);
    }
};

With any luck, we should get a nice API output we can use. We’ve also filtered out keys on the API response we weren’t interested in by narrowing the keys on our structs.

If you’re used to JavaScript and TypeScript where unexpected object keys can slip in, this should come as a welcome addition!

Creating a CLI client

Let’s massage this data into something human-readable.

We’ll run a basic map over our tracks:

fn print_tracks(tracks: Vec<&Track>) {
    for track in tracks {
        println!("🔥 {}", track.name);
        println!("💿 {}", track.album.name);
        println!(
            "🕺 {}",
            track
                .album
                .artists
                .iter()
                .map(|artist| artist.name.to_string())
                .collect::<String>()
        );
        println!("🌎 {}", track.external_urls.spotify);
        println!("---------")
    }
}

Then call on this print function within our match:

match response.status() {
    reqwest::StatusCode::OK => {
        match response.json::<APIResponse>().await {
            Ok(parsed) => print_tracks(parsed.tracks.items.iter().collect()),
            Err(_) => println!("Hm, the response didn't match the shape we expected."),
        };
    }
...

We should also accept any query you could want as a CLI argument:

let args: Vec<String> = env::args().collect();
let search_query = &args[1];
let url = format!(
    "https://api.spotify.com/v1/search?q={query}&type=track,artist",
    query = search_query
);
...

Now, we can run something like this from our terminal for search output:

cargo run "Little Simz"
     Running `target/debug/spotify-search`
🔥 Venom
💿 Venom
🕺 Little Simz
🌎 https://open.spotify.com/track/4WaaWczlVb1UJ24LILsR4C
---------
🔥 Fear No Man
💿 Sometimes I Might Be Introvert
🕺 Little Simz
🌎 https://open.spotify.com/track/6bLkNijhsnr1MWYrO6XnRz
---------
🔥 Venom
💿 GREY Area
🕺 Little Simz
🌎 https://open.spotify.com/track/3A0ITFj6kbb9CggwtPe55f
---------
...

That’s it!

Conclusion

Hopefully, this tutorial showcased how powerful and simple the Reqwest library is. If you need some extended reading, Reqwest’s documentation offers some valuable examples for diving deeper, including:

• Setting cookies
• Exploring other HTTP protocol methods like POST and PUT
• Extended helpers for picking status codes and groups of status codes

If you’d like to explore the finished product, you can find this Spotify search on GitHub!