Configuring the Go HTTP client

When building applications that communicate with outside services/products, we need a common means to establish an understandable connection on both ends. We use APIs to simplify and establish communication between the client and server.

In this tutorial, we’ll show you how to configure and establish a connection between client and server and make HTTP requests to endpoints that are exposed by other projects to provide our application with resources. Making request calls to an API means making an HTTP(s) request to a web server according to the API’s precise documentation.

The client is the host (e.g., the browser) that makes the request to a web server for a specific service or data through the HTTP protocol in the form of a URL and receives a response. The server is a remote computer that accepts and processes the request and sends the appropriate response data using the HTTP/HTTPS protocol.

Golang HTTP Client

The Go standard library provides excellent support for HTTP clients in the net/http package. Throughout this guide, we’ll explore all the configurations a Go program needs to make HTTP/HTTPS requests to external resources.

We’ll use the standard Go client from the net/http library to initialize an HTTP client by simply creating a variable of type http.Client.

// go/src/http-client/main.go
package main
import (
  "fmt"
  "io/ioutil"
  "net/http"
  "time"
)
func main() {
    c := http.Client{Timeout: time.Duration(1) * time.Second}
    resp, err := c.Get("https://go.dev/")
    if err != nil {
        fmt.Printf("Error %s", err)
        return
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)

    if err != nil {
      fmt.Printf("Error %s", err)
      return
    }

    fmt.Printf("Body : %s", body)
}

When creating an HTTP client, we can specify and pass in certain fields to configure the client-server connection.

In the above code snippet, we specified a Timeout field, which is of type time.Duration. When a client opens a connection to the server via HTTP, the server may take some time to respond to the request. This field enables us to specify a maximum waiting time to get a response from the server.

We can specify other fields within http.Client:

• Transport (type http.RoundTripper) — This customizes the process through which HTTP requests are handled and executed within our program
• CheckedRedirect (type func(req *Request, via []*Request)) — In case of redirection with a request, we can use this field to define a function within our program to handle cases of redirections with a request
• Jar (type CookieJar) — We can use this field to add cookies to the HTTP requests

In the above code, we defined the HTTP Client with the DefaultTransport, no CheckRedirect function, no cookies, and a timeout set to one second.

GET and POST requests

In the previous code block, we defined a GET request to a URL, https://go.dev/. We sent out a request to the web server and assigned the response and a possible error value to the variables, resp and err, respectively.

...
  resp, err := c.Get("https://go.dev/")

  if err != nil {
    fmt.Printf("Error %s", err)
    return
  }
...

The above code snippet is similar to the code for making a POST request to a URL. But in this case, we need to append the data we’re sending alongside the POST request within the body of the request to the web server.

...
  postData := bytes.NewBuffer([]byte(`{"post":"boom boom library"}`))
  resp, err := c.Post("https://go.dev/", "application/json", postData)

  if err != nil {
    fmt.Printf("Error %s", err)
    return
  }
...

In the above snippet, we created a new variable, postData (type *bytes.Buffer) to hold the data we want to send along with the request. Then, within the c.Post function, we passed the postData as an argument alongside the URL and content type of the data.

Why not simply put a JSON string as the postData? That’s because this argument must implement the interface io.Reader.

---

Over 200k developers use LogRocket to create better digital experiences

Learn more →

---

Now that we’ve seen how to make a GET or POST request, let’s quickly look at retrieving the response from resp:

...
  defer resp.Body.Close()
  body, err := ioutil.ReadAll(resp.Body)

  if err != nil {
    fmt.Printf("Error %s", err)
    return
  }

  fmt.Printf("Body : %s", body)
...

With the defer keyword, we scheduled a function call to resp.Body.Close to close the resp.Body, which is a stream of data returned from the request once the function returns. This is a necessary part of the program to avoid potential persistent connections to the server.

Adding headers to request

Let’s build methods for each type of request we want to make to the server. This might involve more code, but it gives us the flexibility to own our code. We can then easily append the headers we want alongside the request.

// go/src/http-client/main.go
package main

import (
    "fmt"
    "io/ioutil"
    "net/http"
    "time"
)

func main() {

  c := http.Client{Timeout: time.Duration(1) * time.Second}
  req, err := http.NewRequest("GET", "https://go.dev/", nil)
  if err != nil {
    fmt.Printf("error %s", err)
    return
  }
  req.Header.Add("Accept", `application/json`)

  resp, err := c.Do(req)
  if err != nil {
    fmt.Printf("Error %s", err)
    return
  }

  defer resp.Body.Close()
  body, err := ioutil.ReadAll(resp.Body)
  if err != nil {
    fmt.Printf("Error %s", err)
    return
  }

  fmt.Printf("Body : %s", body)
}

In the above code, we created a client and then, using the http.NewRequest method, we defined a new request. With the arguments, we specified the type of request we want.

The http.Request has a function signature as shown below:

(method, url string, body io.Reader) (*Request, error)

With the first parameter, we specify the method of the request. Then, we specify the URL in the second parameter and the body to hold the data — or nil in the case of a GET request because we do not have a body to send.

Next, we define the Header we want to append to the request, as shown below:

req.Header.Add("Accept", `application/json`)

We use Header fields to add and transmit an additional layer of information to the server about the request. The specification of HTTP 1/1 provides several Header fields:

• Content-Length is the size (in bytes) of the message sent. The default value is 2 if not specified
• User-Agent is the name and version of the program that sends the request. For example, curl/7.16.3 if we use curl to make the request. The default value is Go-http-client/1.1 if not specified
• Authorization provides credentials needed to make a successful request. Credentials can include API key, username/password, JWT, or others
• Accept-Encoding specifies which types of encoding are acceptable in the response. The default value is gzip if not specified
• Content-Type tells the server what type of media will transmitted in the request. The default value is application/json if not specified
• Accept specifies which media types are acceptable for the response.

The Header field of a request implements the type map\[string\][]string where the keys are strings and the values are slices of strings.

Authorizing your requests

The HTTP Authorization request header can provide credentials that the server uses to authenticate a user, allowing access to protected resources.

...
req, err = http.NewRequest("GET", "https://www.xxxx.xxx", nil)
req.Header.Add("Accept", `application/json`)
req.Header.Add("Authorization", fmt.Sprintf("token %s", os.Getenv("TOKEN"))
...

In the above code snippet, we retrieved the access token using the os package in our program. This is better than accessing the token directly. The Getenv method retrieves the environment variable named TOKEN and parses it.

We can also prepend GITHUB_TOKEN=XXX to our go command to pass an environment variable to our program before running it, as shown below:

$ GITHUB_TOKEN=xxxxx go run main.go

Conclusion

In this tutorial, we walked through a simple procedure to configure your HTTP client. Now you can start making API requests to outside resources from your application.

You can modify http.NewRequest with more methods, such as HEAD, PUT, PATCH, DELETE, etc., then pass in a body argument to the function when necessary, or nil when there’s no need for a body. Once the response is retrieved/available to our program, we can consume it within our project, depending on the use case.