Shalitha Suranga Programmer | Author of Neutralino.js and Jerverless

Using JSON in Go: A guide with examples

10 min read 3003

Using JSON in Go: A guide with examples

Golang (also known as Go) is a statically typed, compiled programming language with C-like syntax. Go provides a minimal grammar for general-purpose programming with just 25 keywords.

Nowadays, programmers use Go to build developer tools, cloud computing tools, CLI programs, and desktop and web applications. Go is very popular for building high-performance software systems where concurrency plays a key role.

Go developers often have to work with JSON content. For example, we often have to read JSON files to populate Go objects and write JSON files from existing Go objects. Like any other modern programming language, Go provides a standard library module to work with JSON structures.

In this tutorial, I will explain how to work with JSON in Go with practical examples. Also, I will explain some advanced concepts such as custom JSON encoding and decoding.

Go encoding/json package

Go provides the encoding/json package to handle JSON content via the encoding namespace of the standard library. The encoding/json package offers API functions for generating JSON documents from Go objects — and populating Go objects from the JSON documents. Also, it allows you to customize the JSON-to-Go and Go-to-JSON translation process.

The JSON specification supports both formatted and inline (minified) documents. Therefore, the Go encoding/json package lets developers generate both formatted and minified JSON documents.

Marshaling: Converting Go objects to JSON

What is marshaling in Go?

Encoding Go objects to JSON format is known as marshaling. We can use the Marshal function to convert Go objects to JSON. The Marshal function comes with the following syntax.

func Marshal(v interface{}) ([]byte, error)

It accepts an empty interface. In other words, you can provide any Go data type to the function — an integer, float, string, struct, map, etc. — because all Go data type definitions can be represented with empty interfaces. It returns two values: a byte slice of the encoded JSON and error.

Marshaling simple objects

As mentioned above, we can generate JSON with primitive Go data types. For example, you can convert a Go string to a JSON string.

But because converting primitives is not helpful in real-world software development projects, let’s get started by converting some simple objects. The following code snippet will encode JSON from a map data structure.

We made a custom demo for .
No really. Click here to check it out.

package main
import (
    "fmt"
    "encoding/json"
)
func main() {
    fileCount := map[string]int{
        "cpp": 10,
        "js": 8,
        "go": 10,
    }
    bytes, _ := json.Marshal(fileCount)
    fmt.Println(string(bytes))
}

Here we used string() to convert bytes to a string. Go encodes map data structures to JSON key-value objects. Once you run the above code, you will get an output as shown below.

The output when converting a Go map data struct to a JSON key-value object

You can also encode JSON from a struct, as shown in the following example code.

package main
import (
    "fmt"
    "encoding/json"
)
type Book struct {
    Title string
    Author string
    Year int
}
func main() {
    myBook := Book{"Hello Golang", "John Mike", 2021}
    bytes, _ := json.Marshal(myBook)
    fmt.Println(string(bytes))
}

Here, we have to begin struct field names with an uppercase English letter to make those fields exportable to other packages. If your struct contains a field that starts with a lowercase letter, the encoding/json package won’t include the particular field for the encoding process without throwing any error.

The above code will output the following JSON structure.

{"Title":"Hello Golang","Author":"John Mike","Year":2021}

Marshaling complex objects

In previous examples, we encoded JSON from Go objects like simple maps and structs. If you try to encode integer arrays, string arrays, and primitive variables,  Go will produce simple JSON structures for those elements.

But most of the time, we have to generate JSON files from complex objects in our Go programs, such as for product lists, product details, and various nested data records.

First, let’s encode JSON from a product list. Look at the following example code.

package main
import (
    "fmt"
    "encoding/json"
)
type Seller struct {
    Id int
    Name string
    CountryCode string
}
type Product struct {
    Id int
    Name string
    Seller Seller
    Price int
}
func main() {
    products := []Product{
        Product {
            Id: 50,
            Name: "Writing Book",
            Seller: Seller {1, "ABC Company", "US"},
            Price: 100,
        },
        Product {
            Id: 51,
            Name: "Kettle",
            Seller: Seller {20, "John Store", "DE"},
            Price: 500,
        },
    }
    bytes, _ := json.Marshal(products)
    fmt.Println(string(bytes))
}

The above code initializes a product list with two items. The Product struct has a Seller struct as a nested object — and all products are placed in a products slice. Next, we sent the final product list to the Marshal function to encode it to JSON structure.

Once you run the above code snippet, you will get the following output.

[{"Id":50,"Name":"Writing Book","Seller":{"Id":1,"Name":"ABC Company","CountryCode":"US"},"Price":100},{"Id":51,"Name":"Kettle","Seller":{"Id":20,"Name":"John Store","CountryCode":"DE"},"Price":500}]

As you can see, Go encodes JSON from any complex Go data structure. But now, we have two problems when we look at the above output:

  • The output JSON structure’s keys always start with an upper case English letter — how can we rename JSON fields?
  • When we encode large complex structures, the output becomes literally unreadable — how can we prettify the JSON output?

The Go encoding/json package answers the above questions with additional library features.

Marshaling features

Go offers several features to improve and customize JSON outputs via additional API functions and struct tags.

Renaming fields

You have to begin struct field declarations with an uppercase English letter to let the JSON package access them. As a result, you will always get uppercase English letters for JSON keys. The Go encoding/json package allows developers to rename JSON fields as they wish via JSON struct tags.

The following code snippet encodes JSON from a product object with snake case JSON keys.

package main
import (
    "fmt"
    "encoding/json"
)
type Seller struct {
    Id int `json:"id"`
    Name string `json:"name"`
    CountryCode string `json:"country_code"`
}
type Product struct {
    Id int `json:"id"`
    Name string `json:"name"`
    Seller Seller `json:"seller"`
    Price int `json:"price"`
}
func main() {
    book := Product{
        Id: 50,
        Name: "Writing Book",
        Seller: Seller {1, "ABC Company", "US"},
        Price: 100,
    }
    bytes, _ := json.Marshal(book)
    fmt.Println(string(bytes))
}

As you can see, the above code uses struct tags to rename each exported field. The struct tag is not a mandatory element for the JSON encoding process  —  it’s an optional element to rename a particular struct field during the JSON encoding process.

You will get the following output once you execute the above code.

{"id":50,"name":"Writing Book","seller":{"id":1,"name":"ABC Company","country_code":"US"},"price":100}

Generating JSON with indentation (pretty-print)

The Marshal function generates minimal inline JSON content without any formatting. You can use the MarshalIndent function to encode well-readable JSON with indentation. The following code generates prettified JSON for the previous struct object.

package main
import (
    "fmt"
    "encoding/json"
)
type Seller struct {
    Id int `json:"id"`
    Name string `json:"name"`
    CountryCode string `json:"country_code"`
}
type Product struct {
    Id int `json:"id"`
    Name string `json:"name"`
    Seller Seller `json:"seller"`
    Price int `json:"price"`
}
func main() {
    book := Product{
        Id: 50,
        Name: "Writing Book",
        Seller: Seller {1, "ABC Company", "US"},
        Price: 100,
    }
    bytes, _ := json.MarshalIndent(book, "", "\t")
    fmt.Println(string(bytes))
}

Once you run the above code, it will print a formatted JSON structure, as shown below.

The formatted JSON structure output

Here we used the tab character (\t) for indentation. You can use four spaces, two spaces, eight spaces, etc., for formatting according to your requirement.

Ignoring specific fields from JSON output

Earlier, we used struct tags to rename JSON keys. We can use struct tags to omit specific fields too. If we use json:”-” as the tag, the related struct field won’t be used for encoding. Also, if we use ,omitempty inside the struct tag name string, the related field won’t be used for encoding if the value is empty.

The following code omits the product identifier for encoding. Also, it omits empty country code values from the output.

package main
import (
    "fmt"
    "encoding/json"
)
type Seller struct {
    Id int `json:"id"`
    Name string `json:"name"`
    CountryCode string `json:"country_code,omitempty"`
}
type Product struct {
    Id int `json:"-"`
    Name string `json:"name"`
    Seller Seller `json:"seller"`
    Price int `json:"price"`
}
func main() {
    products := []Product{
        Product {
            Id: 50,
            Name: "Writing Book",
            Seller: Seller {Id: 1, Name: "ABC Company", CountryCode: "US"},
            Price: 100,
        },
        Product {
            Id: 51,
            Name: "Kettle",
            Seller: Seller {Id: 20, Name: "John Store"},
            Price: 500,
        },
    }
    bytes, _ := json.MarshalIndent(products, "", "\t")
    fmt.Println(string(bytes))
}

The above code produces the following output. Note that it doesn’t contain the product identifier and the second item’s country code key.

The output omitting the product identifier and the second object's country code key

Unmarshaling: Converting JSON to Go objects

In the Go environment, the JSON document decoding process is called unmarshaling. We can use the Unmarshal function to convert JSON to Go objects. The Unmarshal function comes with the following syntax.

func Unmarshal(data []byte, v interface{}) error

It accepts two parameters: a bytes slice of the JSON content and an empty interface reference. The function may return an error if there is an error occurred during the decoding process. The Unmarshal function doesn’t create and return Go objects ,  so we have to pass a reference to store the decoded content.

Unmarshaling simple JSON structures

Similar to JSON marshaling, we can unmarshal Go primitive data types such as integers, strings, floats, and booleans. But again, because primitive unmarshaling has no real use cases in most software development projects, we’ll start by decoding the following key-value structure to a Go struct.

{
    "width": 500,
    "height": 200,
    "title": "Hello Go!"
}

The following code decodes the above JSON structure into a struct.

package main
import (
    "fmt"
    "encoding/json"
)
type Window struct {
    Width int `json:"width"`
    Height int `json:"height"`
    Title string `json:"title"`
}
func main() {
    jsonInput := `{
        "width": 500,
        "height": 200,
        "title": "Hello Go!"
    }`
    var window Window
    err := json.Unmarshal([]byte(jsonInput), &window)

    if err != nil {
        fmt.Println("JSON decode error!")
        return
    }

    fmt.Println(window) // {500 200 Hello Go!}
}

The jsonInput variable holds the JSON content as a multiline string. Therefore, we had to convert it to a bytes slice before passing it to the Unmarshal function with the byte[]() type conversion syntax. Here, we checked the value of the returned error object to detect parsing errors.

The above JSON tags are optional in this scenario because the Go encoding/json package typically maps JSON fields to struct fields with a case-insensitive matching.

Similarly, we can decode JSON structures to Go maps, too. Look at the following example code.

package main
import (
    "fmt"
    "encoding/json"
)
func main() {
    jsonInput := `{
        "apples": 10,
        "mangos": 20,
        "grapes": 20
    }`
    var fruitBasket map[string] int
    err := json.Unmarshal([]byte(jsonInput), &fruitBasket)

    if err != nil {
        fmt.Println("JSON decode error!")
        return
    }

    fmt.Println(fruitBasket) // map[apples:10 grapes:20 mangos:20]
}

Unmarshaling complex data structures

Previous unmarshaling examples showed you how to decode simple JSON structures. We often have to decode complex nested JSON structures in our software development projects. The following example demonstrates how you can populate Go objects from a JSON-formatted products list.

package main
import (
    "fmt"
    "encoding/json"
)
type Product struct {
    Id int `json:"id"`
    Name string `json:"name"`
    Seller struct {
        Id int `json:"id"`
        Name string `json:"name"`
        CountryCode string `json:"country_code"`
    } `json:"seller"`
    Price int `json:"price"`
}
func main() {
    jsonInput := `[
    {
        "id":50,
        "name":"Writing Book",
        "seller":{
            "id":1,
            "name":"ABC Company",
            "country_code":"US"
        },
        "price":100
    },
    {
        "id":51,
        "name":"Kettle",
        "seller":{
            "id":20,
            "name":"John Store",
            "country_code":"DE"
        },
        "price":500
    }]
    `
    var products []Product
    err := json.Unmarshal([]byte(jsonInput), &products)

    if err != nil {
        fmt.Println("JSON decode error!")
        return
    }

    fmt.Println(products)
    // [{50 Writing Book {1 ABC Company US} 100} {51 Kettle {20 John Store DE} 500}]
}

As shown in the above code, we need to define a struct first by inspecting the JSON input. This process is a time-consuming task when you work with large complex JSON structures. Therefore, you can use an online tool like JSON-to-Go to create struct definitions based on JSON input.

There is also is a way to access parsed values without creating structs in Go. You can access any value dynamically by creating map[string]interface{} type objects for JSON objects, but this approach leads to very complex, lower-quality source code.

However, you can check the dynamic JSON value access for experimental purposes with the following example code. But, don’t use this approach in production software systems without creating appropriate Go structures because it creates complex and hard-to-test code.

package main
import (
    "fmt"
    "encoding/json"
)
func main() {
    jsonInput := `[
    {
        "id":50,
        "name":"Writing Book",
        "seller":{
            "id":1,
            "name":"ABC Company",
            "country_code":"US"
        },
        "price":100
    },
    {
        "id":51,
        "name":"Kettle",
        "seller":{
            "id":20,
            "name":"John Store",
            "country_code":"DE"
        },
        "price":500
    }]
    `
    var objMap []map[string]interface{}
    err := json.Unmarshal([]byte(jsonInput), &objMap)

    if err != nil {
        fmt.Println("JSON decode error!")
        return
    }

    fmt.Println("Price of the second product:", objMap\[1\]["price"])
}

The above code prints the price of the second product item without Go structs.

Reading JSON files from the filesystem

We used hardcoded JSON strings with the previous examples for demonstration. But, in practice, we load JSON strings from various sources: from the filesystem, over the internet, over local network locations, etc. Most programmers typically use JSON format to store configuration details on the filesystem.

Let’s write some Go code to read and decode JSON data from a file and convert it into Go objects. First, create a file named config.json and input the following content.

{
    "timeout": 50.30,
    "pluginsPath": "~/plugins/",
    "window": {
        "width": 500,
        "height": 200,
        "x": 500,
        "y": 500
    }
}

Now, run the following code to decode the above JSON document into a suitable struct.

package main
import (
    "fmt"
    "io/ioutil"
    "encoding/json"
)
type Config struct {
    Timeout float32
    PluginsPath string
    Window struct {
        Width int
        Height int
        X int
        Y int
    }
}
func main() {

    bytes, err := ioutil.ReadFile("config.json")

    if err != nil {
        fmt.Println("Unable to load config file!")
        return
    }

    var config Config
    err = json.Unmarshal(bytes, &config)

    if err != nil {
        fmt.Println("JSON decode error!")
        return
    }

    fmt.Println(config) // {50.3 ~/plugins/ {500 200 500 500}}
}

The above code reads the JSON file content as bytes with the ioutil.ReadFile function and decodes data records to the Config struct.

Writing JSON files to the filesystem

In previous examples, we printed the encoded JSON content to the console via the Println function. Now we can save those JSON strings as files with the ioutil.WriteFile function, as shown below.

package main
import (
    "io/ioutil"
    "encoding/json"
)
type Window struct {
    Width int `json:"width"`
    Height int `json:"height"`
    X int `json:"x"`
    Y int `json:"y"`
}
type Config struct {
    Timeout float32 `json:"timeout"`
    PluginsPath string `json:"pluginsPath"`
    Window Window `json:"window"`
}
func main() {
    config := Config {
        Timeout: 40.420,
        PluginsPath: "~/plugins/etc",
        Window: Window {500, 200, 20, 20},
    }
    bytes, _ := json.MarshalIndent(config, "", "  ")
    ioutil.WriteFile("config.json", bytes, 0644)
}

The above code writes config.json by encoding the config object as a JSON object. Here we used two spaces for indentation and converted struct fields to camel case JSON keys by using struct tags.

Custom marshaling and unmarshaling

The Go json package is very flexible, and it offers features to override the encoding and decoding process. These features are helpful when you need to transform JSON data records from one format to another during the encoding/decoding process.

Custom marshaling

Assume that you are writing a contacts management app in Go, and you are offering a feature for all users to download a list of contacts in JSON format. Assume that you cannot let non-admin users see all email IDs due to a security policy. In this scenario, you can customize the JSON encoding process via the custom marshaling feature in the Go json package, as shown below.

package main
import (
    "fmt"
    "encoding/json"
    "strings"
)
type Person struct {
    Name string `json:"name"`
    Age int `json:"age"`
    Email string `json:"-"`
}
func main() {
    persons := []Person {
            Person {"James Henrick", 25, "[email protected]"},
            Person {"David Rick", 30, "[email protected]"},
    }
    bytes, _ := json.MarshalIndent(persons, "", "  ")
    fmt.Println(string(bytes))
}
func (p *Person) MarshalJSON() ([]byte, error) {
    type PersonAlias Person
    return json.Marshal(&struct{
        *PersonAlias
        Email string `json:"email"`
    }{
        PersonAlias: (*PersonAlias)(p),
        Email: strings.Repeat("*", 4) + "@mail.com", // alter email
    })
}

The above code outputs all contact details, but it alters the original email addresses due to the security policy. Note that here we need to create another type (alias) from the Person type because if we try to call Marshal function for the original Person type, the program will enter an infinite loop due to the recursive implementation of the encoding process. Once you run the above code snippet, you will see an output like below.

The output for our custom marshaling example

Custom unmarshaling

The Go json package lets you customize the JSON decoding process too. Assume that you need to process a JSON configuration file and need to transform some values during the decoding process. Assume that one configuration field says the temperature in Kelvin, but you need to store the specific value in Celsius.

Look at the following code that implements custom unmarshaling.

package main
import (
    "fmt"
    "encoding/json"
)
type Config struct {
    FunctionName string 
    Temperature float32
}
func main() {
    jsonInput := `{
        "functionName": "triggerModule",
        "temperature": 4560.32
    }`
    var config Config
    err := json.Unmarshal([]byte(jsonInput), &config)

    if err != nil {
        fmt.Println("JSON decode error!")
        return
    }

    fmt.Println(config) // {triggerModule 4287.17}
}
func (c *Config) UnmarshalJSON(data []byte) error {
    type ConfigAlias Config
    tmp := struct {
        Temperature float32
        *ConfigAlias
    }{
        ConfigAlias: (*ConfigAlias)(c),
    }
    if err := json.Unmarshal(data, &tmp); err != nil {
        return err
    }
    c.Temperature = tmp.Temperature - 273.15
    return nil
}

The above code decodes JSON by converting the temperature field’s value from Kelvin to Celsius. Here also we need to create another type (alias) to avoid the infinite loop, similar to custom marshaling.

Conclusion

In this tutorial, we discussed JSON encoding (marshaling) and decoding (unmarshaling) in Go with practical examples. JSON is a widely used, language-independent encoding format. Therefore, almost all Go-based web frameworks internally handle JSON encoding and decoding. For example, the Gin HTTP framework allows you to directly send a struct to API functions without marshaling manually, using the json package.

However, you can use the Go json package with your Go programs without consuming third-party libraries because the json package is a part of the standard library. Also, there are some faster alternatives (according to this benchmark) for the Go json package. But, the json package is a part of the standard library and maintained by the Go development team. Therefore, the Go development team will improve the performance of the encoding/json package with upcoming releases.

: Full visibility into your web apps

LogRocket is a frontend application monitoring solution that lets you replay problems as if they happened in your own browser. Instead of guessing why errors happen, or asking users for screenshots and log dumps, LogRocket lets you replay the session to quickly understand what went wrong. It works perfectly with any app, regardless of framework, and has plugins to log additional context from Redux, Vuex, and @ngrx/store.

In addition to logging Redux actions and state, LogRocket records console logs, JavaScript errors, stacktraces, network requests/responses with headers + bodies, browser metadata, and custom logs. It also instruments the DOM to record the HTML and CSS on the page, recreating pixel-perfect videos of even the most complex single-page apps.

.
Shalitha Suranga Programmer | Author of Neutralino.js and Jerverless

Leave a Reply