What’s new in Go 1.16

Go 1.16, the 17th major version of the Go programming language, has just been released. It is a significant upgrade that brings lots of long-awaited features and refinements to the language. Module-aware mode is enabled by default, Apple silicon support is live, native embedding of static assets is here, and the methods in the io/ioutil package have been reorganized so that it makes logical sense now. In this article, we’ll take a look at some of the highlights from this release.

Native support for Apple silicon

Since its inception, Go has prioritized portability amongst different operating systems and architectures and this is reflected in its support for a wide variety of operating system and architecture combinations.

In the past few months, the release of Apple’s first 64-bit ARM Mac has been one of the most dominant topics amongst developers due to its impressive leap in CPU, GPU, and battery performance. The Go project has promptly responded by adding native support for ARM Macs through the GOOS=darwin and GOARCH=arm64 environmental variables.

If you have an M1 Mac, you’ll now be able to build and run Go programs natively on your computer, and if you’re on a different operating system or an Intel-based Mac, you can target ARM Macs by setting the environmental variables above when building the binary for your program:

GOARCH=arm64 GOOS=darwin go build myapp

Native embedding of static files

One of the best things about Go is that compiled programs can be distributed and executed as a single dependency-free binary file. This advantage is somewhat offset when a program relies on static files such as HTML templates, database migration files, web application assets such as JavaScript, or images files like these files often have to be distributed with the binary unless they are embedded in the binary with the help of a third-party package such as pkger or packr. With the release of Go 1.16, it’s now possible to natively include static files in a Go binary through the new embed package.

Here’s the most basic example of how this feature works. Assuming you have a sample.txt file whose contents are shown below:

Hello from text file

And a main.go file in the same directory with the following contents:

package main

import (
    _ "embed"
    "fmt"
)

//go:embed sample.txt
var text string

func main() {
    fmt.Print(text)
}

The go:embed directive placed above the text variable instructs the compiler to embed the contents of the sample.txt file as a string into the text variable. If you build the program with go build and move the resulting binary to a different location, you will notice that executing it will print out the contents of the embedded file to the standard output. That’s because all the contents of the sample.txt file have been included inside of the binary so that it can be distributed as is:

$ mv main /tmp
$ cd /tmp
$ ./main
Hello from text file

For a more realistic example, let’s say we have a web application project with the following directory structure:

.
├── assets
│   ├── css
│   │   └── style.css
│   └── js
│       └── script.js
├── go.mod
├── index.html
├── main.go
└── random

We can embed all the files in the assets folder and index.html file like this:

package main

import (
    "embed"
    "net/http"
)

//go:embed assets/*
var assets embed.FS

//go:embed index.html
var html []byte

func main() {
    fs := http.FileServer(http.FS(assets))
    http.Handle("/assets/", fs)
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Add("Content-Type", "text/html")
        w.Write(html)
    })
    http.ListenAndServe(":8080", nil)
}

The FS type is useful for embedding a tree of files, such as a directory of web server assets, like in the above example. For embedding a single file like index.html, a variable of type string or []byte is best. If you build and execute the program, and navigate to http://localhost:8080 you will see the contents of the HTML file with the static assets correctly applied:

$ go version
go version go1.16rc1 linux/amd64
$ go build -o main
$ mv main /tmp
$ cd /tmp && ./main

You can download the contents of the index.html, style.css, and script.js file if you want to run the example locally. For more details, see the documentation for the new embed package.

Some gotchas

Before you can use the //go:embed directive, you must import the embed package. Failing this, you will get an error:

$ go run main.go
# command-line-arguments
./main.go:8:3: //go:embed only allowed in Go files that import "embed"

If you’re not directly using any exported identifies from embed, make sure you prefix the import statement with an underscore:

import (
    _ "embed"
)

Another thing to be aware of is that //go:embed only works on package-level variables. If you try to use it inside a function, your code won’t compile:

package main

import (
    _ "embed"
    "fmt"
)

func main() {
    //go:embed index.html
    var html string
    fmt.Println(html)
}

$ go run main.go
# command-line-arguments
./main.go:9:4: go:embed cannot apply to var inside func

Module-aware mode is enabled by default

The introduction of Go modules in Go 1.11 heralded a move away from GOPATH semantics for dependency management. In that initial release and Go 1.12, modules were still experimental and had to be activated with the environment variable GO111MODULE=on. Go 1.13 ensured that module-aware mode was automatically activated whenever a go.mod file is present in the current working directory or a parent directory even if the directory was within the GOPATH and this remained the case in Go 1.14 and 1.15.

With the release of Go 1.16, the GO111MODULE variable now defaults to on which means that module-aware mode is enabled by default regardless of whether a go.mod file is present in the current directory. If you want to revert to the previous behavior, set GO111MODULE to auto.

In other related changes, go build and go test will no longer modify the go.mod and go.sum files by default. Instead, an error will be reported if a module requirement or checksum needs to be added or updated. You can then use go mod tidy or go get to adjust the requirements accordingly.

The go install command is also now module-aware which means that it won’t affect the go.mod file in the current directory or any parent directory, if there is one. Also, it can now take a version number as a suffix. For example:

$ go install github.com/[email protected]

In Go 1.16, the use of go get to build and install packages has been deprecated in favor of go install. In a future release go get will not be able to build and install packages anymore but will act as it currently does with the -d flag enabled meaning it’ll adjust the current module’s dependencies without building packages. The -insecure or -i a flag has also been deprecated.

Package authors can now retract old versions

As of Go 1.16, a new retract directive will be available in go.mod files. This allows package authors to mark older package versions as insecure or broken or if a version was published unintentionally. Here’s how to use it:

module example

go 1.16

retract v1.1.1 // retract single version
retract [v1.1.1, v1.3.2] // closed interval, so anything between v1.1.1 and v1.3.2

The io/ioutil package is now deprecated

The entire ioutil package is now deprecated in Go 1.16 and its functions have been moved to other packages. To be clear, existing code that utilizes this package will continue to work, but you are encouraged to migrate to the new definitions in the io and os packages.

Migration of code using ioutil should be straightforward. A popular method in this package is the ReadAll() method which is often used to read the entire response body from an HTTP request into a slice of bytes. This method has been moved to the io package:

resp, err := http.Get(url)
if err != nil {
    return err
}

defer resp.Body.Close()

// old way: body, err := ioutil.ReadAll(resp.Body)
body, err := io.ReadAll(resp.Body)
if err != nil {
    return err
}

The full list of the new locations of exported io/ioutil methods is shown below:

• ioutil.Discard => io.Discard
• ioutil.NopCloser => io.NopCloser
• ioutil.ReadAll => io.ReadAll
• ioutil.ReadDir => os.ReadDir (returns a slice of os.DirEntry rather than a slice of fs.FileInfo)
• ioutil.ReadFile => os.ReadFile
• ioutil.TempDir => os.MkdirTemp
• ioutil.TempFile => os.CreateTemp
• ioutil.WriteFile => os.WriteFile

The io/fs package

Improvements to the Go standard library are not left out of this release with the addition of the io/fs and testing/testfs packages. These new packages make it easier to abstract a filesystem in tests which makes them more easily reproducible regardless of the operating system they are running on. Accessing files will also be much faster and you won’t have to clean up temporary files afterward.

Prior to Go 1.16, the task of mocking a filesystem often fell to the popular afero package which provides an interface type that must be satisfied to implement a real or simulated filesystem. It also provides some common implementations that provide this interface such as afero.MemMapFs which is a memory-backed filesystem that is useful for mocking in tests.

Unlike afero’s Fs interface which defines 13 methods at the time of writing, the FS interface provided by the io/fs package is quite simple:

type FS interface {
    Open(name string) (File, error)
}

All you need to do to implement this interface is an Open method that can open a file at a path, and return an object that implements the fs.File interface which is shown below:

type File interface {
    Stat() (FileInfo, error)
    Read([]byte) (int, error)
    Close() error
}

One thing you’ll notice from the above interface is the lack of methods that allow you to modify files. That’s because the io/fs package only provides a read-only interface for filesystems, unlike Afero which is more complete in that regard. The reason for this decision is that reading is easier to abstract compared to writing which is more involved.

All this is to say I think the design decision of limiting this proposal to read-only operations is a good one. In fact it was the key insight (by @robpike) that unlocked years of being stuck and let us make any progress defining this interface at all.

Notable mentions

The vet tool now provides a warning when an invalid call to testing.T or testing.B‘s Fatal, Fatalf, or FailNow methods is made from within a goroutine created during a test or benchmark. This is because these methods exit the goroutine instead of the test or benchmark function:

package main

import "testing"

func TestFoo(t *testing.T) {
    go func() {
        if true {
            t.Fatal("Test failed") // exits the goroutine instead of TestFoo
        }
    }()
}

Calls to the above methods can be replaced with t.Error() to signal the failure of the test and a return statement to exit the goroutine:

package main

import "testing"

func TestFoo(t *testing.T) {
    go func() {
        if true {
            t.Error("Test failed.")
            return
        }
    }()
}

There were also several minor updates and fixes to standard library packages. The full list of changes can be found in the release notes.

Conclusion

If you want to explore the full list of bug fixes and features that were included in this release, I encourage you to check out the list of closed issues in the Go 1.16 milestone on GitHub.

Thanks for reading, and happy coding!