5 structured logging packages for Go

On the surface, logging may appear to be a very simple task, requiring only that you write a message to the console or a file. But, when you follow logging best practices, you must consider log levels, structuring your logs, logging to different locations, adding the right amount of context to your logs, and more. Combined, all of these details make logging a complex task.

The idea behind structured logging is for your log entries to have a consistent format that can be easily processed, usually JSON, allowing you to filter log entries in a variety of ways. For example, you can search for logs that contain a specific user ID or error message, or you can filter out entries that pertain to a service. When your logs are structured, it will also be easy to derive relevant metrics from them, like billing information.

​​Why you may consider structured logging packages

​​You should think about structured logging packages for a variety of reasons:

1. ​​Because the Golang inbuilt logging library produces unstructured logs, tracking down logs is difficult and time-consuming
2. ​​Structured logging packages allow you to add more fields to logs in order to query them and make debugging easier
3. ​​It saves time when troubleshooting since structured logs are formatted in JSON, which makes them easier to read, query, and store

When the inbuilt logging library in Go is sufficient

Golang has an inbuilt logging library called log that comes with the default logger, which writes to standard error and adds the date for each logged message without the need for configuration. Log is useful for local development if you need quick feedback from your code.

It also allows you to make custom loggers and save logging outputs to files, despite the lack of log levels (such as debug, warning, or error) and lack of support for outputting logs in JSON format.

In this article, we’ll examine and compare five packages that make structured logging in Go a breeze. Let’s get started!

1. Zap

Zap is a popular structured logging library for Go. Developed by Uber, Zap promises greater performance than other comparable logging packages, even the log package in the standard library.

Zap provides two separate loggers, Logger for situations where performance is critical, and SugaredLogger, which prioritizes ergonomics and flexibility, while still providing a fast speed.

In the example below, we use an instance of the zap.SugaredLogger struct to log a message when the program is executed, producing a structured JSON output that contains the log level information, timestamp, file name, line number, and the log message:

package main

import (
    "log"

    "go.uber.org/zap"
)

func main() {
    logger, err := zap.NewProduction()
    if err != nil {
        log.Fatal(err)
    }

    sugar := logger.Sugar()

    sugar.Info("Hello from zap logger")
}

// Output:
// {"level":"info","ts":1639847245.7665887,"caller":"go-logging/main.go:21","msg":"Hello from zap logger"}

By either modifying the encoder configuration or creating your own from scratch, you can customize the exact fields that you want to appear in the logger. For example, you can change the ts field to timestamp and use a more human-friendly date format by setting the following configuration options:

func main() {
    loggerConfig := zap.NewProductionConfig()
    loggerConfig.EncoderConfig.TimeKey = "timestamp"
    loggerConfig.EncoderConfig.EncodeTime = zapcore.TimeEncoderOfLayout(time.RFC3339)

    logger, err := loggerConfig.Build()
    if err != nil {
        log.Fatal(err)
    }

    sugar := logger.Sugar()

    sugar.Info("Hello from zap logger")
}

// Output:
// {"level":"info","timestamp":"2021-12-18T18:21:34+01:00","caller":"go-logging/main.go:23","msg":"Hello from zap logger"}

If you need to add additional structured context to your logs, you can use any SugaredLogger method that ends with w, like Infow, Errorw, Fatalw, and more. The SugaredLogger type also provides the ability to log a templated message through its printf-style methods, including Infof, Errorf, and Fatalf:

sugar.Infow("Hello from zap logger",
  "tag", "hello_zap",
  "service", "logger",
)

// Output:
// {"level":"info","timestamp":"2021-12-18T18:50:25+01:00","caller":"go-logging/main.go:23","msg":"Hello from zap logger","tag":"hello_zap","service":"logger"}

When logging in a performance-sensitive portion of your application, you can switch to the standard, faster Logger API at any time by calling DeSugar() on a SugaredLogger. However, after doing so, you’ll only be able to add additional structured context to your logs using explicitly typed fields, as follows:

l := sugar.Desugar()

l.Info("Hello from zap logger",
  zap.String("tag", "hello_zap"),
  zap.Int("count", 10),
)

2. Zerolog

Zerolog is a dedicated library for structured JSON logging. Zerolog is designed to prioritize performance using a simpler API; by default, a global logger is provided that you can use for simple logging. To access this logger, import the log subpackage, as shown below:

package main

import (
    "github.com/rs/zerolog"
    "github.com/rs/zerolog/log"
)

func main() {
    zerolog.SetGlobalLevel(zerolog.InfoLevel)

    log.Error().Msg("Error message")
    log.Warn().Msg("Warning message")
    log.Info().Msg("Info message")
    log.Debug().Msg("Debug message")
    log.Trace().Msg("Trace message")
}

// Output:
// {"level":"error","time":"2021-12-19T17:38:12+01:00","message":"Error message"}
// {"level":"warn","time":"2021-12-19T17:38:12+01:00","message":"Warning message"}
// {"level":"info","time":"2021-12-19T17:38:12+01:00","message":"Info message"}

Zerolog allows for seven log levels, ranging from trace, the least severe, to panic, the most severe. You can use the SetGlobalLevel() method to set your preferred logging level for the global logger. In the example above, the logging level is set to info, so only log events with levels greater than or equal to info will be written.

Zerolog also supports contextual logging. Through methods on the zerolog.Event type, which represents a log event, Zerolog makes it easy to add extra fields in each JSON log.

An instance of Event is created through one of the level methods on a Logger, like Error(), then finalized by Msg() or Msgf(). In the example below, we use the process to add context to a log event:

log.Info().Str("tag", "a tag").Int("count", 123456).Msg("info message")

// Output:
// {"level":"info","tag":"a tag","count":123456,"time":"2021-12-20T09:01:33+01:00","message":"info message"}

Logging errors can also be performed through a special Err() method on an Event, which adds an error field to the log message if the error is not nil. If you wish to change the name of this field to something other than error, set the zerolog.ErrorFieldName property as follows:

err := fmt.Errorf("An error occurred")

log.Error().Err(err).Int("count", 123456).Msg("error message")

// Output:
// {"level":"error","error":"An error occurred","count":123456,"time":"2021-12-20T09:07:08+01:00","message":"error message"}

You can check out the docs for more information on adding a stack track to your error logs.

Aside from the global logger, which is accessible through the log subpackage, you can also create other logger instances with custom settings. These loggers can be based on the global logger or another logger created through zerolog.New().

In the example below, we’ll add the name of the service to every log event created through the childLogger, which will help with filtering log events from a specific application in a log aggregation service:

chidLogger := log.With().Str("service", "foo").Logger()

chidLogger.Info().Msg("An info message")

// Output:
// {"level":"info","service":"foo","time":"2021-12-20T13:45:03+01:00","message":"An info message"}

3. Logrus

Logrus provides structured logging for Go applications through an API that is compatible with the standard library logger. If you’re already using the stdlib logpackage, but you need to structure your logs to scale your logging process, it’s easy to switch to Logrus. Simply alias the logrus package to log, as shown in the code below:

package main

import (
  log "github.com/sirupsen/logrus"
)

func main() {
  log.WithFields(log.Fields{
    "tag": "a tag",
  }).Info("An info message")
}

// Output:
// INFO[0000] An info message                               tag="a tag"

Unlike Zap and Zerolog, Logrus doesn’t output JSON by default, but you can easily change this through the SetFormatter() method. You can also change the output from the default standard error to any io.Writer, like an os.File. You can also change the default severity level, which ranges from trace to panic:

func main() {
    log.SetFormatter(&log.JSONFormatter{})
    log.SetOutput(os.Stdout)
    log.SetLevel(log.InfoLevel)

    log.WithFields(log.Fields{
        "tag": "a tag",
    }).Info("An info message")
}

// Output: {"level":"info","msg":"An info message","tag":"a tag","time":"2021-12-20T14:07:43+01:00"}

The standard text and JSON formatters support several options that you can configure to your heart’s content. You can also utilize one of the supported third-party formatters if they fit your needs better.

Contextual logging is supported in Logrus using the WithFields() method, as demonstrated in the previous code snippet. If you want to reuse fields between logging statements, you can save the return value of WithFields() in a variable. Subsequent logging calls made through that variable will output those fields:

childLogger := log.WithFields(log.Fields{
  "service": "foo-service",
})

childLogger.Info("An info message")
childLogger.Warn("A warning message")

// Output:
// {"level":"info","msg":"An info message","service":"foo-service","time":"2021-12-20T14:18:08+01:00"}
// {"level":"warning","msg":"A warning message","service":"foo-service","time":"2021-12-20T14:18:08+01:00"}

Although Logrus is competitive in terms of features in comparison to the other options on this list, it falls behind in performance. At the time of writing, Logrus is currently in maintenance mode, so it may not be the best option for new projects. However, it is certainly a tool I’ll be keeping an eye on.

4. apex/log

apex/log is a structured logging package for Go applications that is inspired by Logrus. The author, TJ Holowaychuk, created the package to simplify the Logrus API and provide more handlers for common use cases. Some of the default handlers include text, json, cli, kinesis, graylog, and elastic search. To view the entire list of default handlers, you can browse the handlers directory, and you can create custom handlers by satisfying the log handler interface.

The example below demonstrates apex/log’s basic features. We’ll use the built-in JSON handler that writes to the standard output, which could be any io.Writer. apex/log uses the WithFields() method to add context to log entries. You can also set up a custom logger through the Logger type, allowing you to configure the handler and log level:

package main

import (
    "os"

    "github.com/apex/log"
    "github.com/apex/log/handlers/json"
)

func main() {
    log.SetHandler(json.New(os.Stdout))

    entry := log.WithFields(log.Fields{
        "service":  "image-service",
        "type":     "image/png",
        "filename": "porsche-carrera.png",
    })

    entry.Info("upload complete")
    entry.Error("upload failed")
}

// Output:
// {"fields":{"filename":"porsche-carrera.png","service":"image-service","type":"image/png"},"level":"info","timestamp":"2022-01-01T11:48:40.8220231+01:00","message":"upload complete"}
// {"fields":{"filename":"porsche-carrera.png","service":"image-service","type":"image/png"},"level":"error","timestamp":"2022-01-01T11:48:40.8223257+01:00","message":"upload failed"}

The apex/log package was designed with log centralization in mind. You can marshal and unmarshal JSON log entries from multiple services as is, without having to process each log entry separately due to differences in field names.

apex/log facilitates this action by placing context fields in a fields property instead of collapsing them at the root level of the JSON object, as in Logrus. This simple change makes it possible to seamlessly utilize the same handlers on the producer side and consumer side:

package main

import (
    "os"

    "github.com/apex/log"
    "github.com/apex/log/handlers/cli"
)

func main() {
    logger := log.Logger{
        Handler: cli.New(os.Stdout),
        Level:   1, // info
    }

    entry := logger.WithFields(log.Fields{
        "service":  "image-service",
        "type":     "image/png",
        "filename": "porsche-carrera.png",
    })

    entry.Debug("uploading...")
    entry.Info("upload complete")
    entry.Error("upload failed")
}

// Output:
// • upload complete           filename=porsche-carrera.png service=image-service type=image/png
// ⨯ upload failed             filename=porsche-carrera.png service=image-service type=image/png

5. Log15

Log15 aims to produce logs that are easily readable by both humans and machines, making it easy to follow best practices. The Log15 package uses a simplified API that forces you to log only key/value pairs in which keys must be strings, while values can be any type you desire. It also defaults its output formatting to logfmt, but this can easily be changed to JSON:

package main

import (
    log "github.com/inconshreveable/log15"
)

func main() {
    srvlog := log.New("service", "image-service")

    srvlog.Info("Image upload was successful", "name", "mercedes-benz.png", "size", 102382)
}

// Output:
// INFO[01-01|13:18:24] Image upload was successful              service=image-service name=mercedes-benz.png size=102382

When creating a new Logger, you may add contextual fields that will be included in each log entry produced by the logger. The provided log level methods like Info() and Error() also provide an easy way to add contextual information after the compulsory first argument, which is the log message. To change the handler used for writing logs, call the SetHandler() method on a Logger.

The handlers provided by Log15 are composable, so you can combine them to create a logging structure that suits your application. For example, in addition to logging all entries to the standard output, you can log errors and higher levels to a file in JSON format:

package main

import (
    log "github.com/inconshreveable/log15"
)

func main() {
    srvlog := log.New("service", "image-service")

    handler := log.MultiHandler(
        log.LvlFilterHandler(log.LvlError, log.Must.FileHandler("image-service.json", log.JsonFormat())),
        log.CallerFileHandler(log.StdoutHandler),
    )

    srvlog.SetHandler(handler)

    srvlog.Info("Image upload was successful")
    srvlog.Error("Image upload failed")
}

// Output:
// INFO[01-01|13:49:29] Image upload was successful              service=image-service caller=main.go:17
// EROR[01-01|13:49:29] Image upload failed                      service=image-service caller=main.go:18

The MultiHandler() method is used to dispatch each log entry to all registered handlers.

In our example, the LvlFilterHandler() writes JSON-formatted entries with a severity level of error or higher to a file. CallerFileHandler adds a caller field to the log output, which contains the line number and file of the calling function. CallerFileHandler wraps the StdoutHandler so that entries are subsequently printed to the standard output after modification.

In addition to CallerFileHandler(), the CallerFuncHandler() and CallerStackHandler() methods are provided for adding the calling function name and a stack trace to each log output, respectively.

If you need a function that is not provided by any of the default handlers, you can also create your own handler by implementing the Handler interface.

Performance comparison

Using the benchmarking suite in the Zap repository, the following results were observed:

Logging a message and ten fields:

Library	Time	Bytes allocated	Objects allocated
Zerolog	767 ns/op	552 B/op	6 allocs/op
:zap: zap	848 ns/op	704 B/op	2 allocs/op
:zap: zap (sugared)	1363 ns/op	1610 B/op	20 allocs/op
Logrus	5661 ns/op	6092 B/op	78 allocs/op
apex/log	15332 ns/op	3832 B/op	65 allocs/op
Log15	20657 ns/op	5632 B/op	93 allocs/op

Logging a message with a logger that already has ten fields of context:

Library	Time	Bytes allocated	Objects allocated
Zerolog	52 ns/op	0 B/op	0 allocs/op
:zap: zap	283 ns/op	0 B/op	0 allocs/op
:zap: zap (sugared)	337 ns/op	80 B/op	2 allocs/op
Logrus	4309 ns/op	4564 B/op	63 allocs/op
apex/log	13456 ns/op	2898 B/op	51 allocs/op
Log15	14179 ns/op	2642 B/op	44 allocs/op

Logging a static string without any context or printf-style templating:

Library	Time	Bytes allocated	Objects allocated
Zerolog	50 ns/op	0 B/op	0 allocs/op
:zap: zap	236 ns/op	0 B/op	0 allocs/op
Standard library	453 ns/op	80 B/op	2 allocs/op
:zap: zap (sugared)	337 ns/op	80 B/op	2 allocs/op
Logrus	1244 ns/op	1505 B/op	27 allocs/op
apex/log	2751 ns/op	584 B/op	11 allocs/op
Log15	5181 ns/op	1592 B/op	26 allocs/op

As you can see, Zerolog and Zap are the two most performant solutions at the time of writing. To verify these results, you should run the benchmarking suite on your machine with the latest versions of each library.

Conclusion

In this article, we’ve examined five libraries for implementing a structured logging approach in Go applications, including Zap, Zerolog, Logrus, apex/log, and Log15. Each library provides features like JSON logging, log levels, the ability to log to several locations, and more, making them suitable logging solutions for any project.

If performance is a deciding factor, you can’t go wrong with Zerolog or non-sugared Zap. Otherwise, I recommend choosing the library with the best API for your project. Thanks for reading, and happy coding!