Introduction to Go Modules

I’ve seen many people online talk about liking Go and using it, but being confused by its dependency system, called Go modules. This blog post aims to provide a simple introduction with examples. It focuses mostly on Unix-based systems like Linux and macOS over Windows.

This post does not cover all possible ways of using Go modules. It’s just a simple introduction with the most common use cases.

Background knowledge

You should know about Semantic Versioning. Pretty much all Go modules are expected to follow it, and it will help you when upgrading modules or releasing your own.

Setup

I assume you already have Go installed. Go modules have preliminary support since Go 1.11, but Go module support improves with with each release. I’d recommend installing at least Go 1.13, but install the latest version if you can.

To begin the setup, you need to be aware of some environment variables. By default GOPATH is set to ~/go, and GOBIN is set to ~/go/bin. I would not recommend changing this. There is also GOMODCACHE (added in Go 1.15) which defaults to GOPATH[0]/pkg/mod, which would be ~/go/pkg/mod if you don’t change any variables. It holds the cache of modules you’ve downloaded.

There is one other variable, GO111MODULE. This variable controls whether Go modules are enabled and being used to manage dependencies. If it’s set to auto or on, you should be good. If it’s set to off, then you must change it. If you’re using a Go version earlier than 1.13, you must set it to on, as the older auto behavior is not appropriate for this tutorial.

To check your current setup, you can run go env and see all the variables Go is working with.

Theory

You can skip this section if you just want to start using Go modules. But this might be helpful for people coming from other languages like Python or Rust.

Most programming languages handle dependencies by having a official registry/repository of packages that users can upload to. Think of PyPI, CPAN, NPM, crates.io, and more. Go takes a much more decentralized approach. Modules can exist anywhere on the Web, and they are defined with an HTTP(S) path.

For example, a valid module path would be github.com/username/repo. It could also be mydomain.com/foo. In either case, to download the module Go will first try to download from a proxy (GOPROXY), and then fallback to looking at the HTML of the URL provided to find where the code lives. For more info on this process and the HTML required, you can read more here.

You will not need to bother with any of this for most Git hosting providers. GitHub, Gitlab, Gitea, etc., will all create the needed HTML for you, and you can just specify a module path like github.com/username/repo.

Creating a module

First, your module needs a folder. To keep things simple, the path to this folder should double as its module path. So if you’re using GitHub, your username is user, and your project is called foo, your project should live at $GOPATH/src/github.com/user/foo, which by default will be ~/go/src/github.com/user/foo.

$ mkdir -p $GOPATH/src/github.com/user/foo
$ cd $GOPATH/src/github.com/user/foo
$ go mod init
go: creating new go.mod: module github.com/user/foo

If you’re really against this layout, you’re welcome to create your projects anywhere. The only difference for this entire tutorial is that you run go mod init differently. You will have to specify the module path, since Go can’t infer it.

$ go mod init github.com/user/foo
go: creating new go.mod: module github.com/user/foo

Running go mod init will create a go.mod file in the project folder. This is how Go declares its dependencies – with a go.mod file, and also a go.sum file that we’ll see later. You should always be uploading these files with the rest of your code.

$ cat go.mod
module github.com/user/foo

go 1.15

Besides declaring your dependencies, the go.mod file also tells Go that this project really is a module, and gives it its path.

Most of the time you shouldn’t need to edit go.mod or go.sum by hand. Instead you should let it be changed by running commands.

Adding dependencies

Alright, now you have a project, presumably with some code, and a go.mod file. Time to add some dependencies. The main commands for that are go get and go mod tidy.

go get github.com/bar/baz         # Latest version
go get github.com/bar/baz@master  # Latest commit on master
go get github.com/bar/baz@e0ffd3a # The module at commit e0ffd3a
go get github.com/bar/baz@v1.2.3  # Version 1.2.3 (using git tags)

go get accepts some other flags that I don’t need to cover here. One flag to avoid is -u, as it updates all the downstream dependencies of a module (not changing a major version), which could cause breakage.

My suggestion would be to use the commit or tag syntax as much as possible, so that you know explicitly what version of dependencies you’re adding, and to not use any flags.

The other important command after adding dependencies is go mod tidy. This command cleans up your go.mod and go.sum files, removing unused entries and adding missing ones. It’s good to run this right after adding a dependency, and before committing your code. You can run this command multiple times without any worry, if everything is in order than it will just change nothing.

import (
    "github.com/bar/baz"
)

func main() {
    baz.Something()
}

Dependencies beyond v1

This is the part that gets a bit trickier. In Go, the module path changes with every major version update. So the path for v1.0.0 of baz, all the way to v1.99.99 of baz is still github.com/bar/baz. But once baz v2.0.0 is released, the module path is now github.com/bar/baz/v2. Note that’s not @v2, but /v2.

This happens because major versions are supposed to represent breaking changes. So it’s important to allow your project to be able to use v2, while one of your dependencies still uses v1. This is achieved by making the module paths different.

The only changes you need to make is to use a slightly different go get command, and to update your code import paths.

go get github.com/bar/baz/v2         # Latest version of v2
go get github.com/bar/baz/v2@e0ffd3a # The module at commit e0ffd3a (must be a commit after v2 release)
go get github.com/bar/baz/v2@v2.2.3  # A specific v2 version

import (
    "github.com/bar/baz/v2" // Uses the new path
)

func main() {
    baz.Something() // Module name is still 'baz', not 'v2' or something
}

Publishing a module

The simplest way to publish a module is to use git, and a git host. Examples include GitHub, Gitlab, sr.ht, etc. Upload your Go code there, and make sure it has at least a go.mod file. Now anyone can use it as a module!

You should probably have versioning though, and you can do that with git tags, or by using the releasing system on your git host. For example, on GitHub you can click the Releases tab, and then “Draft a new release”.

git tag v1.2.3  # Whatever version this commit is
git push --tags

Major version updates to your module

So, you’ve been working on a v1 module for a while, but now you need to make a breaking change. This means you need to release a new major version.

Keep in mind, this is not required from v0 to v1, read about Semantic Versioning to know why. But if you’re already on v1 and want to make a breaking change, then read on.

There are a couple different ways to do this. Assuming you don’t need compatibility with Go versions older than 1.9.7 or 1.10.3, than this method should work fine. If you do, you can read more here.

$ cat go.mod
module github.com/user/foo/v2

go 1.15

2. Update all your internal import statements to use the new /v2 path. Anytime you import your own module in your code, you need to update that import.

# Commit and push, and then...
git tag v2.0.0
git push --tags

Sources

There is also an official technical reference, which should be considered the definitive/primary source for any info. It applies to Go 1.15 and later.