Golds, an experimental Go local docs server, Go docs generation tool, and code reader. NEW!
-- show type implementation relations --
-- smooth code view experiences --
-- and more... --

Type-Unsafe Pointers

We have learned Go pointers from the article pointers in Go. From that article, we know that, comparing to C pointers, there are many restrictions made for Go pointers. For example, Go pointers can't participate arithmetic operations, and for two arbitrary pointer types, it is very possible that their values can't be converted to each other.

The pointers explained in that article are called type-safe pointers actually. Although the restrictions on type-safe pointers really make us be able to write safe Go code with ease, they also make some obstacles to write efficient code for some scenarios.

In fact, Go also supports type-unsafe pointers, which are pointers without the restrictions made for safe pointers. Type-unsafe pointers are also called unsafe pointers in Go. Go unsafe pointers are much like C pointers, they are powerful, and also dangerous. For some cases, we can write more efficient code with the help of unsafe pointers. On the other hand, by using unsafe pointers, it is easy to write bad code which is too subtle to detect in time.

Another big risk of using unsafe pointers comes from the fact that the unsafe mechanism is not protected by the Go 1 compatibility guidelines. Code depending on unsafe pointers works today could break since a later Go version.

If you really desire the code efficient improvements by using unsafe pointers for any reason, you should not only know the above mentioned risks, but also follow the instructions written in the official Go documentation and clearly understand the effect of each unsafe pointer use, so that you can write safe Go code with unsafe pointers.

About the unsafe Standard Package

Go provides a special kind of types for unsafe pointers. We must import the unsafe standard package to use unsafe pointers. The unsafe.Pointer type is defined as
type Pointer *ArbitraryType

Surely, it is not a usual type definition. Here the ArbitraryType just hints that a unsafe.Pointer value can be converted to any safe pointer values in Go (and vice versa). In other words, unsafe.Pointer is like the void* in C language.

Go unsafe pointers mean the types whose underlying types are unsafe.Pointer.

The zero values of unsafe pointers are also represented with the predeclared identifier nil.

The unsafe standard package also provides three functions. Note, An example of using the three functions.
package main

import "fmt"
import "unsafe"

func main() {
	var x struct {
		a int64
		b bool
		c string
	}
	const M, N = unsafe.Sizeof(x.c), unsafe.Sizeof(x)
	fmt.Println(M, N) // 16 32

	fmt.Println(unsafe.Alignof(x.a)) // 8
	fmt.Println(unsafe.Alignof(x.b)) // 1
	fmt.Println(unsafe.Alignof(x.c)) // 8

	fmt.Println(unsafe.Offsetof(x.a)) // 0
	fmt.Println(unsafe.Offsetof(x.b)) // 8
	fmt.Println(unsafe.Offsetof(x.c)) // 16
}

An example which demostrates the last note mentioned above.
package main

import "fmt"
import "unsafe"

func main() {
	type T struct {
		c string
	}
	type S struct {
		b bool
	}
	var x struct {
		a int64
		*S
		T
	}

	fmt.Println(unsafe.Offsetof(x.a)) // 0
	
	fmt.Println(unsafe.Offsetof(x.S)) // 8
	fmt.Println(unsafe.Offsetof(x.T)) // 16
	
	// This line compiles, for c can be reached
	// without implicit pointer indirections.
	fmt.Println(unsafe.Offsetof(x.c)) // 16
	
	// This line doesn't compile, for b must be
	// reached with the implicit pointer field S.
	//fmt.Println(unsafe.Offsetof(x.b)) // error
	
	// This line compiles. However, it prints
	// the offset of field b in the value x.S.
	fmt.Println(unsafe.Offsetof(x.S.b)) // 0
}

Please note, the print results shown in the comments are for the standard Go compiler version 1.15 on Linux AMD64 architecture.

The three functions provided in the unsafe package don't look much dangerous. The signatures of these functions are very impossible to be changed in future Go 1 versions. Rob Pike even ever made a proposal to move the three functions to elsewhere. Most of the unsafty of the unsafe package comes from unsafe pointers. They are as dangerous as C pointers, what is Go safe pointers always try to avoid.

Unsafe Pointers Related Conversion Rules

Currently (Go 1.15), Go compilers allow the following explicit conversions.

By using these conversions, we can convert a safe pointer value to an arbitrary safe pointer type.

However, although these conversions are all legal at compile time, not all of them are valid (safe) at run time. These conversions defeat the memory safety the whole Go type system (except the unsafe part) tries to maintain. We must follow the instructions listed in a later section below to write valid Go code with unsafe pointers.

Some Facts in Go We Should Know

Before introducing the valid unsafe pointer use patterns, we need to know some facts in Go.

Fact 1: unsafe pointers are pointers and uintptr values are integers

Each of non-nil safe and unsafe pointers references another value. However uintptr values don't reference any values, they are just plain integers, though often each of them stores an integer which can be used to represent a memory address.

Go is a language supporting automatic garbage collection. When a Go program is running, Go runtime will check which memory blocks are not used by any value any more and collect the memory allocated for these unused blocks, from time to time. Pointers play an important role in the check process. If a memory block is unreachable from (referenced by) any values still in using, then Go runtime thinks it is an unused value and it can be safely garbage collected.

As uintptr values are integers, they can participate arithmetic operations.

The example in the next subsection shows the differences between pointers and uintptr values.

Fact 2: unused memory blocks may be collected at any time

At run time, the garbage collector may run at an uncertain time, and each garbage collection process may last an uncertain duration. So when a memory block becomes unused, it may be collected at an uncertain time.

For example:
import "unsafe"

// Assume createInt will not be inlined.
func createInt() *int {
	return new(int)
}

func foo() {
	p0, y, z := createInt(), createInt(), createInt()
	var p1 = unsafe.Pointer(y)
	var p2 = uintptr(unsafe.Pointer(z))

	// At the time, even if the address of the int
	// value referenced by z is still stored in p2,
	// the int value has already become unused, so
	// garbage collector can collect the memory
	// allocated for it now. On the other hand, the
	// int values referenced by p0 and p1 are still
	// in using.

	// uintptr can participate arithmetic operations.
	p2 += 2; p2--; p2--

	*p0 = 1                         // okay
	*(*int)(p1) = 2                 // okay
	*(*int)(unsafe.Pointer(p2)) = 3 // dangerous!
}

In the above example, the fact that value p2 is still in using can't guarantee that the memory block ever hosting the int value referenced by z has not been garbage collected yet. In other words, when *(*int)(unsafe.Pointer(p2)) = 3 is executed, the memory block may be collected, or not. It is dangerous to dereference the address stored in value p2 to an int value, for it is possible that the memory block has been already reallocated for another value (even for another program).

Fact 3: the addresses of some values might change at run time

Please read the article memory blocks for details (see the end of the hyperlinked section). Here, we should just know that when the size of the stack of a goroutine changes, the memory blocks allocated on the stack will be moved. In other words, the addresses of the values hosted on these memory blocks will change.

Fact 4: the life range of a value at run time may be not as large as it looks in code

In the following example, the fact value t is still in using can't guarantee that the values referenced by value t.y are still in using.

type T struct {
	x int
	y *[1<<23]byte
}

func bar() {
	t := T{y: new([1<<23]byte)}
	p := uintptr(unsafe.Pointer(&t.y[0]))

	... // use T.x and T.y

	// A smart compiler can detect that the value
	// t.y will never be used again and think the
	// memory block hosting t.y can be collected now.

	// Using *(*byte)(unsafe.Pointer(p))) is
	// dangerous here.

	// Continue using value t, but only use its x field.
	println(t.x)
}

Fact 5: *unsafe.Pointer is a general safe pointer type

Yes, *unsafe.Pointer is a safe pointer type. Its base type is unsafe.Pointer. As it is a safe pointer, according the conversion rules listed above, it can be converted to unsafe.Pointer type, and vice versa.

For example:
package main

import "unsafe"

func main() {
	x := 123                // of type int
	p := unsafe.Pointer(&x) // of type unsafe.Pointer
	pp := &p                // of type *unsafe.Pointer
	p = unsafe.Pointer(pp)
	pp = (*unsafe.Pointer)(p)
}

How to Use Unsafe Pointers Correctly?

The unsafe standard package documentation lists six unsafe pointer use patterns. Following will introduce and explain them one by one.

Pattern 1: convert a *T1 value to unsafe Pointer, then convert the unsafe pointer value to *T2.

As mentioned above, by using the unsafe pointer conversion rules above, we can convert a value of *T1 to type *T2, where T1 and T2 are two arbitrary types. However, we should only do such conversions if the size of T1 is no smaller than T2, and only if the conversions are meaningful.

As a result, we can also achieve the conversions between type T1 and T2 by using this pattern.

One example is the math.Float64bits function, which converts a float64 value to an uint64 value, without changing any bit in the float64 value. The math.Float64bits function does reverse conversions.
func Float64bits(f float64) uint64 {
	return *(*uint64)(unsafe.Pointer(&f))
}

func Float64frombits(b uint64) float64 {
	return *(*float64)(unsafe.Pointer(&b))
}

Please note, the return result of the math.Float64bits(aFloat64) function call is different from the result of the explicit conversion uint64(aFloat64).

In the following example, we use this pattern to convert a []MyString slice to type []string, and vice versa. The result slice and the original slice share the underlying elements. Such conversions are impossible through safe ways,
package main

import (
	"fmt"
	"unsafe"
)

func main() {
	type MyString string
	ms := []MyString{"C", "C++", "Go"}
	fmt.Printf("%s\n", ms)  // [C C++ Go]
	// ss := ([]string)(ms) // compiling error
	ss := *(*[]string)(unsafe.Pointer(&ms))
	ss[1] = "Rust"
	fmt.Printf("%s\n", ms) // [C Rust Go]
	// ms = []MyString(ss) // compiling error
	ms = *(*[]MyString)(unsafe.Pointer(&ss))
}

A practice by using the pattern is to convert a byte slice, which will not be used after the conversion, to a string, as the following code shows. In this conversion, a duplication of the underlying byte sequence is avoided.
func ByteSlice2String(bs []byte) string {
	return *(*string)(unsafe.Pointer(&bs))
}

This is the implementation adopted by the String method of the Builder type supported since Go 1.10 in the strings standard package. The size of a byte slice is larger than a string, and their internal structures are similar, so the conversion is valid.

The converse, converting a string to a byte slice in the similar way, is invalid, for the size of a string is smaller than a byte slice.
func String2ByteSlice(s string) []byte {
	return *(*[]byte)(unsafe.Pointer(&s)) // dangerous!
}

In the pattern 6 section below, a valid implementation to do the same job is introduced.

Pattern 2: convert unsafe pointer to uintptr, then use the uintptr value.

This pattern is not very useful. Usually, we print the result uintptr values to check the memory addresses stored in them. However, there are other both safe and less verbose ways to this job. So this pattern is not much useful.

Example:
package main

import "fmt"
import "unsafe"

func main() {
	type T struct{a int}
	var t T
	fmt.Printf("%p\n", &t)                          // 0xc6233120a8
	println(&t)                                     // 0xc6233120a8
	fmt.Printf("%x\n", uintptr(unsafe.Pointer(&t))) // c6233120a8
}

The outputted addresses might be different for each run.

Pattern 3: convert unsafe pointer to uintptr, do arithmetic operations with the uintptr value, then convert it back

In this pattern, the result unsafe pointer must continue to point into the original allocated memory block. For example:
package main

import "fmt"
import "unsafe"

type T struct {
	x bool
	y [3]int16
}

const N = unsafe.Offsetof(T{}.y)
const M = unsafe.Sizeof(T{}.y[0])

func main() {
	t := T{y: [3]int16{123, 456, 789}}
	p := unsafe.Pointer(&t)
	// "uintptr(p)+N+M+M" is the address of t.y[2].
	ty2 := (*int16)(unsafe.Pointer(uintptr(p)+N+M+M))
	fmt.Println(*ty2) // 789
}

Please note, in this specified example, the conversion unsafe.Pointer(uintptr(p) + N + M + M) shouldn't be split into two lines, like the following code shows. Please read the comments in the code for the reason.
func main() {
	t := T{y: [3]int16{123, 456, 789}}
	p := unsafe.Pointer(&t)
	// ty2 := (*int16)(unsafe.Pointer(uintptr(p)+N+M+M))
	addr := uintptr(p) + N + M + M
	// Now the t value becomes unused, its memory may be
	// garbage collected at this time. So the following
	// use of the address of t.y[2] may become invalid
	// and dangerous! 
	// Another potential danger is, if some operations
	// make the stack grow or shrink here, then the
	// address of t might change, so that the address
	// saved in addr will become invalid (fact 3),
	// though this danger doesn't exist for this
	// specified example.
	ty2 := (*int16)(unsafe.Pointer(addr))
	fmt.Println(*ty2)
}

Such bugs are very subtle and hard to detect, which is why the uses of unsafe pointers are dangerous.

The intermediate uintptr value may also participate in &^ bitwise clear operations to do address alignment, as long as the result unsafe pointer and the original one point into the same allocated memory block.

Another detail which should be also noted is that, it is not recommended to store the end boundary of a memory block in a pointer (either safe or unsafe one). Doing this will prevent another memory block which closely follows the former memory block from being garbage collected. Please read this FAQ item to get more explanations.

Pattern 4: convert unsafe pointers to uintptr values as arguments of syscall.Syscall calls.

From the explanations for the last pattern, we know that the following function is dangerous.
// Assume this function will not inlined.
func DoSomething(addr uintptr) {
	// read or write values at the passed address ...
}

The reason why the above function is dangerous is that the function itself can't guarantee the memory block at the passed argument address is not garbage collected yet. If the memory block is collected or is reallocated for other values, then the operations made in the function body are dangerous.

However, the prototype of the Syscall function in the syscall standard package is as
func Syscall(trap, a1, a2, a3 uintptr) (r1, r2 uintptr, err Errno)

How does this function guarantee that the memory blocks at the passed addresses a1, a2 and a3 are still not garbage collected yet within the function internal? The function can't guarantee this. In fact, compilers will make the guarantee. It is the privilege of calls to syscall.Syscall alike functions.

We can think that, compilers will automatically insert some instructions for each of the unsafe pointer arguments who are converted to uintptr, like the third argument in the following syscall.Syscall call, to prevent the memory block referenced by that argument from being garbage collected or moved.

Please note that, before Go 1.15, it was okay the conversion expressions uintptr(anUnsafePointer) act as sub-expressions of the talked arguments. Since Go 1.15, the requirement becomes a bit stricter: the talked arguments must present exactly as the uintptr(anUnsafePointer) form.

The following call is safe:
syscall.Syscall(SYS_READ, uintptr(fd),
			uintptr(unsafe.Pointer(p)), uintptr(n))
But the following calls are dangerous:
u := uintptr(unsafe.Pointer(p))
// At this time, the value referenced by p might
// have become unused and been collected already,
// or the address of the value has changed.
syscall.Syscall(SYS_READ, uintptr(fd), u, uintptr(n))

// Arguments must be in the "uintptr(anUnsafePointer)"
// form. In fact, the call was safe before Go 1.15.
// But Go 1.15 changes the rule a bit.
syscall.Syscall(SYS_XXX, uintptr(uintptr(fd)),
			uint(uintptr(unsafe.Pointer(p))), uintptr(n))

Again, never use this pattern when calling other functions.

Pattern 5: convert the uintptr result of reflect.Value.Pointer or reflect.Value.UnsafeAddr method call to unsafe pointer

The methods Pointer and UnsafeAddr of the Value type in the reflect standard package both return a result of type uintptr instead of unsafe.Pointer. This is a deliberate design, which is to avoid converting the results of calls (to the two methods) to any safe pointer types without importing the unsafe standard package.

The design requires the return result of a call to either of the two methods must be converted to an unsafe pointer immediately after making the call. Otherwise, there will be small time window in which the memory block allocated at the address stored in the result might lose all references and be garbage collected.

For example, the following call is safe.
p := (*int)(unsafe.Pointer(reflect.ValueOf(new(int)).Pointer()))
On the other hand, the following call is dangerous.
u := reflect.ValueOf(new(int)).Pointer()
// At this moment, the memory block at the address
// stored in u might have been collected already.
p := (*int)(unsafe.Pointer(u))

Note: this pattern also applies to the syscall.Proc.Call and syscall.LazyProc.Call methods on Windows.

Pattern 6: convert a reflect.SliceHeader.Data or reflect.StringHeader.Data field to unsafe pointer, and the inverse.

For the same reason mentioned for the last subsection, the Data fields of the struct type SliceHeader and StringHeader in the reflect standard package are declared with type uintptr instead of unsafe.Pointer.

We convert a pointer to a string to a *reflect.StringHeader pointer value, so that we can manipulate the internal of the string. The same, we can convert a pointer to a slice to a *reflect.SliceHeader pointer value, so that we can manipulate the internal of the slice.

An example of using reflect.StringHeader:
package main

import "fmt"
import "unsafe"
import "reflect"

func main() {
	a := [...]byte{'G', 'o', 'l', 'a', 'n', 'g'}
	s := "Java"
	hdr := (*reflect.StringHeader)(unsafe.Pointer(&s))
	hdr.Data = uintptr(unsafe.Pointer(&a))
	hdr.Len = len(a)
	fmt.Println(s) // Golang
	// Now s and a share the same byte sequence, which
	// makes the bytes in the string s become mutable.
	a[2], a[3], a[4], a[5] = 'o', 'g', 'l', 'e'
	fmt.Println(s) // Google
}

An example of using reflect.SliceHeader:
package main

import (
	"fmt"
	"unsafe"
	"reflect"
	"runtime"
)

func main() {
	a := [6]byte{'G', 'o', '1', '0', '1'}
	bs := []byte("Golang")
	hdr := (*reflect.SliceHeader)(unsafe.Pointer(&bs))
	hdr.Data = uintptr(unsafe.Pointer(&a))

	hdr.Len = 2
	hdr.Cap = len(a)
	fmt.Printf("%s\n", bs) // Go
	bs = bs[:cap(bs)]
	fmt.Printf("%s\n", bs) // Go101
}

In general, we should only get a *reflect.StringHeader pointer value from an actual (already existed) string, or get a *reflect.SliceHeader pointer value from an actual (already existed) slice. We shouldn't do the contrary, such as creating a string from a StringHeader, or creating a slice from a SliceHeader. For example, the following code is dangerous.
var hdr reflect.StringHeader
hdr.Data = uintptr(unsafe.Pointer(new([5]byte)))
// Now the just allocated byte array has lose all
// references and it can be garbage collected now.
hdr.Len = 5
s := *(*string)(unsafe.Pointer(&hdr)) // dangerous!

The following is an example which shows how to convert a string to to a byte slice, by using the unsafe way. Different from the safe conversion from a string to to a byte slice, the unsafe way doesn't allocate a new underlying byte sequence for the result slice in each conversion.
package main

import (
	"fmt"
	"reflect"
	"runtime"
	"strings"
	"unsafe"
)

// The string argument passed to this funciton should
// not be used any more after the function exits.
func String2ByteSlice(str string) (bs []byte) {
	strHdr := (*reflect.StringHeader)(unsafe.Pointer(&str))
	sliceHdr := (*reflect.SliceHeader)(unsafe.Pointer(&bs))
	sliceHdr.Data = strHdr.Data
	sliceHdr.Cap = strHdr.Len
	sliceHdr.Len = strHdr.Len
	return
}

func main() {
	// str := "Golang"
	// For the official standard compiler, the above
	// line will make the bytes in str allocated on
	// an immutable memory zone.
	// So we use the following line instead.
	str := strings.Join([]string{"Go", "land"}, "")
	s := String2ByteSlice(str)
	fmt.Printf("%s\n", s) // Goland
	s[5] = 'g'
	fmt.Println(str) // Golang
}

The docs of the SliceHeader and StringHeader types in the reflect standard package are similar. The docs says the representations of the two struct types may change in a later release. So the above example may become invalid even if the unsafe rules keep unchanged. Fortunately, the current two available Go compilers (the standard Go compiler and the gccgo compiler) both recognize the representations of the two types declared in the reflect standard package.

It is also possible to convert a byte slice to a string by using a similar way. However, it is more recommended to use the way shown in pattern 1 to do this job.

Final Words

From the above content, we know that, for some cases, the unsafe mechanism can help us write more efficient Go code. However, it is very easy to introduce some subtle bugs which have very low possibilities to produce when using the unsafe mechanism. A program with these bugs may run well for a long time, but suddenly behave abnormally and even crash at a later time. Such bugs are very hard to detect and debug.

We should only use the unsafe mechanism when we have to, and we must use it with extreme care. In particular, we should follow the instructions described above.

And again, we should aware that the unsafe mechanism introduced above may change and even become invalid totally in later Go versions, though no evidences this will happen soon. If the unsafe mechanism rules change, the above introduced valid unsafe pointer use patterns may become invalid. So please keep it easy to switch back to the safe implementations for you code depending on the unsafe mechanism.

In the end, it is worth mentioning that a dynamic analysis compiler option -gcflags=all=-d=checkptr is supported since Go Toolchain 1.14. When this option is used, some (but not all) incorrect unsafe pointer uses will be detected at run time. Once such an incorrect use is detected, a panic will occur. (However, it is not recommended to use this option on Windows now.) Thanks to Matthew Dempsky for implementing this great feature!


The Go 101 project is hosted on Github. Welcome to improve Go 101 articles by submitting corrections for all kinds of mistakes, such as typos, grammar errors, wording inaccuracies, description flaws, code bugs and broken links.

If you would like to learn some Go details and facts every serveral days, please follow Go 101's official Twitter account: @go100and1.

The digital versions of this book are available at the following places:
Tapir, the author of Go 101, has spent 3+ years on writing the Go 101 book and maintaining the go101.org website. New contents will continue being added to the book and the website from time to time. Tapir is also an indie game developer. You can also support Go 101 by playing Tapir's games (made for both Android and iPhone/iPad):
Individual donations via PayPal and cryptocurrencies are welcome too:
Bitcoin: 1xucQbv5jujFPPwhyg395ri5yV71hx9g9
Ethereum: 0x5dc4aa2c2bbfaadae373dadcfca11b3358912212