unsafe Standard Package
unsafe standard package to use unsafe pointers. The unsafe.Pointer type is defined as
type Pointer *ArbitraryType
ArbitraryType just hints that a unsafe.Pointer value can be converted to any safe pointer values in Go (and vice versa). In other words, unsafe.Pointer is like the void* in C language.
unsafe.Pointer.
nil.
unsafe standard package has already provided three functions.
func Alignof(variable ArbitraryType) uintptr, which is used to get the address alignment of a value. Please notes, the aligns for struct-field values and non-field values of the same type may be different, though for the standard Go compiler, they are always the same. For the gccgo compiler, they may be different.
func Offsetof(selector ArbitraryType) uintptr, which is used to get the address offset of a field in a struct value. The offset is relative to the address of the struct value. The return results should be always the same for the same corresponding field of values of the same struct type in the same program.
func Sizeof(variable ArbitraryType) uintptr, which is used to get the size of a value (a.k.a., the size of the type of the value). The return results should be always the same for all values of the same type in the same program.
uintptr. Below we will learn that uintptr values can be converted to unsafe pointers (and vice versa).
uintptr.
unsafe.Offsetof function must the struct field selector form value.field. The selector may denote an embedded field, but the field must be reachable without implicit pointer indirections.
package main
import "fmt"
import "unsafe"
func main() {
var x struct {
a int64
b bool
c string
}
const M, N = unsafe.Sizeof(x.c), unsafe.Sizeof(x)
fmt.Println(M, N) // 16 32
fmt.Println(unsafe.Alignof(x.a)) // 8
fmt.Println(unsafe.Alignof(x.b)) // 1
fmt.Println(unsafe.Alignof(x.c)) // 8
fmt.Println(unsafe.Offsetof(x.a)) // 0
fmt.Println(unsafe.Offsetof(x.b)) // 8
fmt.Println(unsafe.Offsetof(x.c)) // 16
}
package main
import "fmt"
import "unsafe"
func main() {
type T struct {
c string
}
type S struct {
b bool
}
var x struct {
a int64
*S
T
}
fmt.Println(unsafe.Offsetof(x.a)) // 0
fmt.Println(unsafe.Offsetof(x.S)) // 8
fmt.Println(unsafe.Offsetof(x.T)) // 16
// This line compiles, for c can be reached
// without implicit pointer indirections.
fmt.Println(unsafe.Offsetof(x.c)) // 16
// This line doesn't compile, for b must be
// reached with the implicit pointer field S.
//fmt.Println(unsafe.Offsetof(x.b)) // error
// This line compiles. However, it prints
// the offset of field b in the value x.S.
fmt.Println(unsafe.Offsetof(x.S.b)) // 0
}
unsafe package don't look much dangerous. The signatures of these functions are very impossible to be changed in future Go 1 versions. Rob Pike even ever made a proposal to move the three functions to elsewhere. Most of the unsafety of the unsafe package comes from unsafe pointers. They are as dangerous as C pointers, what is Go safe pointers always try to avoid.
unsafe package. The new type is IntegerType, The following is its definition. This type doesn't denote a specified type. It just represents any arbitrary integer type. We can view it as a generic type.
type IntegerType int
func Add(ptr Pointer, len IntegerType) Pointer. This function adds an offset to the address represented by an unsafe pointer and return a new unsafe pointer which represents the new address. This function partially covers the usages of the below introduced unsafe pointer use pattern 3.
func Slice(ptr *ArbitraryType, len IntegerType) []ArbitraryType. This function is used to derive a slice with the specified length from a safe pointer, where ArbitraryType is the element type of the result slice.
func String(ptr *byte, len IntegerType) string. This function is used to derive a string with the specified length from a safe byte pointer.
func StringData(str string) *byte. This function is used to get the pointer to the underlying byte sequence of a string. Please note, don't pass empty strings as arguments to this function.
func SliceData(slice []ArbitraryType) *ArbitraryType. This function is used to get the pointer to the underlying element sequence of a slice.
package main
import (
"fmt"
"unsafe"
)
func main() {
a := [16]int{3: 3, 9: 9, 11: 11}
fmt.Println(a)
eleSize := int(unsafe.Sizeof(a[0]))
p9 := &a[9]
up9 := unsafe.Pointer(p9)
p3 := (*int)(unsafe.Add(up9, -6 * eleSize))
fmt.Println(*p3) // 3
s := unsafe.Slice(p9, 5)[:3]
fmt.Println(s) // [9 0 11]
fmt.Println(len(s), cap(s)) // 3 5
t := unsafe.Slice((*int)(nil), 0)
fmt.Println(t == nil) // true
// The following two calls are dangerous.
// They make the results reference
// unknown memory blocks.
_ = unsafe.Add(up9, 7 * eleSize)
_ = unsafe.Slice(p9, 8)
}
import "unsafe"
func String2ByteSlice(str string) []byte {
if str == "" {
return nil
}
return unsafe.Slice(unsafe.StringData(str), len(str))
}
func ByteSlice2String(bs []byte) string {
if len(bs) == 0 {
return ""
}
return unsafe.String(unsafe.SliceData(bs), len(bs))
}
import "unsafe"
// Assume createInt will not be inlined.
//go:noinline
func createInt() *int {
return new(int)
}
func foo() {
p0, y, z := createInt(), createInt(), createInt()
var p1 = unsafe.Pointer(y)
var p2 = uintptr(unsafe.Pointer(z))
// At the time, even if the address of the int
// value referenced by z is still stored in p2,
// the int value has already become unused, so
// garbage collector can collect the memory
// allocated for it now. On the other hand, the
// int values referenced by p0 and p1 are still
// in use.
// uintptr can participate arithmetic operations.
p2 += 2; p2--; p2--
*p0 = 1 // okay
*(*int)(p1) = 2 // okay
*(*int)(unsafe.Pointer(p2)) = 3 // dangerous!
}
p2 is still in use can't guarantee that the memory block ever hosting the int value referenced by z has not been garbage collected yet. In other words, when *(*int)(unsafe.Pointer(p2)) = 3 is executed, the memory block may be collected, or not. It is dangerous to dereference the address stored in value p2 to an int value, for it is possible that the memory block has been already reallocated for another value (even for another program).
t is still in use can't guarantee that the values referenced by value t.y are still in use.
type T struct {
x int
y *[1<<23]byte
}
func bar() {
t := T{y: new([1<<23]byte)}
p := uintptr(unsafe.Pointer(&t.y[0]))
... // use T.x and T.y
// A smart compiler can detect that the value
// t.y will never be used again and think the
// memory block hosting t.y can be collected now.
// Using *(*byte)(unsafe.Pointer(p))) is
// dangerous here.
// Continue using value t, but only use its x field.
println(t.x)
}
*unsafe.Pointer is a general safe pointer type
*unsafe.Pointer is a safe pointer type. Its base type is unsafe.Pointer. As it is a safe pointer, according the conversion rules listed above, it can be converted to unsafe.Pointer type, and vice versa.
package main
import "unsafe"
func main() {
x := 123 // of type int
p := unsafe.Pointer(&x) // of type unsafe.Pointer
pp := &p // of type *unsafe.Pointer
p = unsafe.Pointer(pp)
pp = (*unsafe.Pointer)(p)
}
unsafe standard package documentation lists six unsafe pointer use patterns. Following will introduce and explain them one by one.
*T1 value to unsafe Pointer, then convert the unsafe pointer value to *T2.
*T1 to type *T2, where T1 and T2 are two arbitrary types. However, we should only do such conversions if the size of T1 is no smaller than T2, and only if the conversions are meaningful.
T1 and T2 by using this pattern.
math.Float64bits function, which converts a float64 value to an uint64 value, without changing any bit in the float64 value. The math.Float64frombits function does reverse conversions.
func Float64bits(f float64) uint64 {
return *(*uint64)(unsafe.Pointer(&f))
}
func Float64frombits(b uint64) float64 {
return *(*float64)(unsafe.Pointer(&b))
}
math.Float64bits(aFloat64) function call is different from the result of the explicit conversion uint64(aFloat64).
[]MyString slice to type []string, and vice versa. The result slice and the original slice share the underlying elements. Such conversions are impossible through safe ways,
package main
import (
"fmt"
"unsafe"
)
func main() {
type MyString string
ms := []MyString{"C", "C++", "Go"}
fmt.Printf("%s\n", ms) // [C C++ Go]
// ss := ([]string)(ms) // compiling error
ss := *(*[]string)(unsafe.Pointer(&ms))
ss[1] = "Zig"
fmt.Printf("%s\n", ms) // [C Zig Go]
// ms = []MyString(ss) // compiling error
ms = *(*[]MyString)(unsafe.Pointer(&ss))
// Since Go 1.17, we may also use the
// unsafe.Slice function to do the conversions.
ss = unsafe.Slice((*string)(&ms[0]), len(ms))
ms = unsafe.Slice((*MyString)(&ss[0]), len(ms))
}
unsafe.Slice function to do the conversions:
func main() {
...
ss = unsafe.Slice((*string)(&ms[0]), len(ms))
ms = unsafe.Slice((*MyString)(&ss[0]), len(ss))
}
func ByteSlice2String(bs []byte) string {
return *(*string)(unsafe.Pointer(&bs))
}
String method of the Builder type supported since Go 1.10 in the strings standard package. The size of a byte slice is larger than a string, and their internal structures are similar, so the conversion is valid (for main stream Go compilers). However, despite the implementation may be safely used in standard packages now, it is not recommended to be used in general user code. Since Go 1.20, in general user code, we should try to use the implementation which uses the unsafe.String function, mentioned above in this article.
func String2ByteSlice(s string) []byte {
return *(*[]byte)(unsafe.Pointer(&s)) // dangerous!
}
package main
import "fmt"
import "unsafe"
func main() {
type T struct{a int}
var t T
fmt.Printf("%p\n", &t) // 0xc6233120a8
println(&t) // 0xc6233120a8
fmt.Printf("%x\n", uintptr(unsafe.Pointer(&t))) // c6233120a8
}
package main
import "fmt"
import "unsafe"
type T struct {
x bool
y [3]int16
}
const N = unsafe.Offsetof(T{}.y)
const M = unsafe.Sizeof(T{}.y[0])
func main() {
t := T{y: [3]int16{123, 456, 789}}
p := unsafe.Pointer(&t)
// "uintptr(p)+N+M+M" is the address of t.y[2].
ty2 := (*int16)(unsafe.Pointer(uintptr(p)+N+M+M))
fmt.Println(*ty2) // 789
}
unsafe.Add function to do such address offset operations.
unsafe.Pointer(uintptr(p) + N + M + M) shouldn't be split into two lines, like the following code shows. Please read the comments in the code for the reason.
func main() {
t := T{y: [3]int16{123, 456, 789}}
p := unsafe.Pointer(&t)
// ty2 := (*int16)(unsafe.Pointer(uintptr(p)+N+M+M))
addr := uintptr(p) + N + M + M
// ... (some other operations)
// Now the t value becomes unused, its memory may be
// garbage collected at this time. So the following
// use of the address of t.y[2] may become invalid
// and dangerous!
// Another potential danger is, if some operations
// make the stack grow or shrink here, then the
// address of t might change, so that the address
// saved in addr will become invalid (fact 3).
ty2 := (*int16)(unsafe.Pointer(addr))
fmt.Println(*ty2)
}
&^ bitwise clear operations to do address alignment, as long as the result unsafe pointer and the original one point into the same allocated memory block.
uintptr values as arguments of syscall.Syscall calls.
// Assume this function will not inlined.
func DoSomething(addr uintptr) {
// read or write values at the passed address ...
}
Syscall function in the syscall standard package is as
func Syscall(trap, a1, a2, a3 uintptr) (r1, r2 uintptr, err Errno)
a1, a2 and a3 are still not garbage collected yet within the function internal? The function can't guarantee this. In fact, compilers will make the guarantee. It is the privilege of calls to syscall.Syscall alike functions.
uintptr, like the third argument in the following syscall.Syscall call, to prevent the memory block referenced by that argument from being garbage collected or moved.
uintptr(anUnsafePointer) act as sub-expressions of the talked arguments. Since Go 1.15, the requirement becomes a bit stricter: the talked arguments must present exactly as the uintptr(anUnsafePointer) form.
syscall.Syscall(SYS_READ, uintptr(fd),
uintptr(unsafe.Pointer(p)), uintptr(n))
u := uintptr(unsafe.Pointer(p))
// At this time, the value referenced by p might
// have become unused and been collected already,
// or the address of the value has changed.
syscall.Syscall(SYS_READ, uintptr(fd), u, uintptr(n))
// Arguments must be in the "uintptr(anUnsafePointer)"
// form. In fact, the call was safe before Go 1.15.
// But Go 1.15 changes the rule a bit.
syscall.Syscall(SYS_XXX, uintptr(uintptr(fd)),
uint(uintptr(unsafe.Pointer(p))), uintptr(n))
uintptr result of reflect.Value.Pointer or reflect.Value.UnsafeAddr method call to unsafe pointer
Pointer and UnsafeAddr of the Value type in the reflect standard package both return a result of type uintptr instead of unsafe.Pointer. This is a deliberate design, which is to avoid converting the results of calls (to the two methods) to any safe pointer types without importing the unsafe standard package.
p := (*int)(unsafe.Pointer(reflect.ValueOf(new(int)).Pointer()))
u := reflect.ValueOf(new(int)).Pointer()
// At this moment, the memory block at the address
// stored in u might have been collected already.
p := (*int)(unsafe.Pointer(u))
reflect.Value.UnsafePointer(), which returns a unsafe.Pointer value and is preferred over the two just mentioned functions. That means, the old deliberate design is thought as not good now.
reflect.SliceHeader.Data or reflect.StringHeader.Data field to unsafe pointer, and the inverse.
Data fields of the struct type SliceHeader and StringHeader in the reflect standard package are declared with type uintptr instead of unsafe.Pointer.
*reflect.StringHeader pointer value, so that we can manipulate the internal of the string. The same, we can convert a slice pointer to a *reflect.SliceHeader pointer value, so that we can manipulate the internal of the slice.
reflect.StringHeader:
package main
import "fmt"
import "unsafe"
import "reflect"
func main() {
a := [...]byte{'G', 'o', 'l', 'a', 'n', 'g'}
s := "Java"
hdr := (*reflect.StringHeader)(unsafe.Pointer(&s))
hdr.Data = uintptr(unsafe.Pointer(&a))
hdr.Len = len(a)
fmt.Println(s) // Golang
// Now s and a share the same byte sequence, which
// makes the bytes in the string s become mutable.
a[2], a[3], a[4], a[5] = 'o', 'g', 'l', 'e'
fmt.Println(s) // Google
}
reflect.SliceHeader:
package main
import (
"fmt"
"unsafe"
"reflect"
)
func main() {
a := [6]byte{'G', 'o', '1', '0', '1'}
bs := []byte("Golang")
hdr := (*reflect.SliceHeader)(unsafe.Pointer(&bs))
hdr.Data = uintptr(unsafe.Pointer(&a))
hdr.Len = 2
hdr.Cap = len(a)
fmt.Printf("%s\n", bs) // Go
bs = bs[:cap(bs)]
fmt.Printf("%s\n", bs) // Go101
}
*reflect.StringHeader pointer value from an actual (already existed) string, or get a *reflect.SliceHeader pointer value from an actual (already existed) slice. We shouldn't do the contrary, such as creating a string from a new allocated StringHeader, or creating a slice from a new allocated SliceHeader. For example, the following code is dangerous.
var hdr reflect.StringHeader
hdr.Data = uintptr(unsafe.Pointer(new([5]byte)))
// Now the just allocated byte array has lose all
// references and it can be garbage collected now.
hdr.Len = 5
s := *(*string)(unsafe.Pointer(&hdr)) // dangerous!
package main
import (
"fmt"
"reflect"
"strings"
"unsafe"
)
func String2ByteSlice(str string) (bs []byte) {
strHdr := (*reflect.StringHeader)(unsafe.Pointer(&str))
sliceHdr := (*reflect.SliceHeader)(unsafe.Pointer(&bs))
sliceHdr.Data = strHdr.Data
sliceHdr.Cap = strHdr.Len
sliceHdr.Len = strHdr.Len
return
}
func main() {
// str := "Golang"
// For the official standard compiler, the above
// line will make the bytes in str allocated on
// an immutable memory zone.
// So we use the following line instead.
str := strings.Join([]string{"Go", "land"}, "")
s := String2ByteSlice(str)
fmt.Printf("%s\n", s) // Goland
s[5] = 'g'
fmt.Println(str) // Golang
}
func ByteSlice2String(bs []byte) (str string) {
sliceHdr := (*reflect.SliceHeader)(unsafe.Pointer(&bs))
strHdr := (*reflect.StringHeader)(unsafe.Pointer(&str))
strHdr.Data = sliceHdr.Data
strHdr.Len = sliceHdr.Len
return
}
package main
import (
"fmt"
"reflect"
"unsafe"
)
func Example_Bad() *byte {
var str = "godoc"
hdr := (*reflect.StringHeader)(unsafe.Pointer(&str))
pbyte := (*byte)(unsafe.Pointer(hdr.Data + 2))
return pbyte // *pbyte == 'd'
}
func main() {
fmt.Println(string(*Example_Bad()))
}
func Example_Good1() *byte {
var str = "godoc"
hdr := (*reflect.StringHeader)(unsafe.Pointer(&str))
pbyte := (*byte)(unsafe.Pointer(
uintptr(unsafe.Pointer(hdr.Data)) + 2))
return pbyte
}
// Works since Go 1.17.
func Example_Good2() *byte {
var str = "godoc"
hdr := (*reflect.StringHeader)(unsafe.Pointer(&str))
pbyte := (*byte)(unsafe.Add(unsafe.Pointer(hdr.Data), 2))
return pbyte
}
SliceHeader and StringHeader types in the reflect standard package are similar in that they say the representations of the two struct types may change in a later release. So the above valid examples using the two types may become invalid even if the unsafe rules keep unchanged. Fortunately, at present (Go 1.24), the two available mainstream Go compilers (the standard Go compiler and the gccgo compiler) both recognize the representations of the two types declared in the reflect standard package.
unsafe.String, unsafe.StringData, unsafe.Slice and unsafe.SliceData functions described earlier in this article.
-gcflags=all=-d=checkptr is supported since Go Toolchain 1.14 (it is recommended to use this option on Windows with Go Toolchain 1.15+). When this option is used, some (but not all) incorrect unsafe pointer uses will be detected at run time. Once such an incorrect use is detected, a panic will occur. Thanks to Matthew Dempsky for implementing this great feature!
The Go 101 project is hosted on Github. Welcome to improve Go 101 articles by submitting corrections for all kinds of mistakes, such as typos, grammar errors, wording inaccuracies, description flaws, code bugs and broken links.
If you would like to learn some Go details and facts every serveral days, please follow Go 101's official Twitter account @zigo_101.
reflect standard package.sync standard package.sync/atomic standard package.