Memory Leaking Scenarios

Go specification doesn't specify whether or not the result string and base string involved in a substring expression should share the same underlying memory block to host the underlying byte sequences of the two strings. The standard Go compiler/runtime does let them share the same underlying memory block. This is a good design, which is both memory and CPU consuming wise. But it may cause kind-of memory leaking sometimes.

For example, after the following function f is called, there will be 1M bytes memory leaking (kind of), until the package-level variable s0 is modified again elsewhere.

var s0 string // a package-level variable

func f(s1 string) {
	// Assume s1 is a string much longer than 50.
	s0 = s1[:50]
	// Now, s0 shares the same underlying memory block of s1.
	// s1 is not alive now, but s0 is still alive.
	// Although there are only 50 bytes used in the block,
	// the fact that s0 is still alive prevent the 1M bytes
	// memory block from being collected.
}

To avoid this kind-of memory leaking, we can convert the substring to a []byte value then convert the []byte value back to string.

func f(s1 string) {
	s0 = string([]byte(s1[:50]))
}

The disadvantage of the above way to avoid the kind-of memory leaking is there are two 50-byte duplications happen in the conversion process, one of them is unecessary.

We can make using of one of the optimizations made by the standard Go compiler to avoid one duplication, with a small extra cost of one byte memory wasting.

func f(s1 string) {
	s0 = (" " + s1[:50])[1:]
}

The disadvantage of the above way is the compiler optimization may become invalid later, and the optimization may be not available for other compilers.

The third way to avoid the kind-of memory leaking is to untilize the strings.Builder supported since Go 1.10.

import "strings"

func f(s1 string) {
	var b strings.Builder
	b.Grow(50)
	b.WriteString(s1[:50])
	s0 = b.String()
	// b.Reset() // if b is used elsewhere,
	             // it must be reset here.
}

The disadvantage of the third way is a little verbose (by comparing to the first two ways). A good news is, since Go 1.12 (the next main version to be released at Feb. 2019), we can call the Repeat function with the count argument as 1 in the strings standard package to clone a string. Since Go 1.12, the underlying implementation of strings.Repeat will make use of strings.Builder.

Similarly to substrings, subslices may also cause kind-of memory leaking. In the following code, after the g function is called, most memory occupied by the memory block hosting the elements of s1 will be lost (if no more values reference the memory block).

var s0 []int

func g(s1 []int) {
	// Assume the length of s1 is much larger than 30.
	s0 = s1[len(s1)-30:]
}

If we want to avoid the kind-of memory leaking, we must duplicate the 30 elements for s0, so that the aliveness of s0 will not prevent the memory block hosting the elements of s1 from being collected.

func g(s1 []int) {
	s0 = append([]int(nil), s1[len(s1)-30:]...)
}

In the following code, after the g function is called, the memory block allocated for the first and the last elements of slice s will get lost.

func h() []*int {
	s := []*int{new(int), new(int), new(int), new(int)}
	// do smething ...
	return s[1:3:3]
}

Only if the returned slice is still alive, it will prevent the underlying memory block which hosts the elements of s from being collected, which in consequence prevents the two memory blocks allocated for the first and the last elements of s from being collected, though the two elements are dead.

If we want to avoid such memory leaking, we must reset the pointers stored in the dead elements.

func h() []*int {
	s := []*int{new(int), new(int), new(int), new(int)}
	// do smething ...
	s1 := s[1:3]
	s[0] = nil; s[len(s)-1] = nil
	return s1
}

We often need to reset the pointers for some old slice elements in slice element deletion operations.

Some times, for some logic mistakes in code design, one or more goroutines stay in blocking state for ever (hanging). Such goroutines are called hanging goroutines. Go runtime thinks hanging goroutines are still alive. Once a goroutine becomes hanging, the resources allocated for it and the memory blocks referenced by it will never be garbage collected.

For example, if the following function is called by passing a nil channel argument to it, then the current goroutine will become hanging. so the memory block allocated for s will never get collected.

func k(c <-chan bool) {
	s := make([]int64, 1e6)
	if <-c { // block here for ever if c is nil
		_ = s
		// use s, ...
	}
}

If we pass a non-nil channel to the above function, and there are no other goroutines will send values to the channel since the function is called, The current goroutine will also become hanging.

Sometimes, we deliberately makes the main goroutine in a program hang to prevent the program from exiting. Generally, most other hanging goroutine cases are unexpected. We should avoid such cases happening.

Setting a finalizer for a value which is a member of a cyclic reference group may prevent all memory blocks allocated for the cyclic reference group from being collected. This is real memory leaking, not kind of.

After the following function is called and exits, the memory blocks allocated for x and y are not guaranteed to be garbage collected in future garbage collecting.

func memoryLeaking() {
	type T struct {
		v [1<<20]int
		t *T
	}

	var finalizer = func(t *T) {
		 fmt.Println("finalizer called")
	}

	var x, y T

	// SetFinalizer will make x escape to heap.
	runtime.SetFinalizer(&x, finalizer)

	// Following two lines combined will make
	// x and y not collectable.
	x.t, y.t = &y, &x // y also escapes to heap.
}

So, please avoid setting finalizers for values in a cyclic reference group.

By the way, we shouldn't use finalizers as object destructors.

Please read this article for details.