for Loop Semantic Changes in Go 1.22: Be Aware of the Impact
for loops, including both for-range loops and traditional 3-clause for ..; ..; .. {...} loops (which will be abbreviated as for;; in the remaining content of this article).
for loops which loop variables are declared within the loops are changed (we call such loop variables as freshly-declared loop variables in the remaining content). For example, in the following piece of code, the semantics of the former two loops are not changed, but the latter two ones are changed (from Go 1.21 to 1.22).
for k, v = range aContainer {...}
for a, b, c = f(); condition; statement {...}
for k, v := range aContainer {...}
for a, b, c := f(); condition; statement {...}
//demo1.go
package main
func main() {
c, out := make(chan int), make(chan int)
m := map[int]int{1: 2, 3: 4}
for i, v := range m {
go func() {
<-c
out <- i+v
}()
}
close(c)
println(<-out + <-out)
}
$ gotv 1.21. run demo1.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run demo1.go 14 $ gotv 1.22. run demo1.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run demo1.go 10
14 (very probably, when without the participation of the channel c);
10 (always, even without the participation of the channel c).
for loop is shared by all iterations during executing the loop. The two new created goroutines are executed after the execution of the for loop, in which case, the final values of the i and v loop variables are 3 and 4. (3+4) + (3+4) gives 14.
for loop will be instantiated as a distinctive instance at the start of each iteration. In other words, it is per-iteration scoped now. So the values of the i and v loop variables used in the two new created goroutines are 1 2 and 3 4, respectively. (1+2) + (3+4) gives 10.
c, there is a data race condition present in the above program code, which should be a clear fact for a competent Go programmer. In order to avoid data race and get the same result as the new semantics, the loop in the program should be re-written as:
for i, v := range m {
i, v := i, v // this line is added
go func() {
out <- i+v
}()
}
// demo2.go
package main
func main() {
c, out := make(chan int), make(chan int)
for i := 1; i <= 3; i++ {
go func() {
<-c
out <- i
}()
}
close(c)
println(<-out + <-out + <-out)
}
$ gotv 1.21. run demo2.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run demo2.go 12 $ gotv 1.22. run demo2.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run demo2.go 6
for;; loops.
for-range loops is well-justified. The new semantics of for-range loops become more intuitive. The change only affects for k, v := range .. {...} loops, in which the := symbol strongly suggests that the loop variables are per-iteration scoped. No implications are introduced. The impact of the change is almost positive.
for;; loops is insufficient. The main reason provided by the proposal makers is to make a consistency with for-range loops (they are both for loops). However, It's not intuitive at all to think the loop variables in the following alike loops are per-iteration scoped.
for a, b, c := anExpression; aCondition; postStatement {
... // loop body
}
a, b, c := anExpression statement is only executed once during the execution of the loop, so it is intuitive that the loop variables are only explicitly instantiated once during the execution of the loop. The new semantics make the the loop variables instantiated at each iteration, which means there must be some implicit code to do the job. This is true. Go 1.22+ specification says:
{
a_last, b_last, c_last := anExpression
pa_last, pb_last, pc_last = &a_last, &b_last, &c_last
first := true
for {
a, b, c := *pa_last, *pb_last, *pc_last
if first {
first = false
} else {
postStatement
}
if !(aCondition) {
break
}
pa_last, pb_last, pc_last = &a, &b, &c
... // loop body
}
}
// demo-defer.go
package main
import "fmt"
func main() {
for counter, n := 0, 2; n >= 0; n-- {
defer func(v int) {
fmt.Print("#", counter, ": ", v, "\n")
counter++
}(n)
}
}
$ gotv 1.21. run demo-defer.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run demo-defer.go #0: 0 #1: 1 #2: 2 $ gotv 1.22. run demo-defer.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run demo-defer.go #0: 0 #0: 1 #0: 2
counter is never effectively increased. Why? I'm sorry. As mentioned above, it is some hard to clearly explain the new semantics and I don't think I have the ability to do this. You may get it from the following equivalent code:
func main() {
counter_last, n_last := 0, 2
p_counter_last, p_n_last := &counter_last, &n_last
first := true
for {
counter, n := *p_counter_last, *p_n_last
if (first) {
first = false
} else {
n--
}
if !(n >= 0) {
break
}
p_counter_last, p_n_last = &counter, &n
defer func(v int) {
fmt.Print("#", counter, ": ", v, "\n")
counter++
}(n)
}
}
// search.go
package main
import "fmt"
func demoFilter(n int) bool {
return n & 1 == 0;
}
// Search values and return them without perverting order.
func search(start, end int)(r []int) {
var count = 0
for i, index := start, 0; i <= end; i++ {
if demoFilter(i) {
count++
defer func(value int) {
r[index] = value
index++
}(i)
}
}
r = make([]int, count) // only allocate once
return
}
func main() {
fmt.Println(search(0, 9))
}
$ gotv 1.21. run search.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run search.go [8 6 4 2 0] $ gotv 1.22. run search.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run search.go [0 0 0 0 0]
for;; loops might be okay to be per-iteration scoped, but some are strongly expected to be whole-loop scoped, such as the index and counter loop variables shown above. I ever suggested to allow re-declaration statements as postStatement of for;; loops to explicitly specify which variables are per-iteration scoped. For example, in the following loop code, n is per-iteration scoped but counter is whole-loop scoped.
for counter, n := 0, 2; n >= 0; n := n - 1 { ... }
// demo-closure-1.go
package main
import "fmt"
func main() {
var printN func()
for n := 0; n < 9; n++ {
if printN == nil {
printN = func() {
fmt.Println(n)
}
}
}
printN()
}
$ gotv 1.21. run demo-closure-1.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run demo-closure-1.go 9 $ gotv 1.22. run demo-closure-1.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run demo-closure-1.go 0
printN closure captures is the only instance of the loop variable, which final value is 9. However, since Go 1.22, what the printN closure captures is the first instance of the loop variable, which final value is 1. That is the reason of the behavior difference between the two Go versions.
// demo-closure-2.go
package main
import (
"bytes"
"fmt"
)
func main() {
var printBuf func()
for buf, i := (bytes.Buffer{}), byte('a'); i <= 'z'; i++ {
if printBuf == nil {
printBuf = func() {
fmt.Printf("%s\n", buf.Bytes())
}
}
buf.WriteByte(i)
}
printBuf()
}
$ gotv 1.21. run demo-closure-2.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run demo-closure-2.go abcdefghijklmnopqrstuvwxyz $ gotv 1.22. run demo-closure-2.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run demo-closure-2.go a
package main
func main() {
var next func()
for i := 0; i < 3; next() {
print(i)
next = func() {
i++
}
}
}
012 then exits immediately)
postStatement. Each of the instances is instantiated in one iteration.
// demo-pointer1.go
package main
import "fmt"
func main() {
for i, p := 0, (*int)(nil); p == nil; fmt.Println(p == &i) {
p = &i
}
}
$ gotv 1.21. run demo-pointer1.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run demo-pointer1.go true $ gotv 1.22. run demo-pointer1.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run demo-pointer1.go false
p == &i, p points to the first instance of i, whereas &i takes the address of the second instance of i. So the comparison evaluation result is false.
func main() {
i_last, p_last := 0, (*int)(nil)
p_i_last, p_p_last := &i_last, &p_last
first := true
for {
i, p := *p_i_last, *p_p_last
if first {
first = false
} else {
fmt.Println(p == &i)
}
if !(p == nil) {
break
}
p_i_last, p_p_last = &i, &p
p = &i
}
}
// demo-pointer2.go
package main
import "fmt"
func main() {
var p *int
for i := 0; i < 3; *p++ {
p = &i
fmt.Println(i)
}
}
$ gotv 1.21. run demo-pointer2.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run demo-pointer2.go 0 1 2 $ gotv 1.22. run demo-pointer2.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run demo-pointer2.go 0 0 0 ...
for ...; ...; postStatement {
... // no continue statements here
}
for ...; ...; {
... // no continue statements here
postStatement
}
// demo-pointer3.go
package main
import "fmt"
func pointerDemo1() {
for i, p := 0, (*int)(nil); p == nil; {
p = &i
fmt.Println(p == &i) // the old 3rd clause
}
}
func pointerDemo2() {
var p *int
for i := 0; i < 3; {
p = &i
fmt.Println(i)
*p++ // the old 3rd clause
}
}
func main() {
pointerDemo1();
pointerDemo2();
}
$ gotv 1.22. run demo-pointer3.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run demo-pointer3.go true 0 1 2
sync.Mutex, sync/atomic.Int64, bytes.Buffer, strings.Builder, and container/list.List, etc.
wg will be (implicitly) copied at the start of each loop iteration.
// demo-nocopy1.go
package main
import (
"sync"
"time"
)
func process() (wait func()) {
for wg, i := (sync.WaitGroup{}), 0; i < 3; i++ {
if (wait == nil) {
wait = wg.Wait
}
wg.Add(1)
go func(v int) {
defer wg.Done()
if (v > 0) {
time.Sleep(time.Second/8)
}
println(v)
}(i)
}
return
}
func main() {
process()()
}
$ gotv 1.21. run demo-nocopy1.go [Run]: $HOME/.cache/gotv/tag_go1.21.8/bin/go run demo-nocopy1.go 0 2 1 $ gotv 1.22. run demo-nocopy1.go [Run]: $HOME/.cache/gotv/tag_go1.22.1/bin/go run demo-nocopy1.go 0 $ gotv 1.22. vet demo-nocopy1.go [Run]: $HOME/.cache/gotv/tag_go1.22.1/bin/go vet demo-nocopy1.go
go vet command in Go toolchain versions prior to 1.24 can't catch such implicit duplication of no-copy values, regardless of whether the loop variable wg is captured in the loop body or not.
no-copy checks occur during run time. The go vet command in Go toolchain v1.24+ still can't catch such cases. Let's view an example which uses strings.Builder (each strings.Builder value contains a pointer field which should point to itself):
// demo-nocopy2.go
package main
import "strings"
import "fmt"
func foo(pb *strings.Builder) {}
var bar = foo
func a2z_foo() string {
for b, i := (strings.Builder{}), byte('a'); ; i++ {
b.WriteByte(i)
foo(&b) // <- difference is here
if i == 'z' {
return b.String()
}
}
}
func a2z_bar() string {
for b, i := (strings.Builder{}), byte('a'); ; i++ {
b.WriteByte(i)
bar(&b) // <- difference is here
if i == 'z' {
return b.String()
}
}
}
func main() {
fmt.Println("foo:", a2z_foo())
fmt.Println("bar:", a2z_bar())
}
$ gotv 1.21. run demo-nocopy2.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run demo-nocopy2.go foo: abcdefghijklmnopqrstuvwxyz bar: abcdefghijklmnopqrstuvwxyz $ gotv 1.22. run demo-nocopy2.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run demo-nocopy2.go foo: abcdefghijklmnopqrstuvwxyz panic: strings: illegal use of non-zero Builder copied by value goroutine 1 [running]: ...
a2z functions are consistent. No loop variables duplication happens, so there will be no panic.
no-copy check functions. But the check only works for the a2z_bar function, not for the a2z_foo function.
no-copy check work for the a2z_foo function? Because of two compiler optimizations. In one optimization, the compiler omits the foo(&b) line in the a2z_foo function. The optimization is actually valid. The optimization is conservative so that the compiler doesn't omits the bar(&b) line in the a2z_bar function. The consequence is, in the other optimization, the compiler (mistakenly) thinks the fresh-declared loop variable b in the a2z_foo function can be instantiated only once for the entire loop, even though its semantics suggest per-iteration instantiation.
a2z functions in the above program should be still consistent since Go 1.22. They should both panic.
go vet command in the latest Go toolchain versions still can't catch such implicit duplication of no-copy values.
bytes.Buffer type, which values should also not be copied. However, neither compile-time checks or run-time checks are made for them.
bar function.
// demo-largesize.go
package main
import (
"fmt"
"time"
)
const N = 1 << 18
func foo(f func([]byte, int)) {
a := [N]byte{}
for i := 0; i < len(a); i++ {
f(a[:], i)
}
}
func bar(f func([]byte, int)) {
for a, i := [N]byte{}, 0; i < len(a); i++ {
f(a[:], i)
}
}
func main() {
readonly := func(x []byte, k int) {}
bench := func(f func(func([]byte, int))) time.Duration {
start := time.Now()
f(readonly)
return time.Since(start)
}
fmt.Println("foo time:", bench(foo))
fmt.Println("bar time:", bench(bar))
}
$ gotv 1.21. run aaa.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run aaa.go foo time: 689.79µs bar time: 690.988µs $ gotv 1.22. run aaa.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run aaa.go foo time: 734.85µs bar time: 18.795043596s
bar function between Go 1.21 and 1.22 (note that 1s == 1,000,000us). Why? Because, with the official standard Go compiler 1.22, the loop variable a in the bar function is duplicated in each iteration. Whereas in prior versions, such duplication is always needless.
// demo-concurency1.go
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func() {
defer wg.Done()
fmt.Println(i)
}()
}
wg.Wait()
}
i at each iteration. Prior to Go 1.22, there is a clear data race condition present in the program, because the loop variable i is only instantiated once during the whole loop. All the new created goroutines will read the single instance but the main goroutine will modify it. The following outputs prove this fact:
$ CGO_ENABLED=1 gotv 1.21. run -race demo-concurency1.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run -race demo-concurency1.go 3 3 ================== WARNING: DATA RACE ... ================== 3
i := i line at the start of the loop body. Go 1.22 fixes the specified data race problem by changing the semantics of for;; loops, without modifying the old problematic code. This can be verified by the following outputs:
$ CGO_ENABLED=1 gotv 1.22. run -race demo-concurency1.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run -race demo-concurency1.go 1 2 0
for;; loops. But is it worth it to fix such a small problem by introducing magical implicit code?
// demo-concurency2.go
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
for i := 0; i < 3; i++ {
wg.Add(1)
go func() {
defer wg.Done()
i++ // <-- add this line
fmt.Println(i)
}()
}
wg.Wait()
}
i. But the answer is "no", because there is an implicit assignment at the start of each iteration and the implicit assignment uses an instance of the loop variable as source value (a.k.a. the main goroutine reads it), however the instance is modified in a new created goroutine.
$ CGO_ENABLED=1 gotv 1.22. run -race demo-concurency2.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run -race demo-concurency2.go ================== WARNING: DATA RACE ... ================== 2 1 3 Found 1 data race(s)
// demo-concurency3.go
package main
import (
"fmt"
"sync"
)
const NumWorkers = 3
func isGold(num uint64) bool {
return num & 0xFFFFF == 0
}
func main() {
var c = make(chan uint64)
var m sync.Mutex
for n, i := 0, uint64(0); n < NumWorkers; n++ {
go func() {
for {
m.Lock()
i++
v := i
m.Unlock()
if isGold(v) {
c <- v
}
}
}()
}
for n := range c {
fmt.Println("Found gold", n)
}
}
$ CGO_ENABLED=1 gotv 1.21. run -race demo-concurency3.go [Run]: $HOME/.cache/gotv/tag_go1.21.7/bin/go run -race demo-concurency3.go Found gold 1048576 Found gold 2097152 Found gold 3145728 ... ^C $ CGO_ENABLED=1 gotv 1.22. run -race demo-concurency3.go [Run]: $HOME/.cache/gotv/tag_go1.22.0/bin/go run -race demo-concurency3.go ================== WARNING: DATA RACE ... ================== Found gold 1048576 Found gold 1048576 Found gold 1048576 Found gold 2097152 Found gold 2097152 Found gold 2097152 Found gold 3145728 Found gold 3145728 Found gold 3145728 ... ^C
i := i line at the start of the loop body. Yes, this is still the best suggestion to avoid data race for such situations in the Go 1.22+ era. Is this a mockery of the new semantics (of for;; loops)?
i should be declared outside the loop block (since Go 1.22).
for;; semantics introduced in Go 1.22. There might be more, I'm not sure.
//go:build go1.xy comment directive at the start of a source file to specify the language version for the source file. (However, the //go:build go1.xy comment directive in a Go source file might still be ignored due to potential bugs in the toolchain implementation. For example, the Go core team is not willing to fix this known bug for Go toolchain v1.22.x versions.)
-gcflags=-lang=go1.xy compiler option when building a Go binary to specify the Go language version for the seed files you're passing to the compiler.
go 1.xy directive line in the go.mod file of a Go module to specify the default Go language version for all of the source files in the Go module. A missing go directive line is assumed as go 1.16. The effects of the directive line are described here.
go run command to run Go code as scripts (a set of Go source files without go.mod files). If a Go script source file doesn't contain a //go:build go1.xy comment directive and the -gcflags=-lang=go1.xy compiler option is not specified, then the behavior of the code in the script file is compiler dependent, just as what the above examples show.
for;; loops, etc.), then things may not go well.
for;; loops in the module's code before bumping the language version to Go 1.22+ in the go.mod file. Especially pay attention to those freshly-declared loop variables which are not modified in postStatement.
for;; loops if you worry about getting bitten by the pitful of the new semantics
for;; loops behave the same with either the old semantics or the new semantics. But if you're unsure about it, you can always rewrite the following alike loops
for a, b, c := anExpression; aCondition; postStatement {
... // loop body
}
{
a, b, c := anExpression
for ; aCondition; postStatement {
... // loop body
}
}
a and c are instantiated per loop iteration, but b will be only instantiated once during the whole loop.
{
a, b, c := anExpression
for ; aCondition; postStatement {
a, c := a, c
... // loop body
}
}
for-range loops is positive, while the impact of the new semantics of for;; loops is negative. This is just my personal opinion.
for;; loops might require additional debug time in code writing and additional cognitive effort during code review and understanding in some cases.
for;; loops might introduce potential performance degradation and data race issues in existing code, requiring careful review and potential adjustments. Depending on specific cases, such issues might be found in time or not.
for;; loops are rare and tiny, whereas the drawbacks are more prominent and serious.
for;; loops anyway.
The Go 101 project is hosted on Github. Welcome to improve Go 101 articles by submitting corrections for all kinds of mistakes, such as typos, grammar errors, wording inaccuracies, description flaws, code bugs and broken links.
If you would like to learn some Go details and facts every serveral days, please follow Go 101's official Twitter account @zigo_101.