Accepted proposal: grow an existing map with maps.Grow

Liz Fong-Jones’s proposal to add maps.Grow was accepted on October 29. It gives you a way to make room in a map that already exists, much like make(M, n) does for a new one.

A Go map grows on its own as you add entries. Each time it runs out of room, the runtime allocates a larger hash table, rehashes the entries into it, and leaves the old table for the garbage collector. A loop that adds thousands of entries can trigger this many times, and each time costs CPU and creates more garbage. If you know the final size when you create the map, you can pass it to make and skip all that extra work.

But that only works if you’re the one creating the map. If someone else created it and you’re only adding entries, you’re stuck with however much room it has.

net/http/httputil.ReverseProxy hits this on every request. It copies the upstream response headers into the client’s response:

func copyHeader(dst, src http.Header) {
    for k, vv := range src {
        for _, v := range vv {
            dst.Add(k, v)
        }
    }
}

The proxy calls this as copyHeader(rw.Header(), res.Header). rw is the http.ResponseWriter, and rw.Header() returns the header map that the server will write back to the client. The server created that map before the proxy ever saw it. The proxy knows how many headers are coming but can’t pass that number to the map, so the map may grow several times inside this loop.

The usual workaround is to build a bigger map yourself and swap it in. libhoney-go does this. It keeps an event’s fields in a map and hands that map out through a Fields() method. A simplified version looks like this:

type fieldHolder struct {
    data map[string]any
}

// Fields returns a reference to the event's field map.
func (f *fieldHolder) Fields() map[string]any {
    return f.data
}

func (f *fieldHolder) AddFields(src map[string]any) {
    if len(src) > len(f.data) {
        m := make(map[string]any, len(f.data)+len(src))
        maps.Copy(m, f.data) // (1)
        maps.Copy(m, src)
        f.data = m // (2)
        return
    }
    maps.Copy(f.data, src)
}

Here:

• (1) every existing field gets copied again just to move it into the bigger map
• (2) f.data now points to a new map, so anyone who called Fields() earlier still holds the old one and won’t see the new fields

Even with the extra copy, that change cut CPU use by about 5% in production. Copying everything once was cheaper than letting the map grow step by step.

With maps.Grow, you can skip all those shenanigans:

// Grow increases m's capacity, if necessary, so that at least n more
// entries can be added without further growth in the typical case, as if
// m had been created with make(M, len(m)+n). Grow never shrinks m.
func Grow[M ~map[K]V, K comparable, V any](m M, n int)

The proxy can make room before the loop starts:

func copyHeader(dst, src http.Header) {
    maps.Grow(dst, len(src))
    for k, vv := range src {
        for _, v := range vv {
            dst.Add(k, v)
        }
    }
}

And libhoney can grow f.data in place. Callers who got the map from Fields() still hold the same map, so they see the new fields:

func (f *fieldHolder) AddFields(src map[string]any) {
    maps.Grow(f.data, len(src))
    maps.Copy(f.data, src)
}

If you’ve used slices.Grow, you might expect maps.Grow to return the grown map. It doesn’t, and the reason is how Go stores slices and maps:

• A slice is a small slice struct. It holds a pointer to an array, a length, and a capacity. When slices.Grow needs more room, it makes a bigger array and copies the elements into it. Your old slice still points to the old array. That’s why slices.Grow returns a new slice and you write s = slices.Grow(s, n).
• A map is a pointer to a maps.Map struct in the runtime. When maps.Grow needs more room, it puts a larger hash table behind that pointer. The pointer itself doesn’t change, so every variable holding the map sees the new table. There’s nothing to return.

A nil map is a nil pointer with no table behind it, so Grow panics if you ask it to add room:

var m map[string]int
maps.Grow(m, 0)  // fine, nothing to add
maps.Grow(m, 10) // panic: assignment to entry in nil map
maps.Grow(m, -1) // panics, n is negative

If you don’t have a map yet, use make.

The n you pass is a hint, just like the size you give make. The doc comment calls Grow best effort, so the map may still grow later. You also can’t ask a map for its capacity. An earlier proposal to allow cap on maps was declined.

The proposal also changes maps.Copy.

That only covers copying one map into another. The header loop above calls Header.Add instead, so it still needs its own maps.Grow call. So does code that fills a map from a slice:

func index(dst map[string]User, users []User) {
    maps.Grow(dst, len(users))
    for _, u := range users {
        dst[u.ID] = u
    }
}

If most of those users are already in dst, the extra room goes unused and you’ve spent memory for nothing.

The implementation, the Copy change, and a follow-up that uses maps.Grow in httputil and cgi are all still under review. The change is expected to land in Go 1.28.