qbit/go
1
0
Fork 0
mirror of https://github.com/golang/go synced 2024-11-18 18:04:46 -07:00
Code Releases Activity

cmd/api: limit concurrent 'go list' calls to GOMAXPROCS

Browse Source 
Each invocation of 'go list' may consume a significant quantity of
system resources, including buffers for reading files and RAM for the
runtime's memory footprint.
Very small builders may even hit swap as a result of that load,
further exacerbating resource contention.

To avoid overloading small builders, restrict 'go list' calls to
runtime.GOMAXPROCS as it is set at the first call to loadImports.

This also somewhat improves running time even on larger machines: on
my workstation, this change reduces the wall time for 'go test
cmd/api' by around 100ms.

Updates #38537

Change-Id: I968e0f961a8f1d84c27e1ab8b621b9670dcfd448
Reviewed-on: https://go-review.googlesource.com/c/go/+/228998
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit is contained in:
Bryan C. Mills 2020-04-20 11:32:05 -04:00
parent 40a144b94f
commit 75e79adaf9
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
src/cmd/api/goapi.go
View File

@ -444,6 +444,11 @@ type listImports struct {
var listCache sync.Map // map[string]listImports, keyed by contextName
// listSem is a semaphore restricting concurrent invocations of 'go list'.
var listSem = make(chan semToken, runtime.GOMAXPROCS(0))
type semToken struct{}
// loadImports populates w with information about the packages in the standard
// library and the packages they themselves import in w's build context.
//
@ -468,6 +473,9 @@ func (w *Walker) loadImports() {
imports, ok := listCache.Load(name)
if !ok {
listSem <- semToken{}
defer func() { <-listSem }()
cmd := exec.Command(goCmd(), "list", "-e", "-deps", "-json", "std")
cmd.Env = listEnv(w.context)
out, err := cmd.CombinedOutput()