CL 239754 eagerly initialized the environment. This turns out to be a
problem for gopls, which calls ApplyFixes with no ProcessEnv.
Reinitializing it every time seriously harmed the performance of
unimported completions. Move back to lazy initialization.
Working with invalid options has caused a lot of confusion; this is only
the most recent. We have to maintain backwards compatibility in the
externally visible API, but everywhere else we can require fully
populated options. That includes the source byte slice and the options.
LocalPrefix is really more of an Option than an attribute of the
ProcessEnv, and it is needed in ApplyFixes where we really don't want to
have to pass a ProcessEnv. Move it up to Options.
Change-Id: Ib9466c375a640a521721da4587091bf93bbdaa3c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/241159
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This CL got away from me a little.
For a number of reasons, the existing goimports API of passing in values
for various GO* values was not working. For one, the number of necessary
variables kept growing. For another, we tried to avoid calling `go env`
in GOPATH mode by using `build.Default`, but that turns out to be buggy;
see golang/go#39838. And finally, it created massive confusion about
whether the values were intended to be read from the OS environment, or
fully evaluated by the `go` command.
There are only two users of the internal imports API, so there really
shouldn't need to be more than two modes. For the command line tool, we
have to call `go env` to deal with the `go/build` bug. So we just do it.
Tests use that same path, but can augment the enviroment to set
themselves up. In contrast, `gopls` needs to fully specify the
environment. It can simply pass in the fully evaluated GO* values.
Finally, make the change I was actually here to make: propagate
GOMODCACHE and use it where appropriate.
Fixesgolang/go#39761.
Change-Id: I720c69839d91d66d98e94dfc5f065ba0279c5542
Reviewed-on: https://go-review.googlesource.com/c/tools/+/239754
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
An import path like "foo/bar.v1" is still a local path, not an external
package, and should be grouped as such.
Change-Id: I06be3c01076f616a3cdc8e23bc9c056643035ad1
Reviewed-on: https://go-review.googlesource.com/c/tools/+/234111
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This change attempts to fix a concurrency error that would cause
textDocument/CodeLens, textDocument/Formatting, textDocument/DocumentLink,
and textDocument/Hover from failing on go.mod files.
The issue was that the go command would return a potential concurrency
error since the ModHandle and the ModTidyHandle are both using the
temporary go.mod file.
Updates golang/go#37824
Change-Id: I6cd63c1f75817c7308e033aec473966536a2a3bd
Reviewed-on: https://go-review.googlesource.com/c/tools/+/224917
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In all cases, use a Logf field to configure debug logging. Non-nil means
that logging is enabled through the given function.
Fixes accidental debug spam from goimports, which had a separate Debug
flag that was supposed to guard logging, but wasn't used when creating
the gocommand.Invocation.
Change-Id: I448fa282111db556ac2e49801268d0affc19ae30
Reviewed-on: https://go-review.googlesource.com/c/tools/+/221557
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
goimports has a fast path for stdlib imports that was not checking for
import cycles. Add the check.
Fixesgolang/go#37063.
Change-Id: I46c98c317d8f06f83018fef9ef7edf9222e6b3f3
Reviewed-on: https://go-review.googlesource.com/c/tools/+/217958
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We were assuming that all in-memory packages were equally useful. That's
not true for projects with a large dependency tree. Call into the
imports code to score them.
While I'm here, score the main module above direct deps.
Updates golang/go#36591.
Change-Id: I07c56dd3ff7338e76f3643e18d35abc1b52d6763
Reviewed-on: https://go-review.googlesource.com/c/tools/+/215023
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Now that we can detach scans, it's easy to kick off a background refresh
that doesn't block the user. Performance may be a bit worse until the
scan finishes, but that's the price we pay for freshness.
Adaptively rate-limit the refresh rate so that we don't turn the user's
computer into a hot plate.
Change-Id: Icbe8f384f44a219b57465da22d9becc19001eab8
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212022
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
x_tests can access exports from the test variant of the package under
test. Change loadExportsFromFiles to understand that mode, and use it
where appropriate.
I didn't want to come up with a cache key for for the test variant, so
for now we bypass the cache in these situations.
Fixesgolang/go#29979.
Change-Id: I9959a08da97bbee64c5bcd56e06f548486693156
Reviewed-on: https://go-review.googlesource.com/c/tools/+/213221
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We only need to return a relatively small number of completions to the
user. There's no point continuing once we have those, so switch the
completion functions to be callback-based, and cancel once we've got
what we want.
Change-Id: Ied199fb1f41346819c7237dfed8251fa3ac73ad7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212634
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We have multiple use cases for scanning: goimports, import completion,
and unimported completions. All three need slightly different features,
and the latter have very different performance considerations. Scanning
everything all at once and returning it was not good enough for them.
Instead, design the API as a series of callbacks for each
directory/package: first we discover its existence, then we load its
package name, then we load its exports. At each step the caller can
choose whether to proceed with the package. Import completion can stop
before loading exports, goimports can apply its directory name
heuristics, and in the future we'll be able to stop the scan short once
we've found all the results we want for completions.
I don't intend any significant changes here but there may be some little
ones around the edges.
Change-Id: I39c3aa08cc0e4793c280242c342770f62e101364
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212631
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We don't use the go/packages resolver in goimports, and as we develop
gopls it only becomes harder to use there. Give up.
Change-Id: Ic8b566c6dd730b23b0c81d7d34a41f16fe0be7e8
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212630
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We intend to use the GOPATH resolver's results during LSP
autocompletion. That means we have to be able to cache its data, same as
we do for modules. Convert it to use a dirInfoCache.
Cache exports in the dirInfoCache. Along the way, store exports as slices
rather than maps. We don't need the extra structure the vast majority of
the time, and the memory overhead is nontrivial.
Change-Id: If267d6b00da2163a960b93b2cf2088ec2538f73d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/205162
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Add a function that returns all the exported identifiers associated with
a package name that doesn't have an import yet. This will allow
completions like rand<> to return rand.Seed (from math/rand) and
rand.Prime (from crypto/rand).
Updates golang/go#31906
Change-Id: Iee290c786de263d42acbfabd76bf0edbf303afc9
Reviewed-on: https://go-review.googlesource.com/c/tools/+/204204
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
When proposing packages to import, we can propose more relevant packages
first. Introduce that concept to the pkg struct, and sort by it when
returning candidates.
In all cases we prefer stdlib packages first. Then, in module mode, we
prefer packages that are in the module's dependencies over those that
aren't. We could go further and prefer direct deps over indirect too,
but I didn't have the code for that handy.
I also changed the alphabetical sort from import path to package name,
because that's what the user sees first in the UI.
Updates golang/go#31906
Change-Id: Ia981ee9ffe3202e2a68eef3a36f65e81849a4ac2
Reviewed-on: https://go-review.googlesource.com/c/tools/+/204203
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
At tip, $GOROOT/src/go.mod is 1.14, $GOROOT/src/vendor exists, and so
vendor mode is automatically enabled. That causes golang/go#34826.
This will be fixed by my upcoming vendoring CL, but for now skip.
Updates golang/go#34826
Change-Id: I5fff51fff54cf83e6369ae76bf3b19cfb7b5ac15
Reviewed-on: https://go-review.googlesource.com/c/tools/+/203257
Run-TryBot: Heschi Kreinick <heschi@google.com>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Since a user's module cache is generally going to be much bigger than
their main module, one would expect that caching just information about
the module cache would be sufficient. It turns out that's not correct.
When we discover something in the module cache, we have to make sure
that a different version of it isn't already in scope. Doing that can
require information about the main module or replace targets, so that
needs to be cached too.
Concretely, when I'm working in x/tools, if a scan discovers a version
of x/tools in the module cache, it should usually ignore that version.
But that might not be true in more complicated cases, especially those
involving nested modules whose boundaries change.
So, cache everything except GOROOT. Since the new data is mutable,
we store it separately from the module cache data so that it can be
discarded easily between runs.
Change-Id: I47364f6c0270fee03af8898fec6c85d1b9c8d780
Reviewed-on: https://go-review.googlesource.com/c/tools/+/202045
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Scan most sources, including GOPATH, the module cache, the main module,
and replace targets as appropriate. Use the cached stdlib instead of
scanning GOROOT.
We heavily cache the contents of the module cache, so performance is
decent. But we have to look at all the modules not in the module cache
too to get the right versions of modules (see
(*ModuleResolver).canonicalize), which currently isn't cached at all,
even just for a single run. That ends up being pretty expensive.
The implementation changes are relatively small; add package name
loading to scan(), cache that result, and allow callers to control what
directories are scanned so that it can skip GOROOT.
I also cleared out most of the stdlib from the unimported completion
test and added a simple external completion to it for safety's sake.
Change-Id: Id50fd4703b1126be35a000fe90719e19c3ab84bf
Reviewed-on: https://go-review.googlesource.com/c/tools/+/199178
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Before this commit, when running imports.Process concurrently, the program
panics with a fatal error due to concurrent map iterations and map writes.
This CL fixes this by adding a copy of the map to the packageInfo structure.
Fixed#34895
Change-Id: If009e6108813f86495c7e20e69739186b8b236d7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/200865
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
When running in GOROOT/src, `go list -m all` shows the std package (or
cmd package) as the main module. This confuses goimports into adding
std/ or cmd/ at the beginning of import paths. Skip canonicalization for
paths under GOROOT.
Fixesgolang/go#31814
Change-Id: Iff5cc7e2a2053e4cc87c1a579a4c47d856cd0a2e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/195063
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This exposes the candidate imports that are discovered, even if there is
not a particular reference that requires it to be imported. Currently,
this only produces results for standard library packages.
This is useful for autocompletion on unimported packages.
Change-Id: Iafd883153d451a0ef1dae29b24d4d48530c855f7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/189999
Run-TryBot: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
When an import is added to the ast, the import declarations are merged
together into the first import declaration. Since this is a part of
the formatting functionality of goimports, do this during formatting
regardless.
The merging pass was added to astutil.AddNamedImport in order to address
issue golang/go#14075. This joined imports from other blocks into the first
import declaration, so that a single block of imports isn't split across
multiple blocks.
This functionality is more of a formatting change than a fix imports
change, in line with sorting the imports, which occurs even when
FormatOnly. The formatting was only applied when an import was added
(not renamed or deleted). This change makes formatting by goimports
more consistent across runs and is not dependent on the exact fixes
that need to be applied.
Change-Id: Icb90bf694ff35e2d6405a3d477cf82fcd3e697e0
Reviewed-on: https://go-review.googlesource.com/c/tools/+/189941
Run-TryBot: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
goimports should not attempt to load package names for an empty list
of packages. 'go list' interprets an empty argument list as '.', which
may or may not be a package.
Fixesgolang/go#33175
Change-Id: Id8df148432437295905c52f2d5fb73d890277569
Reviewed-on: https://go-review.googlesource.com/c/tools/+/188379
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
A recent change to the go command broke these tests. Mark them as
go/packages incompatible for now, until we have time to fix them.
Updates golang/go#33175
Change-Id: Ib4c594bd032107fb1245bfe18fc80392fbab0730
Reviewed-on: https://go-review.googlesource.com/c/tools/+/186838
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
The imports ProcessEnv contains cached module and filesystem state. This change
allows gopls to use the same ProcessEnv and resolver across multiple calls to the
internal/imports library.
A ProcessEnv belongs to a view, because the cached module state depends
on the module that is open in the workspace.
Since we do not yet track whether the 'go.mod' file has changed, we
conservatively reset the cached state in the module resolver before
every call to imports.Process.
Change-Id: I27c8e212cb0b477ff425d5ed98a544b27b7d92ee
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184921
Run-TryBot: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
golang.org/cl/170238 forgot that it's okay for an x_test to import a
package in the same directory. Only skip the candidate if the package
we're looking for has the same name as the one being fixed.
Fixesgolang/go#32440
Change-Id: I7806d9f4174ca217ac83887da5e23b28cd102095
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181338
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Users of packagestest will create modules that don't exist on the
Internet and can change. There's no point in checking sum.golang.org for
them under any circumstances.
Similarly for the various goimports tests.
Fixesgolang/go#32216.
Change-Id: Id9a6b660564cb744530bf9d209fca19008fb9c4f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/178722
Reviewed-by: Ian Cottrell <iancottrell@google.com>
For various reasons we need an internal-facing imports API. Move imports
to internal/imports, leaving behind a small wrapper package. The wrapper
package captures the globals at time of call into the options struct.
Also converts the last goimports tests to use the test helpers, and
fixes go/packages in module mode to work with empty modules, which was
necessary to get those last tests converted.
Change-Id: Ib1212c67908741a1800b992ef1935d563c6ade32
Reviewed-on: https://go-review.googlesource.com/c/tools/+/175437
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
