We have multiple use cases for scanning: goimports, import completion,
and unimported completions. All three need slightly different features,
and the latter have very different performance considerations. Scanning
everything all at once and returning it was not good enough for them.
Instead, design the API as a series of callbacks for each
directory/package: first we discover its existence, then we load its
package name, then we load its exports. At each step the caller can
choose whether to proceed with the package. Import completion can stop
before loading exports, goimports can apply its directory name
heuristics, and in the future we'll be able to stop the scan short once
we've found all the results we want for completions.
I don't intend any significant changes here but there may be some little
ones around the edges.
Change-Id: I39c3aa08cc0e4793c280242c342770f62e101364
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212631
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We don't use the go/packages resolver in goimports, and as we develop
gopls it only becomes harder to use there. Give up.
Change-Id: Ic8b566c6dd730b23b0c81d7d34a41f16fe0be7e8
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212630
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We intend to use the GOPATH resolver's results during LSP
autocompletion. That means we have to be able to cache its data, same as
we do for modules. Convert it to use a dirInfoCache.
Cache exports in the dirInfoCache. Along the way, store exports as slices
rather than maps. We don't need the extra structure the vast majority of
the time, and the memory overhead is nontrivial.
Change-Id: If267d6b00da2163a960b93b2cf2088ec2538f73d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/205162
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Add a function that returns all the exported identifiers associated with
a package name that doesn't have an import yet. This will allow
completions like rand<> to return rand.Seed (from math/rand) and
rand.Prime (from crypto/rand).
Updates golang/go#31906
Change-Id: Iee290c786de263d42acbfabd76bf0edbf303afc9
Reviewed-on: https://go-review.googlesource.com/c/tools/+/204204
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
When proposing packages to import, we can propose more relevant packages
first. Introduce that concept to the pkg struct, and sort by it when
returning candidates.
In all cases we prefer stdlib packages first. Then, in module mode, we
prefer packages that are in the module's dependencies over those that
aren't. We could go further and prefer direct deps over indirect too,
but I didn't have the code for that handy.
I also changed the alphabetical sort from import path to package name,
because that's what the user sees first in the UI.
Updates golang/go#31906
Change-Id: Ia981ee9ffe3202e2a68eef3a36f65e81849a4ac2
Reviewed-on: https://go-review.googlesource.com/c/tools/+/204203
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
At tip, $GOROOT/src/go.mod is 1.14, $GOROOT/src/vendor exists, and so
vendor mode is automatically enabled. That causes golang/go#34826.
This will be fixed by my upcoming vendoring CL, but for now skip.
Updates golang/go#34826
Change-Id: I5fff51fff54cf83e6369ae76bf3b19cfb7b5ac15
Reviewed-on: https://go-review.googlesource.com/c/tools/+/203257
Run-TryBot: Heschi Kreinick <heschi@google.com>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Since a user's module cache is generally going to be much bigger than
their main module, one would expect that caching just information about
the module cache would be sufficient. It turns out that's not correct.
When we discover something in the module cache, we have to make sure
that a different version of it isn't already in scope. Doing that can
require information about the main module or replace targets, so that
needs to be cached too.
Concretely, when I'm working in x/tools, if a scan discovers a version
of x/tools in the module cache, it should usually ignore that version.
But that might not be true in more complicated cases, especially those
involving nested modules whose boundaries change.
So, cache everything except GOROOT. Since the new data is mutable,
we store it separately from the module cache data so that it can be
discarded easily between runs.
Change-Id: I47364f6c0270fee03af8898fec6c85d1b9c8d780
Reviewed-on: https://go-review.googlesource.com/c/tools/+/202045
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Scan most sources, including GOPATH, the module cache, the main module,
and replace targets as appropriate. Use the cached stdlib instead of
scanning GOROOT.
We heavily cache the contents of the module cache, so performance is
decent. But we have to look at all the modules not in the module cache
too to get the right versions of modules (see
(*ModuleResolver).canonicalize), which currently isn't cached at all,
even just for a single run. That ends up being pretty expensive.
The implementation changes are relatively small; add package name
loading to scan(), cache that result, and allow callers to control what
directories are scanned so that it can skip GOROOT.
I also cleared out most of the stdlib from the unimported completion
test and added a simple external completion to it for safety's sake.
Change-Id: Id50fd4703b1126be35a000fe90719e19c3ab84bf
Reviewed-on: https://go-review.googlesource.com/c/tools/+/199178
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Before this commit, when running imports.Process concurrently, the program
panics with a fatal error due to concurrent map iterations and map writes.
This CL fixes this by adding a copy of the map to the packageInfo structure.
Fixed#34895
Change-Id: If009e6108813f86495c7e20e69739186b8b236d7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/200865
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
When running in GOROOT/src, `go list -m all` shows the std package (or
cmd package) as the main module. This confuses goimports into adding
std/ or cmd/ at the beginning of import paths. Skip canonicalization for
paths under GOROOT.
Fixesgolang/go#31814
Change-Id: Iff5cc7e2a2053e4cc87c1a579a4c47d856cd0a2e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/195063
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This exposes the candidate imports that are discovered, even if there is
not a particular reference that requires it to be imported. Currently,
this only produces results for standard library packages.
This is useful for autocompletion on unimported packages.
Change-Id: Iafd883153d451a0ef1dae29b24d4d48530c855f7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/189999
Run-TryBot: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
When an import is added to the ast, the import declarations are merged
together into the first import declaration. Since this is a part of
the formatting functionality of goimports, do this during formatting
regardless.
The merging pass was added to astutil.AddNamedImport in order to address
issue golang/go#14075. This joined imports from other blocks into the first
import declaration, so that a single block of imports isn't split across
multiple blocks.
This functionality is more of a formatting change than a fix imports
change, in line with sorting the imports, which occurs even when
FormatOnly. The formatting was only applied when an import was added
(not renamed or deleted). This change makes formatting by goimports
more consistent across runs and is not dependent on the exact fixes
that need to be applied.
Change-Id: Icb90bf694ff35e2d6405a3d477cf82fcd3e697e0
Reviewed-on: https://go-review.googlesource.com/c/tools/+/189941
Run-TryBot: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
goimports should not attempt to load package names for an empty list
of packages. 'go list' interprets an empty argument list as '.', which
may or may not be a package.
Fixesgolang/go#33175
Change-Id: Id8df148432437295905c52f2d5fb73d890277569
Reviewed-on: https://go-review.googlesource.com/c/tools/+/188379
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
A recent change to the go command broke these tests. Mark them as
go/packages incompatible for now, until we have time to fix them.
Updates golang/go#33175
Change-Id: Ib4c594bd032107fb1245bfe18fc80392fbab0730
Reviewed-on: https://go-review.googlesource.com/c/tools/+/186838
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
The imports ProcessEnv contains cached module and filesystem state. This change
allows gopls to use the same ProcessEnv and resolver across multiple calls to the
internal/imports library.
A ProcessEnv belongs to a view, because the cached module state depends
on the module that is open in the workspace.
Since we do not yet track whether the 'go.mod' file has changed, we
conservatively reset the cached state in the module resolver before
every call to imports.Process.
Change-Id: I27c8e212cb0b477ff425d5ed98a544b27b7d92ee
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184921
Run-TryBot: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
golang.org/cl/170238 forgot that it's okay for an x_test to import a
package in the same directory. Only skip the candidate if the package
we're looking for has the same name as the one being fixed.
Fixesgolang/go#32440
Change-Id: I7806d9f4174ca217ac83887da5e23b28cd102095
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181338
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Users of packagestest will create modules that don't exist on the
Internet and can change. There's no point in checking sum.golang.org for
them under any circumstances.
Similarly for the various goimports tests.
Fixesgolang/go#32216.
Change-Id: Id9a6b660564cb744530bf9d209fca19008fb9c4f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/178722
Reviewed-by: Ian Cottrell <iancottrell@google.com>
For various reasons we need an internal-facing imports API. Move imports
to internal/imports, leaving behind a small wrapper package. The wrapper
package captures the globals at time of call into the options struct.
Also converts the last goimports tests to use the test helpers, and
fixes go/packages in module mode to work with empty modules, which was
necessary to get those last tests converted.
Change-Id: Ib1212c67908741a1800b992ef1935d563c6ade32
Reviewed-on: https://go-review.googlesource.com/c/tools/+/175437
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
