A user that's editing packages in their workspace may introduce parse
errors (missing package statement), but we still want to be able to add
imports for them. Continue past parse errors.
Fixesgolang/go#39315.
Change-Id: I3bbf428e7b9ef32a87258af2dafbe0d7b86b7348
Reviewed-on: https://go-review.googlesource.com/c/tools/+/237686
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
An import path like "foo/bar.v1" is still a local path, not an external
package, and should be grouped as such.
Change-Id: I06be3c01076f616a3cdc8e23bc9c056643035ad1
Reviewed-on: https://go-review.googlesource.com/c/tools/+/234111
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This change attempts to fix a concurrency error that would cause
textDocument/CodeLens, textDocument/Formatting, textDocument/DocumentLink,
and textDocument/Hover from failing on go.mod files.
The issue was that the go command would return a potential concurrency
error since the ModHandle and the ModTidyHandle are both using the
temporary go.mod file.
Updates golang/go#37824
Change-Id: I6cd63c1f75817c7308e033aec473966536a2a3bd
Reviewed-on: https://go-review.googlesource.com/c/tools/+/224917
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In all cases, use a Logf field to configure debug logging. Non-nil means
that logging is enabled through the given function.
Fixes accidental debug spam from goimports, which had a separate Debug
flag that was supposed to guard logging, but wasn't used when creating
the gocommand.Invocation.
Change-Id: I448fa282111db556ac2e49801268d0affc19ae30
Reviewed-on: https://go-review.googlesource.com/c/tools/+/221557
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Over time we have accumulated a disturbing number of ways to run the Go
command, each of which supported slightly different features and
workarounds. Combine all of them.
This unavoidably introduces some changes in debug logging behavior; I
think we should try to fix them incrementally and consistently.
Updates golang/go#37368.
Change-Id: I664ca8685bf247a226be3cb807789c2fcddf233d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/220737
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This change adds a buildFlags variable to the processEnv struct rather than appending them to the GOFLAGS by using a space separator.
Fixesgolang/go#37108
Change-Id: I4331066c30fa51f0133504d723132527b00ce74a
Reviewed-on: https://go-review.googlesource.com/c/tools/+/218857
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
goimports has a fast path for stdlib imports that was not checking for
import cycles. Add the check.
Fixesgolang/go#37063.
Change-Id: I46c98c317d8f06f83018fef9ef7edf9222e6b3f3
Reviewed-on: https://go-review.googlesource.com/c/tools/+/217958
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We were assuming that all in-memory packages were equally useful. That's
not true for projects with a large dependency tree. Call into the
imports code to score them.
While I'm here, score the main module above direct deps.
Updates golang/go#36591.
Change-Id: I07c56dd3ff7338e76f3643e18d35abc1b52d6763
Reviewed-on: https://go-review.googlesource.com/c/tools/+/215023
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Now that we can detach scans, it's easy to kick off a background refresh
that doesn't block the user. Performance may be a bit worse until the
scan finishes, but that's the price we pay for freshness.
Adaptively rate-limit the refresh rate so that we don't turn the user's
computer into a hot plate.
Change-Id: Icbe8f384f44a219b57465da22d9becc19001eab8
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212022
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
x_tests can access exports from the test variant of the package under
test. Change loadExportsFromFiles to understand that mode, and use it
where appropriate.
I didn't want to come up with a cache key for for the test variant, so
for now we bypass the cache in these situations.
Fixesgolang/go#29979.
Change-Id: I9959a08da97bbee64c5bcd56e06f548486693156
Reviewed-on: https://go-review.googlesource.com/c/tools/+/213221
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Resolvers are lazy initialized. That worked fine until the addition of
the scan semaphore -- it's not a good idea to create that lazily, since
you can't synchronize on a channel that doesn't exist.
Specifically, this caused a gopls hang when completion finished without
needing to use the resolver. In that case, we'd call ClearForNewScan/Mod
on an uninitialized resolver, and then hang receiving from a nil
channel.
Instead, eagerly initialize where convenient, and particularly the scan
semaphore.
Change-Id: I3adb1ae76b751650995e50f50346e06fd9c9f88d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/214258
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Filesystem walks of large GOPATHs/module caches can take seconds,
especially on systems with slow filesystems like MacOS and WSL. We don't
want to block completion requests on walks finishing. At the same time,
cancelling a walk midway through results in an unusable cache, where we
don't know which parts have been scanned so far.
The best option is to run the walks in a separate goroutine. Then we can
detach and let them finish. On the other side, we need to be able to
reattach for the next completion request.
Introduce a new method on caches, ScanAndListen, which first processes
all the items in the cache, then notifies of any new items. This allows
us to reattach to an existing scan without missing anything.
The background scan introduces concurrency to the resolvers where there
wasn't any before. We can't use mutexes, because there's no way to stop
Lock() when a context expires. Use a 1-element semaphore channel to
accomplish the same effect.
Along the way: Only rescan GOPATH if the resolver has been cleared. None
of this makes sense for GOPATH without that. Fix a bug where we were
scanning the main module twice in module mode. Stop loading exports in
module tests, it slows them down a ton.
Change-Id: I978efae733ccba0c0cdc8e8fe6892bf5f15feac8
Reviewed-on: https://go-review.googlesource.com/c/tools/+/213217
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
An important part of letting the callback choose what to load is...not
loading the stuff it doesn't want.
Change-Id: I4048d7aed756b6ebc26fb6f8e384f44c64281f90
Reviewed-on: https://go-review.googlesource.com/c/tools/+/213129
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
When finding completion candidates, we can use the same tricks
goimports uses to ignore directories that look irrelevant.
Change-Id: I114a3d4e487aed7f59fc48b2f86d42129baf5183
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212859
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Now that we have all these callbacks, it's strange to have a list of
root types to exclude on the side. Merge that into the callback.
Change-Id: I8dc88e095362a8d2e180196ad9b81e17d4d34949
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212858
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
In scan implementations, stop after cancellation, and swallow the
context's error for convenience.
In the module implementation specifically, try to avoid scanning if the
cache is enough to satisfy the user. When we do have to scan, prioritize
module dependencies before the whole cache.
Change-Id: I23dc98df016f9fca4f31c7ded3d11bc257c29b94
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212857
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We only need to return a relatively small number of completions to the
user. There's no point continuing once we have those, so switch the
completion functions to be callback-based, and cancel once we've got
what we want.
Change-Id: Ied199fb1f41346819c7237dfed8251fa3ac73ad7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212634
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We have multiple use cases for scanning: goimports, import completion,
and unimported completions. All three need slightly different features,
and the latter have very different performance considerations. Scanning
everything all at once and returning it was not good enough for them.
Instead, design the API as a series of callbacks for each
directory/package: first we discover its existence, then we load its
package name, then we load its exports. At each step the caller can
choose whether to proceed with the package. Import completion can stop
before loading exports, goimports can apply its directory name
heuristics, and in the future we'll be able to stop the scan short once
we've found all the results we want for completions.
I don't intend any significant changes here but there may be some little
ones around the edges.
Change-Id: I39c3aa08cc0e4793c280242c342770f62e101364
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212631
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We don't use the go/packages resolver in goimports, and as we develop
gopls it only becomes harder to use there. Give up.
Change-Id: Ic8b566c6dd730b23b0c81d7d34a41f16fe0be7e8
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212630
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Expose ImportPathToAssumedName (internally) and use it in an LSP
completion case that doesn't go through the usual imports code.
Fixesgolang/go#35401.
Change-Id: If87912072e11e22c542f7474841e53467a33ef2b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/206890
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
I forgot to set .Name on completion candidates. That meant that if the
package name didn't match the import path, you'd first get an import
without a name, then it would be added when you organized imports.
Change-Id: Ic374de872324effa6bc04c1440c659d7a182d17f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/205503
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Loading completion suggestions can be slow, especially in GOPATH mode
where basically anything can change at any time. As a compromise, cache
everything for 30 seconds. Specifically, after a completion operation
finishes, if the cache is more than 30 seconds old, refresh it
asynchronously. That keeps user-facing latency consistent, without
chewing up CPU when the editor isn't in use. It does mean that if you
walk away for an hour and come back, the first completion may be stale.
In module mode this is relatively benign. The only things the
longer caching affects are the main module and replace targets, and
relevant packages in those will generally be loaded by gopls, so they'll
have full, up-to-date type information regardless.
In GOPATH mode this may be more troublesome, since it affects
everything. In particular, go get -u of a package that isn't imported
yet won't be reflected until the cache period expires. I think that's a
rare enough case not to worry about.
Change-Id: Iaadfd0ff647cda2b1dcdead9254b5492b397e86e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/205163
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We intend to use the GOPATH resolver's results during LSP
autocompletion. That means we have to be able to cache its data, same as
we do for modules. Convert it to use a dirInfoCache.
Cache exports in the dirInfoCache. Along the way, store exports as slices
rather than maps. We don't need the extra structure the vast majority of
the time, and the memory overhead is nontrivial.
Change-Id: If267d6b00da2163a960b93b2cf2088ec2538f73d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/205162
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Add a function that returns all the exported identifiers associated with
a package name that doesn't have an import yet. This will allow
completions like rand<> to return rand.Seed (from math/rand) and
rand.Prime (from crypto/rand).
Updates golang/go#31906
Change-Id: Iee290c786de263d42acbfabd76bf0edbf303afc9
Reviewed-on: https://go-review.googlesource.com/c/tools/+/204204
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
When proposing packages to import, we can propose more relevant packages
first. Introduce that concept to the pkg struct, and sort by it when
returning candidates.
In all cases we prefer stdlib packages first. Then, in module mode, we
prefer packages that are in the module's dependencies over those that
aren't. We could go further and prefer direct deps over indirect too,
but I didn't have the code for that handy.
I also changed the alphabetical sort from import path to package name,
because that's what the user sees first in the UI.
Updates golang/go#31906
Change-Id: Ia981ee9ffe3202e2a68eef3a36f65e81849a4ac2
Reviewed-on: https://go-review.googlesource.com/c/tools/+/204203
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
CL 203820 removes an assumption in go/build that srcDir is in the main
module, since in general it need not be. That requires the use of some
other mechanism for callers to communicate the correct location of the
main module.
Fortunately, we already have a WorkingDir field on the ProcessEnv
struct that does exactly that. We can simply propagate it through if
the corresponding field is present on go/build.Context.
Updates golang/go#34860
Change-Id: Idabf9ae06d8383a72772d5a589fae1d10f206c01
Reviewed-on: https://go-review.googlesource.com/c/tools/+/203857
Reviewed-by: Heschi Kreinick <heschi@google.com>
Scan most sources, including GOPATH, the module cache, the main module,
and replace targets as appropriate. Use the cached stdlib instead of
scanning GOROOT.
We heavily cache the contents of the module cache, so performance is
decent. But we have to look at all the modules not in the module cache
too to get the right versions of modules (see
(*ModuleResolver).canonicalize), which currently isn't cached at all,
even just for a single run. That ends up being pretty expensive.
The implementation changes are relatively small; add package name
loading to scan(), cache that result, and allow callers to control what
directories are scanned so that it can skip GOROOT.
I also cleared out most of the stdlib from the unimported completion
test and added a simple external completion to it for safety's sake.
Change-Id: Id50fd4703b1126be35a000fe90719e19c3ab84bf
Reviewed-on: https://go-review.googlesource.com/c/tools/+/199178
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Before this commit, when running imports.Process concurrently, the program
panics with a fatal error due to concurrent map iterations and map writes.
This CL fixes this by adding a copy of the map to the packageInfo structure.
Fixed#34895
Change-Id: If009e6108813f86495c7e20e69739186b8b236d7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/200865
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
This exposes the candidate imports that are discovered, even if there is
not a particular reference that requires it to be imported. Currently,
this only produces results for standard library packages.
This is useful for autocompletion on unimported packages.
Change-Id: Iafd883153d451a0ef1dae29b24d4d48530c855f7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/189999
Run-TryBot: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Have the imports resolvers load the exports for packages. This allows
each resolver to provide its own implementation of loading exports,
beyond reading from the directory.
Change-Id: I813f2ca59271a1698874556e8771243ac008f46f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/188759
Run-TryBot: Suzy Mueller <suzmue@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Get quick fixes for the diagnostics related to import errors. These
fixes add, remove, or rename exactly one import.
This change exposes the individual fixes found by the imports package,
and then applies each of them separately to the source. Since applying each
fix requires a new ast anyway, we pass in the source to be parsed each time.
Change-Id: Ibcbfa703d21b6983d774d2010716da8c25525d4f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/188059
Run-TryBot: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Fix typos and fix descriptions of methods where the behavior described
in the comment is not implemented by that method.
Change-Id: I2c34caff43399dcb4f0e5e41b67189d8d8404fd5
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184697
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
goimports should not attempt to load package names for an empty list
of packages. 'go list' interprets an empty argument list as '.', which
may or may not be a package.
Fixesgolang/go#33175
Change-Id: Id8df148432437295905c52f2d5fb73d890277569
Reviewed-on: https://go-review.googlesource.com/c/tools/+/188379
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
strings.Trim treats the second parameter as a set of characters you
want to trim. It does not look for an entire string to trim.
This fix will maintain the current behavior, simply eliminating the dupe
character in the set.
Should we instead mean to really trim the entire string, this needs a
different fix.
Change-Id: Id3fa4105421819edc6a898efb1ffab26c8cea67a
GitHub-Last-Rev: 198e429869711ee1510fffe5a993acb07eff4502
GitHub-Pull-Request: golang/tools#142
Reviewed-on: https://go-review.googlesource.com/c/tools/+/187497
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Ian Cottrell <iancottrell@google.com>
The imports ProcessEnv contains cached module and filesystem state. This change
allows gopls to use the same ProcessEnv and resolver across multiple calls to the
internal/imports library.
A ProcessEnv belongs to a view, because the cached module state depends
on the module that is open in the workspace.
Since we do not yet track whether the 'go.mod' file has changed, we
conservatively reset the cached state in the module resolver before
every call to imports.Process.
Change-Id: I27c8e212cb0b477ff425d5ed98a544b27b7d92ee
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184921
Run-TryBot: Suzy Mueller <suzmue@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
A pass is responsible for fixing the imports of a given file. It now
finds the necessary changes to make without applying the result to the
ast, which may be desirable to give a user more control about what
changes will be applied to their program. This change splits the process
of finding the fixes from making the modifications to the ast to allow
this functionality to be easily possible.
Change-Id: Ibf8ca247c35539f91de4be90c634f0db9a939d07
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184197
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Suzy Mueller <suzmue@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change modifies gopls to use the internal goimports library, which
allows us to manually configure the ProcessEnv. We also add a logger to
the ProcessEnv to allow this change not to conflict with gopls's logging
mechanism.
Fixesgolang/go#32585
Change-Id: Ic9aae69c7cfbc9b1f2e66aa8d812175dbc0065ce
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184198
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
golang.org/cl/170238 forgot that it's okay for an x_test to import a
package in the same directory. Only skip the candidate if the package
we're looking for has the same name as the one being fixed.
Fixesgolang/go#32440
Change-Id: I7806d9f4174ca217ac83887da5e23b28cd102095
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181338
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
I forgot to actually save the resolver instances...
Change-Id: Ieff4f22d29038895dc15be97491476f414544a5c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181337
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Users of packagestest will create modules that don't exist on the
Internet and can change. There's no point in checking sum.golang.org for
them under any circumstances.
Similarly for the various goimports tests.
Fixesgolang/go#32216.
Change-Id: Id9a6b660564cb744530bf9d209fca19008fb9c4f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/178722
Reviewed-by: Ian Cottrell <iancottrell@google.com>
For various reasons we need an internal-facing imports API. Move imports
to internal/imports, leaving behind a small wrapper package. The wrapper
package captures the globals at time of call into the options struct.
Also converts the last goimports tests to use the test helpers, and
fixes go/packages in module mode to work with empty modules, which was
necessary to get those last tests converted.
Change-Id: Ib1212c67908741a1800b992ef1935d563c6ade32
Reviewed-on: https://go-review.googlesource.com/c/tools/+/175437
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
