If typeCheck() returned an error, we could get into a state where a
package had an entry in the pcache, but the package's files had an
empty "pkgs" map. When we got a DidChange event for one of the files,
no packages would get invalidated since the file's "pkgs" was
empty. This resulted in the cached typeCheck() error persisting
indefinitely. Fix by never caching pcache entries on error.
An easy way to reproduce the problem was to delete the package name
from a file. For example, edit "package foo" to be just
"package". This caused the package to get stuck with an "AST for %s
has an invalid position" error.
Change-Id: I330bf9e419852dffa0f2dee94b56226367488dd1
GitHub-Last-Rev: 18be7078521b942694c76f799a2d520eee47167d
GitHub-Pull-Request: golang/tools#135
Reviewed-on: https://go-review.googlesource.com/c/tools/+/185839
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
also change the return type to be and end function and not an incomplete span
Change-Id: Icd99d93ac98a0f8088f33e905cf1ee3fe410c024
Reviewed-on: https://go-review.googlesource.com/c/tools/+/185349
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
For all uses inside the lsp we use the detatch logic instead
For tests we build it in the test harness instead
This is in preparation for things on the context becomming important
Change-Id: I7e6910e0d3581b82abbeeb09f9c22a99efb73142
Reviewed-on: https://go-review.googlesource.com/c/tools/+/185677
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We err on the side of refreshing package metadata if something goes
wrong getting a file's AST, but we don't want to refresh in the case
of context cancellation. Now we check for that error explicitly.
In particular, I noticed that completions would stop working when
typing quickly. Refreshing the metadata triggers "go list" calls which
can take a long time in certain cases.
Change-Id: I1b0c580e5541b1536a69ccaef241d7e8c5720d60
GitHub-Last-Rev: 6a82bfb586f93ef5e8e5996b11e06ffc7808f529
GitHub-Pull-Request: golang/tools#130
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184977
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This adds the ability to tie a background context to the context that created it
in traces, and also cleans up and annotates the context used in type checking.
This gives us detailed connected traces of all the type checking and parsing
logic.
Change-Id: I32721220a50ecb9b4404a4e9354343389d7a5219
Reviewed-on: https://go-review.googlesource.com/c/tools/+/183757
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This uses the new opencensus compatability layer to add telementry to some of
the functions in the lsp, in order to allow us to understand their costs and
call patterns.
Change-Id: I7df820cd4eace7a4840ac6397d5df402369bf0a7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/183419
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
The first deadlock involved differing mutex acquisition order in two
code paths:
1. loadParseTypecheck() holds the "mcache" mutex then eventually
acquires the "handleMu" file mutex.
2. (*goFile).invalidateContent() acquires the "handleMu" mutex first and
then the "mcache" mutex.
Fix by changing the acquisition order in invalidateContent().
The second deadlock involved the file watcher. The two code paths
involved were:
1. (*goFile).GetPackages() holds the view mutex and eventually calls
(*WatchMap).Watch, which acquires the watcher mutex.
2. (*session).openOverlay acquires the watcher mutex as it triggers a
file's callbacks, and then the callback
"(*goFile).invalidateContent" acquires the view mutex.
Fix by not holding the watcher mutex as we invoke the callbacks.
Fixesgolang/go#32910
Change-Id: I9d060e0d80fd86a317a1d6c7aaa736a8ce10bd07
GitHub-Last-Rev: 04944fa0249c0e6f1022a415787e23abce21bc2e
GitHub-Pull-Request: golang/tools#129
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184880
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change modifies gopls to use the internal goimports library, which
allows us to manually configure the ProcessEnv. We also add a logger to
the ProcessEnv to allow this change not to conflict with gopls's logging
mechanism.
Fixesgolang/go#32585
Change-Id: Ic9aae69c7cfbc9b1f2e66aa8d812175dbc0065ce
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184198
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
As per discussion on golang/go#32810, to avoid the `go list` storm caused by many
files being opened, we check if the file content opened is equivalent to
the content on disk. If so, we mark this file as "on disk" so that we
don't send it as an overlay to go/packages.
Updates golang/go#32810
Change-Id: I0a520cf91bbe933c9afb76d0842f5556ac4e5b28
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184257
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
There was a situation where we were trying to re-acquire a lock that was
already held. This change solves this issue.
Change-Id: I97cf6bad7e7c219a267e3ca5d174a2573f70ebe2
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184217
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change moves from marking a package with `go list` errors as
missing, to marking a package with no files as missing.
Change-Id: Ibad1e67518d8a7f4c4bde416c53ab8132ae534e3
Reviewed-on: https://go-review.googlesource.com/c/tools/+/184039
Reviewed-by: Ian Cottrell <iancottrell@google.com>
There has been a race condition that occasionally appears in test runs
on TryBots. Multiple threads perform type-checking, so they may race on
setting the fields of the *goFiles. Add a mutex to synchronize this.
Change-Id: If52c9d792c6504fc89044964998b06de7dfbd19c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/183978
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
typeCheck() was swallowing context.Canceled errors and leaving the
cached package in a bad state. In particular, after two rapid changes
to imports I was left in the "no package for file" error mode until I
change my imports again. The second change canceled the first change
which ended up sticking a skeleton *pkg in the package cache instead
of propagating the canceled error.
Change-Id: I15b072188c3359d9cd1812bd49e72548ba214250
GitHub-Last-Rev: 240f61718fbb5bfc787bbfaaaae1d38925d7c405
GitHub-Pull-Request: golang/tools#126
Reviewed-on: https://go-review.googlesource.com/c/tools/+/183940
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change adds supports for a package belonging to multiple files.
It requires additional packages.Loads for all of the packages to which a
file belongs (for example, if a non-test file also belongs to a package's
test variant).
For now, we re-run go/packages.Load for each file we open, regardless of
whether or not we already know about it.
This solves the issue of packages randomly belonging to a test or not.
Follow-up work needs to be done to support multiple packages in
references, rename, and diagnostics.
Fixesgolang/go#32791Fixesgolang/go#30100
Change-Id: I0a5870a05825fc16cc46d405ef50c775094b0fbb
Reviewed-on: https://go-review.googlesource.com/c/tools/+/183628
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This change just separates minor changes made along the course of the
memoization CL out into their own change. This will clean up the diffs
in the memoization CL.
Change-Id: I7d59e05ba6472af5f1bf516b1e5b879a5815b9a5
Reviewed-on: https://go-review.googlesource.com/c/tools/+/183250
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
So we can surface their code actions later.
Change-Id: I735e5d025a1250861d49db227f5a79453f599140
Reviewed-on: https://go-review.googlesource.com/c/tools/+/182837
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The `if err :=` block creates a shadow err value that is then discarded
after the block.
Change-Id: I78f6a7298ac5d3d86ece056a9e328bcee9fdc683
Change-Id: I78f6a7298ac5d3d86ece056a9e328bcee9fdc683
GitHub-Last-Rev: 4bac8fb982fa3a0724206422b1a4dbc5e41c34a9
GitHub-Pull-Request: golang/tools#114
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181777
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Without this accessing gof.ast.file.Pos() will panic if gof.ast.file is nil
Change-Id: I829f3667d201f026fcf0475f4fdabce0aced58f8
Change-Id: I829f3667d201f026fcf0475f4fdabce0aced58f8
GitHub-Last-Rev: 777c015d691bf0d14cab5bb0bca6749cc9bdf605
GitHub-Pull-Request: golang/tools#119
Reviewed-on: https://go-review.googlesource.com/c/tools/+/182420
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This changes the packageErrorSpan function into the listErrorSpan.
Previously, this was causing the gopls-generated errors to get parsed,
which would result in attempts to send diagnostics for invalid filenames.
Fixesgolang/go#32603
Change-Id: I7a54ed8884b78beb3f894598f18a24ed232f7412
Reviewed-on: https://go-review.googlesource.com/c/tools/+/182460
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This change does not actually use the token handle for GetToken right
now, but implements the approach for memoizing *token.Files.
Change-Id: I75919f4e97abd6893b202c021adecd2c9dbfc2be
Reviewed-on: https://go-review.googlesource.com/c/tools/+/182277
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This change ensures that we always populate a new set of missing imports
when we re-load a package. Otherwise, we might keep stale missing
imports around.
Change-Id: Id9b6b8201e3ed093a7b9ece97a81e298de612a5e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/182459
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This adds an IDs map to the metadata cache, which maps package paths to
IDs. This is only ever used by the Import function in the type checker.
Change-Id: I8677d9439895bc6cbca5072e3fa9fddad4e165d5
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181683
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Previously when you added a new file to an existing package, the new
file would get stuck with the "no package for file" error until you
saved the file and then made changed a different file in the
package. There were two changes required to fix the errors:
First, we need to invalidate the package cache when a new file is
added to a package so that the package will actually re-parse and
re-type check. We now notice if file names changed when updating a
package's metadata and invalidate the package cache accordingly.
Second, when dealing with overlay (unsaved) files, we need to map
the *goFile to the package even if we fail to parse the
file (e.g. the new file fails to parse when it is empty). If we don't
map it to the package, the package won't get refreshed as the file is
changed.
Fixesgolang/go#32341
Change-Id: I1a728fbedc79da7d5fe69554a5893efcd1e1d902
GitHub-Last-Rev: e7c3d4c1f8f73b12c87ee76d868cc04893e55808
GitHub-Pull-Request: golang/tools#111
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181417
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This change replaces the strings that were previously used for both the
ID and package path fields. This is a precursor to the change that will
replace the uses of package path with package ID.
Change-Id: I353e98aedede9b85c7a183fdd49048ff43b1e26d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181757
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Previously these were only applied from inside parseFiles, which also made it
harder to refactor the remaining parse logic.
This theoretically means fixup is now called in more places than it was before,
but should cause no change in behaviour.
Change-Id: Ic6d006c1d36daca7514626653aaedf90d76e1d0f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181544
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
If the context is canceled (or times out) during parsing, we were
previously caching the package with no *ast.Files. Any further LSP
queries against that package would fail because the package is already
loaded, but none of the files are mapped to the package. Fix by
propagating non-parse errors as "fatal" errors in
parseFiles. typeCheck will propagate these errors and not cache the
package.
I also fixed the package cache to not cache errors loading
packages. If you get an error like "context canceled" then none of the
package's files are mapped to the package. This prevents the package
from ever getting unmapped when its files are updated. I also added a
retry mechanism where if the current request is not canceled but the
package failed to load due to a previous request being canceled, this
request can try loading the package again.
Updates golang/go#32354, golang/go#32360
Change-Id: I466ddb8d336aeecf6e50f9f6d040787a86a60ca0
GitHub-Last-Rev: 5f1e7ef9c883b76a9c1b3636936d91ec0821d922
GitHub-Pull-Request: golang/tools#110
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181317
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We weren't comparing the right values when checking to see if a file's
import statements had changed. This was causing us to not refetch a
package's metadata when needed in certain cases. In particular, if you
typed out an import path by hand you would get stuck with "no metadata
for package" until you added or deleted another import line to trigger
metadata refresh.
Updates golang/go#32516, golang/go#32232
Change-Id: I1c99e9ee56d18f02c44c749d33679aa4e561c620
GitHub-Last-Rev: 20d46b30b520572b05e3228d9e43ba46dfd1be3d
GitHub-Pull-Request: golang/tools#112
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181578
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
We panic if the uri was not a valid file uri instead
They always are a valid file URI, and we would fail miserably to cope if they were
not anyway, and there are lots of places where we need to be able to get the filename
and don't want to cope with an error that cannot occur.
If we ever have not file uri's, you will have to check if it is a file before calling
.Filename, which seems reasonable anyway.
Change-Id: Ifb26a165bd43c2d310378314550b5749b09e2ebd
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181017
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This change marks any unresolved packages discovered as imports by
go/packages.Load. It re-runs go/packages.Load for any package with
missing imports, which will result in the LSP registering changes in
dependencies.
However, this means that we re-run go/packages.Load on
package's with unresolved imports much more than we normally would, so
it may result in a slowdown or unexpected behavior. I'm not sure if this
is necessarily the correct approach here.
Updates golang/go#32232
Change-Id: Id611fa1876e42c88ca2c3e4db30da66dc66945fc
Reviewed-on: https://go-review.googlesource.com/c/tools/+/180537
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This change enables the trimmed ASTs behavior and fixes the broken
tests by caching parse errors along with the ASTs that are being reused.
Change-Id: Ia8e7cdb138c62a8be88659f801506e03f213fe8c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181357
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
This change marks the overlay for a saved file as "on disk".
This will reduce the number of overlays we provide to go/packages, which
can be expensive.
Updates golang/go#31796, golang/go#32457
Change-Id: I8e69503ab80bba29caf4e42491d87e643bf17f1a
Reviewed-on: https://go-review.googlesource.com/c/tools/+/181123
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Peter Weinberger <pjw@google.com>
This change correctly invalidates the cache when we
have to go from a trimmed to untrimmed AST.
The "ignoreFuncBodies" behavior is still disabled due to a racy test.
Updates golang/go#30309
Change-Id: I6b89d1d2140d77517616cb3956721a157c25ab71
Reviewed-on: https://go-review.googlesource.com/c/tools/+/180857
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
On FileHandle Read now just returns the data hash and error
This makes it more obvious that you should handle the error, rather than hiding
it all in a struct.
We also change the way we get and return content, the main source.File
constructs now hold a FileHandle that then updates on invalidation
Change-Id: I20be1b995355e948244342130eafec056df10081
Reviewed-on: https://go-review.googlesource.com/c/tools/+/180417
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
This updates overlays immeditely, and uses the handle identity change to
correctly update the content on demand.
Fixesgolang/go#32348
Change-Id: I3125a6350cac358b7c0f7dc11f2bd11ae1f41031
Reviewed-on: https://go-review.googlesource.com/c/tools/+/179922
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We split aquiring a "handle" from reading a files contents so that we can do the
former eagerly and the latter lazily.
We also "version" the handles so that the same file at different versions is a
different handle.
Change-Id: I06cc346d4b4c77d784aa454702c54689f2f177e0
Reviewed-on: https://go-review.googlesource.com/c/tools/+/179917
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Add 'buildFlags' config to processConfig and pass that value to packages.Config.
We can avoid incorrect diagnostics such as if current source codes require any build tags.
Change-Id: Id191469ec75eedaa82b75ec4fdec084fa78c2c5d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/178782
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
This change trims the function bodies from the ASTs of files belonging to
dependency packages. In these cases, we do not necessarily need full
file ASTs, so it's not necessary to store the function bodies in memory.
This change will reduce memory usage. However, it will also slow down
the case of a user opening a file in a dependency package, as we will
have to re-typecheck the file to get the full AST. Hopefully, this
increase in latency will not be significant, as we will only need to
re-typecheck a single package (all the dependencies should be cached).
Updates golang/go#30309
Change-Id: I7871ae44499c851d1097087bd9d3567bb27db691
Reviewed-on: https://go-review.googlesource.com/c/tools/+/178719
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
