1
0
mirror of https://github.com/golang/go synced 2024-10-01 06:08:32 -06:00
Commit Graph

33 Commits

Author SHA1 Message Date
Ian Cottrell
f13409bbeb internal/telemetry: clean up test data
the current implementation likes to sort maps, so we make sure
tag lists are in sorted order already so that a stable encoder
produces the same result

Change-Id: Ia7ce05f35edb636817c354d9df02de753a48fe1d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/210216
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-12-20 23:47:30 +00:00
Ian Cottrell
c7346ecdc6 internal/telemetry: remove an extraneous comment
There was a fragment of a sentence that must have been from a previous version
(as it talks about a return value for a function that does not have one).

Change-Id: I9d154fe10711344f93e1d49b68a811dbc9772710
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212241
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-12-20 23:47:11 +00:00
Ian Cottrell
814139985e internal/telemetry: obey the onlyErrors flag in the log writer
has no impact because there are no use cases that don't set it to true right now

Change-Id: I2bc485226078c710bdc36397b96755cdce82d9cc
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212242
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-12-20 23:46:55 +00:00
Ian Cottrell
2f6d8bf0ad internal/lsp: change tests to use the main exporter
Now the tests are at a high enough level, we can
switch them to using the full exporter by bulding
a fake http client to bind it to. This lets us
use the true public interface and also excercise
more of the functionality in the tests.
With this we are now ready to replace the entire
implementation safely.

Change-Id: Ifdbf6230de3ec7c7c5381c840b135cb7a0bc1e55
Reviewed-on: https://go-review.googlesource.com/c/tools/+/209161
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-12-20 19:55:05 +00:00
Emmanuel T Odeke
2ad5dca7a5 telemetry/log: sample reference for benchmarking harness
Serves as a reference harness to get benchmarking started.
Steps to run benchmark:

$ go test -run=^$ -bench=. -count=10

After you grab the output, put the various comparisons
into files before.txt and after.txt then edit those
result names to have a common name e.g.
    s/BenchmarkLoggingNoExporter/BenchmarkIt/g
    s/BenchmarkNoTracingNoMetricsNoLogging/BenchmarkIt/g
    s/BenchmarkLoggingStdlib/BenchmarkIt/g

Now run benchstat:

* All compared
$ benchstat no_log.txt tellog.txt stdlog.txt
name \ time/op    no_log.txt  tellog.txt   stdlog.txt
It-8              289ns ± 2%  2780ns ± 3%  5100ns ± 3%

name \ alloc/op   no_log.txt  tellog.txt   stdlog.txt
It-8              80.0B ± 0%  728.0B ± 0%  568.0B ± 0%

name \ allocs/op  no_log.txt  tellog.txt   stdlog.txt
It-8               5.00 ± 0%   32.00 ± 0%   28.00 ± 0%

* No logging vs telemetry log
$ benchstat no_log.txt tellog.txt
name  old time/op    new time/op    delta
It-8     289ns ± 2%    2780ns ± 3%  +862.31%  (p=0.000 n=10+9)

name  old alloc/op   new alloc/op   delta
It-8     80.0B ± 0%    728.0B ± 0%  +810.00%  (p=0.000 n=10+10)

name  old allocs/op  new allocs/op  delta
It-8      5.00 ± 0%     32.00 ± 0%  +540.00%  (p=0.000 n=10+10)

* No logging vs Standard library "log"
$ benchstat no_log.txt stdlog.txt
name  old time/op    new time/op    delta
It-8     289ns ± 2%    5100ns ± 3%  +1665.16%  (p=0.000 n=10+9)

name  old alloc/op   new alloc/op   delta
It-8     80.0B ± 0%    568.0B ± 0%   +610.00%  (p=0.000 n=10+10)

name  old allocs/op  new allocs/op  delta
It-8      5.00 ± 0%     28.00 ± 0%   +460.00%  (p=0.000 n=10+10)

* telemetry log vs Standard library "log"
$ benchstat tellog.txt stdlog.txt
name  old time/op    new time/op    delta
It-8    2.78µs ± 3%    5.10µs ± 3%  +83.43%  (p=0.000 n=9+9)

name  old alloc/op   new alloc/op   delta
It-8      728B ± 0%      568B ± 0%  -21.98%  (p=0.000 n=10+10)

name  old allocs/op  new allocs/op  delta
It-8      32.0 ± 0%      28.0 ± 0%  -12.50%  (p=0.000 n=10+10)

Change-Id: I53b15e9da315615278c576f3a60108435417a9f7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/212078
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
2019-12-19 18:19:13 +00:00
Ian Cottrell
99399511c1 internal/telemetry: lift the tests up to the request level
rather than testing events or metrics directly, we
test them in the context of a full message that would
get sent to the agent

Change-Id: I238a7f9ab7b1456d1b4b2bac2519d814928f2283
Reviewed-on: https://go-review.googlesource.com/c/tools/+/209098
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-27 15:41:43 +00:00
Ian Cottrell
7360bd5c0f internal/telemtry: changed to a simpler threading model for stats
this is not a final solution, but it makes it easier to debug
and reason about, and does not require a go routine or buffered
channel

Change-Id: I758758ac80fcd525ab5264e34c48941766a8db11
Reviewed-on: https://go-review.googlesource.com/c/tools/+/208664
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-26 22:52:16 +00:00
Ian Cottrell
9fe613bd66 internal/lsp: build the wire.Node lazily
this is needed to move to a model where we do not need to have
the wire form at all

Change-Id: I3b3693e027b568de4e8b4d10f7d5dd022a616e2e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/208958
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-26 22:51:56 +00:00
Ian Cottrell
fc82735a89 internal/telemetry: delay the conversion of metrics
convert to wire form only when we need to encode it

Change-Id: Ib36ee2fbef773241a30b9aa707ebf311119b8705
Reviewed-on: https://go-review.googlesource.com/c/tools/+/208658
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Nathan Dias <nathan.dias@orijtech.com>
2019-11-26 18:15:43 +00:00
Ian Cottrell
89d49d945a internal/telemetry: make the metrics tests external
This is the same changes we already made to the trace tests, where we check the
encoded json instead of the wire structs, and we make the tests external using
the public encoding function only

Change-Id: I289d46dd5ad0efe0b2fba0a71b188b4f9edc5676
Reviewed-on: https://go-review.googlesource.com/c/tools/+/208504
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-25 14:11:28 +00:00
Ian Cottrell
f5828fc9a1 internal/telemetry: merge the metrics tests
This merges all the metrics tests into something that converts top level metrics
only, and relies on that to cover all the lower level functions

This is in preparation for moving the tests to be external without exposing all
those functions.

Change-Id: Iecce1ddc522db9fc926817e4e748d718cbc740a9
Reviewed-on: https://go-review.googlesource.com/c/tools/+/208499
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-24 02:19:06 +00:00
Ian Cottrell
2a6ccf25d7 internal/telemetry: compare the compact JSON
It is sufficient to just compact the JSON to compare it rather than
bothering to decode it.

Change-Id: Ied39d462eec77623187422a9c58f83f8d1630269
Reviewed-on: https://go-review.googlesource.com/c/tools/+/208498
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-22 23:29:04 +00:00
Nathan Dias
f774e2e2e5 internal/telemetry/export/ocagent: update usages of BucketOptions to use pointers
This change updates initializations of DistributionValue to pass in a pointer for
BucketOptions rather than a value so that the custom MarshalJSON for
*BucketOptionsExplicit will be called when marshalling distributions.

Updates golang/go#33819

Change-Id: I6d1a393dfa67d626bf5cdd2cfbc0c931d376f60c
Reviewed-on: https://go-review.googlesource.com/c/tools/+/208401
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-22 08:00:28 +00:00
Nathan Dias
df8e87c2ce internal/telemetry/export/ocagent: add tutorial for exporting metrics
This change adds a README that describes how to export metrics from
Go tools to Prometheus via OpenCensus.

Updates golang/go#33819

Change-Id: Ic417b539fd5ef95fee642321bd30997146cbedb3
Reviewed-on: https://go-review.googlesource.com/c/tools/+/208400
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-22 07:16:40 +00:00
Nathan Dias
2d3bb8ce05 internal/telemetry/export/ocagent: correctly JSON marshal BucketOptionsExplicit
This change adds a custom MarshalJSON method to BucketOptionsExplicit for
marshalling parity with protobuf/jsonpb. This allows the OpenCensus service
to correctly decode a distribution's BucketOptions.

Updates golang/go#33819

Change-Id: Ia9dc868e1cbfc32a956f6a276dfd1591f7d4d31a
Reviewed-on: https://go-review.googlesource.com/c/tools/+/208398
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-22 06:42:13 +00:00
Nathan Dias
96ad48e4b0 internal/telemetry/export/ocagent: correctly marshal points to JSON
This change adds a custom MarshalJSON func to Point so that values
are formatted the same way jsonpb formats them. This allows the
OpenCensus service to determine the concrete type of the point's
value when unmarshaling.

What works:
* Points with Int64, Double, and Distribution values will marshal correctly.

What does not work:
* Points with Summary values will not marshal.

Updates golang/go#33819

Change-Id: Ia76ebff4e0e2b6ff2ddf72b8d6187f49069d4cad
Reviewed-on: https://go-review.googlesource.com/c/tools/+/207838
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-21 17:24:11 +00:00
Ian Cottrell
947d4aa893 internal/lsp: make the ocagent test external
They now call a public encode function and have no knowledge of the private
conversion functions or any of the contents of the wire package.

Change-Id: I4364a4d9d1efe4bc872627556e537936b5880231
Reviewed-on: https://go-review.googlesource.com/c/tools/+/207903
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-21 04:05:51 +00:00
Ian Cottrell
9c44060bd0 internal/telemetry: delay the conversion to wire format
do it just as we are encoding, this is in preparation for making the
encoding faster and cheaper

Change-Id: Ic9715e2750adf276eac152e321d96e249190eacb
Reviewed-on: https://go-review.googlesource.com/c/tools/+/207902
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-21 03:35:56 +00:00
Ian Cottrell
95cb2a1a7e internal/telemetry: make test event with functions
This will allow us to use the public API to generate the events.
This avoids creating the structs by hand.
It also helps tests the other API.

Change-Id: Ic90cbbaf6fc97c2a3e6a5ff64dccdff0d65ec865
Reviewed-on: https://go-review.googlesource.com/c/tools/+/206885
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
2019-11-13 22:35:46 +00:00
Ian Cottrell
023c5eeaf4 internal/lsp: change annotation tests to json
This checks against the json form rather than the wire structs
This will allow the encoding method to change without the test changing.

Change-Id: Iefe84cc03bc6e82627acd031e0991c31b4fed315
Reviewed-on: https://go-review.googlesource.com/c/tools/+/206884
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
2019-11-13 22:35:34 +00:00
Nathan Dias
be03d4f470 internal/telemetry/export/ocagent: attach timestamps to metrics
This change attaches start timestamps to timeseries and end
timestamps to the points in each timeseries. Int64Data,
Float64Data, HistogramInt64Data, and HistogramFloat64Data
have also had an EndTime field added to keep track of the last
time the metric was updated.

What works:
* Start and end timestamps will now be attached to timeseries.

What does not work yet:
* MetricDescriptors will not have a unit attached.
* No labels will be attached to timeseries.
* Distributions will not have SumOfSquaredDeviation attached.

Updates golang/go#33819

Change-Id: I692e1676bb1e31de26c1f799b96428fc9a55d6c7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/203060
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-10-24 07:31:07 +00:00
Nathan Dias
e4d7c6f25b internal/telemetry/export/ocagent: convert Histogram metrics
This change adds support for converting HistogramFloat64Data and
HistogramInt64Data to *wire.Metric. Timestamps are not attached
as they are not yet available.

What works:
* convertMetric will now convert HistogramInt64Data and
HistogramFloat64Data to *wire.Metric.

What does not work yet:
* StartTime and EndTime will not be attached to timeseries and
points.
* MetricDescriptors will not have a unit attached.
* No labels will be attached to timeseries.
* Distributions will not have SumOfSquaredDeviation attached.

Updates golang/go#33819

Change-Id: Iee52ab751542ee1ade07ef32120de853b41fd27b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/200538
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-10-17 16:38:57 +00:00
Emmanuel T Odeke
7667e13ae9 internal/telemetry: modify *Subscriber to take timeAt
Modifies the various Subscriber signatures to take in
an "at" time.Time field, which captures the time that
a measurement was made. This field will then be used
when exporting points as their EndTime.

Fixes #34490.

Change-Id: I9e87b65e374876ae3d4df0abedb64bb7dcb221b6
Reviewed-on: https://go-review.googlesource.com/c/tools/+/199577
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
2019-10-11 20:42:57 +00:00
Nathan Dias
0337d82405 ocagent: convert Int64Data and Float64Data metrics to *wire.Metric.
Histogram data still needs to be converted and
timestamps are not attached as they are not yet available.

What works:

* convertMetric will now convert Int64Data and Float64Data.

What does not work yet:

* Histogram64Int and Histogram64Float will still not be converted.
* StartTime and EndTime will not be attached to timeseries and points.
* MetricDescriptors will not have a unit attached.
* no labels will be attached to timeseries.

Updates golang/go#33819

Change-Id: I65f9af716ba6282e809d0a9d10777d70475e4c83
GitHub-Last-Rev: 10820a9971e1f4c0529fadc567b2533256c2e961
GitHub-Pull-Request: golang/tools#170
Reviewed-on: https://go-review.googlesource.com/c/tools/+/199857
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
2019-10-10 07:50:00 +00:00
Ainar Garipov
feee8acb39 all: fix more typos
Change-Id: I978ad5e1800ebfceb78aaced438331a8341715d4
Reviewed-on: https://go-review.googlesource.com/c/tools/+/194697
Reviewed-by: Toshihiro Shiino <shiino.toshihiro@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-11 15:13:14 +00:00
George Hartzell
fa2c3f315e internal/telemetry/export/ocagent: fix typo in comment
Change-Id: I26a6950518441850481f66d0c8844b16e908cffd
GitHub-Last-Rev: f83d960fddfa3ac65a7b766f7fc549ef80de560d
GitHub-Pull-Request: golang/tools#151
Reviewed-on: https://go-review.googlesource.com/c/tools/+/193917
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-06 17:19:43 +00:00
Jan Steinke
11cc3c157e internal/lsp: add additional information when logging errors
Fixes golang/go#33499

This logs additional information compatible with `golang.org/x/xerrors` like the frames.

Change-Id: If25c3979cf294dbe55b0e3d9d999b24c1ff8900d
GitHub-Last-Rev: 210fa40bd29de99a9052eb67d197154f4bd74e10
GitHub-Pull-Request: golang/tools#144
Reviewed-on: https://go-review.googlesource.com/c/tools/+/189344
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-26 18:41:59 +00:00
Ian Cottrell
65e3620a7a internal/telemetry: add the ability to flush telemetry data
Change-Id: I5de33edc352a5202fdf9e38538a224dd6adafedb
Reviewed-on: https://go-review.googlesource.com/c/tools/+/190799
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-08-21 16:29:56 +00:00
Ian Cottrell
fa80cb575d internal/telemetry: pass the http.Client to the ocagent
This will allow us to configure the connection at need.
It will also allow us to intercept the content for tests.

Change-Id: Id7d34f2d56f233eae112bea97cccab1f2a88de55
Reviewed-on: https://go-review.googlesource.com/c/tools/+/190798
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-08-21 16:29:14 +00:00
Ian Cottrell
d9ab56aa29 internal/telemetry: change concurrency model
This changes to use a mutex and directly execute the less performance
sensitive telemetry calls (tracing and logging) and then uses a submission
queue only for stats adjustments as those are much more sensitive (but it
should also be easier to keep up with them in bursts)

Fixes golang/go#33692

Change-Id: Ia59a8975f21dfbfcf115be1f1d11b097be8dd9c8
Reviewed-on: https://go-review.googlesource.com/c/tools/+/190737
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-08-20 03:27:56 +00:00
Ian Cottrell
5b08f89bfc internal/telemetry: clean up the exporter api
this shuffles things so there a single exporter API rather than an observer
It also removes most of the globals.
per telemetry type.

Change-Id: Iaa82abe2ded1fff9df8e067ed4a55bcbd9d9591f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/190405
Run-TryBot: Ian Cottrell <iancottrell@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-08-15 23:56:12 +00:00
Ian Cottrell
922a4ee32d internal/telemetry: extract units to their own package
This makes the code read slightly better, and more closely
aligns with the open telemetry code.

Change-Id: I87eaf7d08b802f7862f896f2654456ee6a7681e3
Reviewed-on: https://go-review.googlesource.com/c/tools/+/190404
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-08-15 21:28:32 +00:00
Ian Cottrell
156eb2ae29 internal/lsp: split the telemetry library out
This is a straight move of some code with no changes.
It splits the part of the telemetry code that will become a standalone library from the bit that belongs in the lsp.

Change-Id: Icedb6bf1f3711da9251450531729984df6df7787
Reviewed-on: https://go-review.googlesource.com/c/tools/+/190403
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-15 21:28:02 +00:00