jba
Repos
63
Followers
179

Errors as side notes

39
0

Fast, sparse bitsets

20
1

demo of detail error formatting

8
2

Checking Go templates for validity.

22
1

experiments in using Go generics

8
1

Events

issue comment
proposal: log/slog: structured, leveled logging

@jellevandenhooff, regarding your alternative design for contexts and loggers: I think you'll find that people will now have to pass both Loggers and contexts around together. From there it's hard to resist adding the Logger to the context.

Created at 5 days ago
issue comment
proposal: log/slog: structured, leveled logging

@deefdragon, thanks for the data. Looks like it is reasonable to omit Trace. For Fatal, we decided not to complicate Handlers further by adding flushing. If, as we suspect, most log.Fatal uses are in CLI programs, then most of those uses wouldn't change to slog anyway—CLI tools are probably better off with the formatted output of log.

Created at 5 days ago
issue comment
proposal: log/slog: structured, leveled logging

Though the ergonomics for text vs JSON logging aren't wonderful -- the text logger prints an array of raw bytes instead.

@jellevandenhooff, https://go-review.googlesource.com/c/exp/+/453495 will fix that.

Created at 5 days ago
issue comment
x/vulndb: potential Go vuln in github.com/codenotary/immudb: CVE-2022-39199

Duplicate of #1118

Created at 1 week ago
issue comment
x/vulndb: potential Go vuln in github.com/codenotary/immudb: CVE-2022-36111

Duplicate of #1117

Created at 1 week ago
issue comment
x/vulndb: potential Go vuln in github.com/tailscale/tailscale: CVE-2022-41924

Duplicate of #1120

Created at 1 week ago
issue comment
x/vulndb: potential Go vuln in github.com/tailscale/tailscale: CVE-2022-41925

Duplicate of #1119

Created at 1 week ago
issue comment
x/vulndb: potential Go vuln in github.com/codenotary/immudb/pkg/client: GHSA-672p-m5jq-mrh8

@julieqiu

Created at 1 week ago
issue comment
x/vulndb: potential Go vuln in github.com/lightningnetwork/lnd: CVE-2022-39389

The package is importable, and has importers. @julieqiu

Created at 1 week ago
issue comment
x/vulndb: potential Go vuln in github.com/duke-git/lancet: CVE-2022-41920

This is an importable package, but has zero importers. @julieqiu

Created at 1 week ago
issue comment
proposal: log/slog: structured, leveled logging

@deefdragon, The fact that two major log packages omit TRACE was enough evidence for us to omit it. I believe it's better to leave it out now and add it later if necessary.

Created at 1 week ago
issue comment
proposal: log/slog: structured, leveled logging

Is this a fair summary of how to avoid global state?

  • using slog.Info, slog.Error calls
  • using slog.FromContext on a context with no logger
  • using the "log" package
  • importing any code that does any of the above

@AndrewHarrisSPU, looks right to me, once you add calling slog.Default.

To anticipate a question, yes, calling slog.FromContext on a context with no logger is the one case you can't easily catch statically. But we feel the convenience of being able to log from anywhere outweighs that.

Created at 1 week ago
issue comment
proposal: log/slog: structured, leveled logging

@ancientlore, good point about a constant for "err". Added in https://go.dev/cl/451293.

Regarding your other concerns, I think it's best if we keep ReplaceAttr simple, at least until we see significant demand for the kind of features you want. For now it would be best to define your own handler. If you just need to pre-process attributes, your handler can probably wrap a JSONHandler.

Created at 1 week ago
issue comment
proposal: log/slog: structured, leveled logging

As a third-party package author, I would like guidance on the where to get my logger from.

Speaking for myself, I generally use the context if one is passed down, HTTP request handlers being the prime example. It's also fine to add it to a struct that the package exports. For packages that have both, I would think about whether I want common attributes to be part of the dynamic call chain, or associated with the struct value.

Created at 1 week ago
issue comment
proposal: log/slog: structured, leveled logging

When logging another string containing JSON-encoded data, I found myself wishing that the output included that data without escaping. But I can also see how passing through raw JSON (not created with explicit groups) might cause problems in downstream log processing.

@jellevandenhooff, does json.RawMessage help? If not, can you give an example?

Created at 1 week ago
issue comment
proposal: log/slog: structured, leveled logging

One thing I found surprising is that a slog.Value doesn't work using the two-argument style logging API.

@jellevandenhooff, This is a bug, fixed in https://go.dev/cl/451291.

Created at 1 week ago
issue comment
proposal: log/slog: structured, leveled logging

It seems like, if there's a global/default logger, a global/default reference level would also make sense? HandlerOptions could be made to observe that - if the Level field is nil, take whatever reference Leveler the global/default is using, or export that Leveler from slog, something like this?

@AndrewHarrisSPU, currently the built-in Handlers and their options are self-contained—they make no reference to global state. I think we should keep it that way. Think of it from the perspective of someone who is trying to avoid all global state. They can do so by avoiding the default logger and its global functions, always creating their own loggers and handlers. They wouldn't want global state smuggled back in by omitting an option.

Created at 1 week ago
issue comment
proposal: log/slog: structured, leveled logging

doc improvement : specify in the documentation that Logger.WIth (resp Logger.WithGroup) call Handler.WithAttrs (resp Handler.WithGroup) on the logger's handler.

@LeGEC, this was done for With. https://go.dev/cl/451290 adds it for WithGroup.

Created at 1 week ago
issue comment
x/vulndb: potential Go vuln in github.com/shamaton/msgpack/v2: GHSA-jr77-8gx4-h5qh

Duplicate of #972

Created at 2 weeks ago
issue comment
x/vulndb: potential Go vuln in github.com/shamaton/msgpack/v2: GHSA-jr77-8gx4-h5qh

Duplicate of #972.

Created at 2 weeks ago
issue comment
x/vulndb: potential Go vuln in github.com/russellhaering/gosaml2: GHSA-gq5r-cc4w-g8xf

Duplicate of #46

Created at 2 weeks ago
issue comment
x/vulndb: potential Go vuln in github.com/russellhaering/gosaml2: GHSA-prjq-f4q3-fvfr

Duplicate of #46

Created at 2 weeks ago
issue comment
proposal: log/slog: structured, leveled logging

I'd make the log level be the last part of the chain, so it's (almost) impossible to forget it

@lpar, then you couldn't filter on level until you'd done all the work. That's a big disadvantage for systems that have tons of debug logging that they want disabled most of the time.

You've found the great weakness of this style of API for logging: you either give up early filtering or risk forgetting the terminator.

I'd like switching the level at runtime to be a one-liner.

Here is how to switch the level for the entire program at runtime.

Create a LevelVar:

var globalLevel = new(slog.LevelVar) // Info by default

Set up a default logger that uses it:

func main() {
    slog.SetDefault(slog.New(slog.HandlerOptions{Level: globalLevel}.NewJSONHandler(os.Stderr)))
    ...
}

When you want to turn on debug logging, set the LevelVar:

globalLevel.Set(slog.DebugLevel)

That's it.

Created at 2 weeks ago
issue comment
proposal: log/slog: structured, leveled logging

the design of the convenience functions Debug, Info, Warn and Error doesn't make a clear distinction between (a) levels, and (b) logging an error type

@tehsphinx, that's deliberate (and controversial. We think it will be an ergonomic win overall, since the large majority of calls at the Error level have an associated error. Yes, to downgrade an Error to a Warning you have to turn the error value into an attribute; but you didn't have to make it an attribute in the first place, and most Errors don't get downgraded, so the total toil is less.

An error type could be added to the errors package that can transport fields and when the error is logged the fields are taken from the error and logged, too.

This is easy enough to do on your own: just define your own error type with the fields you want, and give it a LogValue method to log those fields. I don't think anything we could add to the errors package would be as good, because it would have to use something like a map[string]any for the fields, which would be slow and type-unsafe.

As for stack traces, you can always write a handler to output one for the log call. Stack traces in errors are out of scope for this proposal.

Created at 2 weeks ago
issue comment
proposal: log/slog: structured, leveled logging

@cschneider4711:

Correlation-ID for a Request

See this discussion item for motivation. We have Logger.WithContext and slog.Ctx to support this.

Ring-Buffer style of Debug-preserving Appending on Error-Conditions

It sounds like you could implement this as a handler.

Secret Filter for Logs

Redacting secrets and PII is definitely important for logs. We expect most production-strength handlers will provide a way to do this. Our built-in handlers support the ReplaceAttr option, which allows you to provide a function to filter the attributes of a log record.

Created at 2 weeks ago
issue comment
proposal: log/slog: structured, leveled logging

@cschneider4711, good thoughts about security. But I don't think Security should be a level. You said it yourself:

... ease counting and grouping of such events from logs by having a clearer distinction from the other logs as being a potential security-event...

Exactly—you want to easily filter security-relevant logs. For that, you just need a boolean attribute named "security". As for standardizing on the name or having a convenience function, I'll give my standard answer: we just don't know the exact shape this feature would need to take, so it's better to leave it out now and think about adding it later.

Created at 2 weeks ago
issue comment
proposal: log/slog: structured, leveled logging

I came up with a reflect-based approach to automate this

@ancientlore, reflecting on structs and looking at field tags seems like a good idea to me; I imagine we'll see several implementations like yours. But I don't believe it's ready to be in this proposal. It could be something we add later, when we're more clear on the desired features.

Created at 2 weeks ago
issue comment
proposal: log/slog: structured, leveled logging

I found a little issue in the text handler @ancientlore, thanks for finding the bug. Fixed in https://go-review.googlesource.com/c/exp/+/449696.

Created at 2 weeks ago
issue comment
x/pkgsite: wrong file links for submodules

We do, code is at https://go.googlesource.com/pkgsite/+/refs/heads/master/internal/source. I retracted that proposal (read the end). I just closed it to avoid confusion.

Created at 2 weeks ago
closed issue
proposal: x/pkgsite: new source meta tag

(Edited: now a proposal.)

This issue describes a new HTML meta tag for referring to Go source files in online documentation. It is an official Go proposal, though it doesn't affect the Go language or tools.


For many years, the go-source meta tag has allowed godoc.org and other source-browsing tools to provide links to Go source for import paths that use the go-import meta tag. With the advent of modules, the go-source meta tag in its current form cannot be used, because it does not support versions. While we could just extend go-source to add a parameter to the templates, we could also take this opportunity to improve it in other ways.

We propose a new tag, go-source-v2, with the following properties:

  • Module versions are supported.
  • Source files are described relative to modules rather than packages.
  • Additional information can be provided, so that module-browsing tools like pkg.go.dev can display repo information and render README and similar files.
  • The anomalies listed at the end of the current spec are resolved.

Structure

For certain common code-hosting sites, like GitHub and Bitbucket, no go-source-v2 tag is necessary. See "Implicit Source Information" below for details.

For a module with path M, the tag should appear in the <head> of the page served by GETting https://M?go-get=1. The tag should look like

<meta name="go-source-v2" content="home directory file line-suffix raw">

where:

  • home is the URL of the repo root. If _, then the repo root (third component) of the go-import tag on the same page is used; if there is no go-import tag or the tag’s second component is mod, then no repo is specified. This does not preclude serving source files but it does prevent tools from linking to the repo or providing repo-based signals, like number of stars and forks.
  • directory is a URL template for linking to a directory of files. It supports two parameters:
    • {revision} is replaced by an identifier for the (approximate) VCS revision. See "Revision Parameters" below for more.
    • {dir} is replaced by the directory relative to the module (not repo) root.
  • file is a URL template for linking to an entire file. In addition to {revision} and {dir}, it also supports {file}, the basename of the file.
  • line-suffix will be appended to file to obtain a URL for a file at a particular line. It supports only the parameter {line}, the 1-based integer line number.
  • raw is a URL template for linking to the raw contents of a file. It supports the {revision}, {dir} and {file} parameters as defined above. While file should display a file for people (with line numbers and syntax highlighting, perhaps), raw should serve the raw bytes of the file. It can be used to rewrite links in README files and the like.

After a tool replaces a template’s parameters, it should remove doubled and trailing slashes. This should make go-source’s {/dir} parameter unnecessary. In theory, a site could serve a path differently depending on whether it had a trailing slash, but we are unaware of any code-hosting site that makes this distinction.

Any component of the tag’s contents can be omitted by using an underscore.

Here’s an example of the go-import and go-source-v2 meta tags for the gopkg.in/yaml.v2 module:

<meta name="go-import" content="gopkg.in/yaml.v2 git https://gopkg.in/yaml.v2">

<meta name="go-source-v2" content="
    github.com/go-yaml/yaml
    https://github.com/go-yaml/yaml/tree/{revision}/{dir}
    https://github.com/go-yaml/yaml/blob/{revision}/{dir}/{file}
    #L{line}
    https://github.com/go-yaml/yaml/raw/{revision}/{dir}/{file}
">

Revision Parameters

Tools should derive {revision} from the module version as follows:

  • For pseudo-versions, use the commit hash (the part after the final hyphen).
  • For semantic versions, use the version after removing any +incompatible suffix. Use other version specifiers (like master) as is.

For a nested module, {revision} is not actually the tag name. A nested module N at version v1.2.3 has tag N/v1.2.3, but {revision} will be v1.2.3. The templates must account for this. For instance, if example.com served directories using GitHub-style URLs, and example.com/mod/nest were a nested module under example.com/mod, then its directory template might be https://example.com/mod/tree/nest/{revision}/nest/{dir}. The first occurrence of nest is part of the tag that identifies the version of example.com/mod/nest.

Implicit Source Information

If the https://M?go-get=1 page for module M has a go-import meta tag that refers to a repo whose domain matches one of the following glob patterns, then no go-source-v2 tag is needed:

  • github.com
  • bitbucket.org
  • *.googlesource.com
  • gitlab.com
  • gitlab.* (if the site behaves like gitlab.com)

The templates for these sites are well-known, and are provided below.

There is one problem: for a major version greater than 1, the templates for “major branch” and “major subdirectory” conventions differ (See https://research.swtch.com/vgo-module for a discussion of these conventions.) To determine the right template, make a HEAD request for the go.mod file using each template, and select the one that succeeds. For example, for module github.com/a/b/v2 at version v2.3.4, probe both github.com/a/b/blob/v2.3.4/go.mod (the location of the go.mod file using the “major branch” convention) and github.com/a/b/blob/v2.3.4/v2/go.mod (its location using “major subdirectory”).

Standard Patterns

In these patterns, REPO is the repo URL and MS is the suffix of the module path without the repo prefix. These can be determined from the go-import tag and the path of the go-get=1 URL.

github.com:

  • directory: REPO/tree/{revision}/MS/{dir}
  • file: REPO/blob/{revision}/MS/{dir}/{file}
  • line suffix: #L{line}
  • raw: REPO/raw/{revision}/MS/{dir}/{file}

gitlab.com, gitlab.*:

  • directory: REPO/tree/{revision}/MS/{dir}
  • file: REPO/blob/{revision}/MS/{dir}/{file}
  • line suffix: #L{line}
  • raw: REPO/raw/{revision}/MS/{dir}/{file}

bitbucket.org:

  • directory: REPO/src/{revision}/MS/{dir}
  • file: REPO/src/{revision}/{dir}/MS/{file}
  • line suffix: #lines-{line}
  • raw: REPO/raw/{revision}/MS/{dir}/{file}

*.googlesource.com:

  • directory: REPO/+/{revision}/{dir}
  • file: REPO/+/{revision}/{dir}/{file}
  • line suffix: #{line}
  • raw: not supported

Sites that won’t work

Code-hosting sites running Gitea cannot be accommodated by the source linking scheme described above, or indeed by any scheme that has only the information available from the module zip. Gitea source URLs are different for branches, tags and commit hashes, and for the last only the full hash will work. Since revisions should always be tags, the templates for a Gitea site can use the tag form of the source URL. But there is no template that will work with the abbreviated hash at the end of a pseudo-version.

While a source-browsing tool could clone the repo and resolve the abbreviated hash locally, that work should be outside the scope of the tool. Instead, we suggest that a gitea.com contributor add URL routes that can work with partial hashes.

The same problem exists for code.dumpstack.io (which appears to be a rebranded gitea).

Whatever software is used for https://blitiri.com.ar/ has the same issues, and one additional one: there doesn’t seem to be any URL for tags.

Created at 2 weeks ago