brancz
Repos
151
Followers
744
Following
6

Continuous profiling for analysis of CPU and memory usage, down to the line number and throughout time. Saving infrastructure cost, improving performance, and increasing reliability.

2702
148

❄️ Coolest database around 🧊 Embeddable column database written in Go.

831
45

eBPF based always-on profiler auto-discovering targets in Kubernetes and systemd, zero code changes or restarts needed!

227
35

The Prometheus monitoring system and time series database.

45634
7377

Production-Grade Container Scheduling and Management

94049
32966

Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.

11208
1680

Events

issue comment
PromQL parser mistakingly interpreting UTF8 escape character

@brancz Hm yeah. Optimally the autocompletion would be able to understand the string context it's in (double-quoted string vs. backtick-quoted) and escape any inserted label values accordingly, if necessary.

Yeah agreed, that's option 2 that I suggested tl;dr the parser is behaving as expected, but the query builder/tooling is not quite.

Created at 2 days ago
issue comment
PromQL parser mistakingly interpreting UTF8 escape character

The issue is really just that the table output is really dumb and doesn't do any escaping (besides automatic HTML safety stuff):

It's not just the table though, when you autocomplete and select the suggested value I think the behavior is unexpected. Quick screen recording to demonstrate what I mean:

https://user-images.githubusercontent.com/4546722/204038170-011c5f6e-c8ca-4e39-b53b-9a0d2797cc58.mov

Created at 2 days ago
issue comment
PromQL parser mistakingly interpreting UTF8 escape character

cc @roidelapluie @juliusv

Created at 2 days ago
issue comment
PromQL parser mistakingly interpreting UTF8 escape character

I don’t think we’ve established yet whether the parser is faulty or the tooling around query completion is behaving inconsistently.

Created at 2 days ago
opened issue
PromQL parser mistakingly interpreting UTF8 escape character

What did you do?

We use the PromQL parser in the Parca project. We noticed something interesting when label values contained the \x2d value. Ultimately we noticed that the PromQL parser interprets this, not as the string value, but rather interprets the UTF-8 escape character, which in this case is the dash (-).

A resulting inconsistency can be triggered using the following config:

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]
        labels:
          faulty_label: \x2d

Then query Prometheus, for example using the prometheus_build_info metric.

If we then use the UI to copy the label-matcher, and insert it into the query and query, however, we get an empty result.

Only if we explicitly escape the label value do we get the result we were looking for with the previous query:

What did you expect to see?

I expected one of the following two, for Prometheus' behavior to be consistent within itself:

  • The PromQL parser retains the actual string literal and does not interpret the UTF-8 escape character.
  • Table renders the label-value in its escaped form, the autocompletion suggests the label-value in its escaped form, and therefore copying the matcher by clicking on it in the table also uses the escaped form.

What did you see instead? Under which circumstances?

Described in the first section.

System information

Darwin 21.6.0 arm64

Prometheus version

$ ./prometheus --version
prometheus, version 2.40.3 (branch: HEAD, revision: 84e95d8cbc51b89f1a69b25dd239cae2a44cb6c1)
  build user:       root@692b092db288
  build date:       20221124-09:15:54
  go version:       go1.19.3
  platform:         darwin/arm64


### Prometheus configuration file

```yaml
scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:9090"]
        labels:
          faulty_label: \x2d

Alertmanager version

n/a

Alertmanager configuration file

n/a

Logs

n/a
Created at 2 days ago
brancz delete branch vercel_deploy_hook
Created at 3 days ago

Re-deploy Parca Docs after release

Fixes #7

Merge pull request #8 from parca-dev/vercel_deploy_hook

Re-deploy Parca Docs after release

Created at 3 days ago
Auto deploy parca.dev after a release

https://github.com/parca-dev/parca-agent/blob/44b5a4154a6bbdee010d7dfae7d9363b45a6e3ad/.github/workflows/release.yml#L146

Created at 3 days ago
pull request closed
Re-deploy Parca Docs after release

Fixes #7

Secret already added.

Created at 3 days ago
issue comment
Update Parca Agent docs

Is this a draft because it’s incomplete? I’d be more than happy to merge partial updates getting things out as we get them done.

Created at 3 days ago
issue comment
Encode PromQL Matricies with Arrow

Is there a particular reason we’re doing this in the query range API? I personally find this an odd placement and usage of arrow, typically arrow the format used within a query engine for zero copy computations. Have we considered doing this rather for remote read? That seems like the more natural space for it to me. And streaming there would also work very well as we already have machinery available from the proto streaming variant.

Created at 3 days ago
issue comment
pkg/profile: Replace gzip package to reduce allocs

I'm especially impressed that this didn't need any pooling of byte buffers, just the replacement 🤯

Created at 3 days ago
issue comment
pkg/profile: Replace gzip package to reduce allocs

Just deployed this on the demo cluster, and I'm happy to report that this got us a healthy ~20% decrease in CPU usage, nice job, and thank you very much!

Created at 3 days ago
issue comment
.NET support

I know next to nothing about .NET, so I can only go by docs or other web searches. From the docs, I understand that ahead-of-time compilation to a regular self-contained Linux ELF binary (which should work just fine with Parca Agent if debuginfos are available) is only supported since .NET 7, which appears to have only been released on November 8th, 2022, so it appears to me it's rather the bleeding edge than an old feature.

Created at 3 days ago
issue comment
--cors-allowed-origins issue

If so, could you run date on all machines and share the output?

Created at 3 days ago
issue comment
--cors-allowed-origins issue

Could this potentially be a time synchronization issue? We've certainly seen issues with time synchronization before. Are you running the Parca Agent and Parca server on a different machine that you're using your browser on?

Created at 3 days ago
issue comment
Provide debuginfos for linkerd2-proxy

cc @stevej @olix0r @siggy

Created at 3 days ago
issue comment
Provide debuginfos for linkerd2-proxy

Also for reference, I have tried to build a container image with the debug=1 flag (which includes sufficient debuginfos to get the above things working), and here are some stats:

Builing from v2.188.0 release with zero modifications

[+] Building 332.9s (21/21) FINISHED
localhost/linkerd/proxy   HEAD.4e172043     a57cf56c46e4   4 minutes ago    92.7MB

Building from v2.188.0 release with debug=1

[+] Building 377.6s (21/21) FINISHED
localhost/linkerd/proxy   HEAD.4e172043     8566dec1a8f5   20 minutes ago   978MB

Change

This means there is approximately a ~13% increase in build time and about 10-11x artifact size.

Created at 3 days ago
issue comment
Provide debuginfos for linkerd2-proxy

I want to clarify why I selected "maybe" on whether I'd like to work on this. I'd be happy to be a supporting function, but since this is primarily build pipeline work and I very rarely interact with the rust ecosystem, I find it hard to believe that I'd be of much use. I'd be more than happy to do anything I can though!

Created at 3 days ago
opened issue
Provide debuginfos for linkerd2-proxy

What problem are you trying to solve?

I'm trying to profile the linkerd2-proxy, among everything else in my infrastructure, to understand where on a global level all CPU resources are being spent. The linkerd2-proxy, while efficient is meant to be deployed in practically every pod in a Kubernetes, cluster so while an individual container may not use a lot of resources, it would be great to understand what that looks like globally.

How should the problem be solved?

To do this, debuginfos need to be available somehow. This could be in a variety of different ways. My preference would be to produce debuginfos as part of the build process of the linkerd2-proxy, when creating the release tarball. Using the recently stabilized split-debuginfo configuration option, the production binary would be created and in a separate file the debuginfos would be written. These debuginfos can then be separately distributed, ideally through a debuginfod server, from where profilers and debuggers can automatically find the debuginfos using the binary's BuildID (which are identical in the "production" binary and the debuginfo file, that's how they're linked).

I think distributing the binary and the debuginfos this way it the best as there is only ever a single binary being produced per release, removing the possibility that the debug build it in any way different from a non-debug build.

Any alternatives you've considered?

There could be other ways, just like having a container images that contains a binary that has the debuginfos included. This would work as well, but it would require a user to explicitly change the infrastructure to make it debuggable, which is not ideal.

How would users interact with this feature?

Users would primarily interact with it indirectly, as it enables profilers such as Parca to discover the debuginfos automatically without the user having to do anything. Same goes for regular Linux perf as well as debuggers such as gdb or delve.

Would you like to work on this feature?

maybe

Created at 3 days ago
issue comment
feat(dotnet): add native AOT example

The docs page does say

Limited diagnostic support (debugging and profiling).

To me this looks like there are just no debuginfos in this binary to symbolize. What does nm <binary> output?

At first sight I would say either framepointers are there or dwarf unwinding is working, as the memory addresses look roughly like they could be correct and have a stack depth higher than one. (not an explanation for Maxim or the rest of the maintainers, but anyone who may be reading who's unfamiliar: if the unwinding is broken in some way we typically see a very very high memory address and maximum 1-2 stack depth)

Created at 3 days ago
issue comment
.NET support

Perhaps this is what you meant, but just in case if not, we did recently add an example of what to do to get .NET application working today: https://github.com/parca-dev/parca-demo/pull/18

We're still trying to figure out why ahead-of-time compiled .NET binaries aren't working (they should to our knowledge, it might be something small): https://github.com/parca-dev/parca-demo/pull/23

The ultimate goal is that what needs to be enabled in #18 is not necessary, which we haven't started the work for yet, so in the meantime that is the best way to use Parca with .NET.

Created at 3 days ago
issue comment
--cors-allowed-origins issue

I have a feeling this is not at all cors related, but much rather the same problem as reported in https://github.com/parca-dev/parca-agent/issues/1060

Created at 3 days ago