morrisonlevi
Repos
24
Followers
207
Following
1

Events

delete branch
morrisonlevi delete branch paullgdc/fix_nightly_use_breaking_release
Created at 1 day ago

Use normal toolchain with cbindgen (#57)

  • Use normal toolchain with cbindgen.
  • Turn off macro expansion for telemetry ffi cbindgen

These changes allow us to make a release without changing our build infrastructure.

Created at 1 day ago
pull request closed
Use normal toolchain with cbindgen

What does this PR do?

A brief description of the change being made with this pull request.

Motivation

What inspired you to submit this pull request?

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

Created at 1 day ago

Update lock file

Created at 1 day ago
pull request opened
Bump version to v0.9.0

What does this PR do?

Bumps version to v.0.9.0.

Created at 1 day ago
create branch
morrisonlevi create branch levi/v0.9.0
Created at 1 day ago

Rename and document stop_and_forget_profiling

Created at 2 days ago
issue comment
Add `initialize_before_fork` functions

I've updated this to your suggestions, @ivoanjo, and also to resolve a conflict with the main branch 👍🏻

Created at 3 days ago

Add AAS Resource id (#50)

  • Add AAS Resource id
  • Add more tags for AAS

fix nightly lints surprise build breakage (#51)

  • fix nightly lints surprise build breakage

  • speed up cargo bundle-licenses

  • improve caching on all builds

  • make license check even faster

  • continue running lints even if one fails

  • simplify the lints

  • use stable version instead of pinned nightly

  • add cache key description

Add initialize_before_fork

On Apple platforms in particular, some things need to be initialized before a fork, and ideally before threads are created.

Created at 3 days ago

Bump version to v0.10.2

Created at 3 days ago

Fix #1750 configurable max payload size (#1751)

update relenv after release (#1754)

Prevent deadlocks when forking

  • Prior to the fork, all threads will wait on the same barrier.
  • After the fork in the parent process, all threads will wait on the same barrier. They are then free to resume regular operations.
  • After the fork in the child process, the barrier will not be joined, the profiler object will be forgotten, and profiling will be disabled.

It should be easy to support profiling the child process from here.

Created at 3 days ago

Add initialize_before_fork

Created at 3 days ago
issue comment
Make load_root_certs() public

Indeed, there is a whole class of types out there that can trigger these issues. I don't know the Apple ecosystem well enough to say, "yes, definitely, we've fixed all the things" so I guess we'll be playing whack-a-mole. Exposing it through FFI is important too, good call.

Created at 3 days ago
delete branch
morrisonlevi delete branch gleocadie/retrive-aas-resource-id
Created at 3 days ago

Move VmInterrupts back to locking

Created at 4 days ago

Move VmInterrupts back to locking

Created at 4 days ago
pull request opened
[profiling] Prevent deadlocks when forking

Description

Deadlocks are prevented by the following technique:

  • Prior to the fork, all profiling threads will wait on the same barrier.
  • After the fork in the parent process, all threads will wait on the same barrier. They are then free to resume regular operations.
  • After the fork in the child process, the barrier will not be joined, the profiler object will be forgotten, and profiling will be disabled.

Additionally, some things which used locks before have been replaced by sending messages. This was done to simplify interactions (threads are always interacting through message passing now).

It should be easy to support profiling the child process from here.

Readiness checklist

  • [ ] Changelog has been added to the release document.
  • [ ] Tests added for this feature/bug.

Reviewer checklist

  • [ ] Appropriate labels assigned.
  • [ ] Milestone is set.
Created at 4 days ago
create branch
morrisonlevi create branch levi/forking
Created at 4 days ago
pull request opened
Make load_root_certs() public

What does this PR do?

Makes the load_root_certs() function public.

Motivation

From the function's documentation:

/// Load the root certificates into a new store. One of the reasons this
/// function is public is that on Apple platforms, it must be called before
/// a before a process forks, and ideally before any threads are created.
/// Users may run into errors like the following if they do not:
/// > objc[25938]: +[__NSCFConstantString initialize] may have been in
/// > progress in another thread when fork() was called. We cannot safely
/// > call it or ignore it in the fork() child process. Crashing instead. Set
/// > a breakpoint on objc_initializeAfterForkError to debug.

Additional Notes

If the codebase using libdatadog might fork, then this should be called, at least on Apple platforms.

How to test the change?

Call this function on an Apple platform, then fork, and note that you don't crash, whereas on older versions of libdatadog you might (depends potentially on if anything else has fully initialized the Apple frameworks).

Created at 4 days ago
create branch
morrisonlevi create branch levi/load_root_certs
Created at 4 days ago
delete branch
morrisonlevi delete branch ivoanjo/intake-v24-v2
Created at 4 days ago

Report profiling data in v2.4 intake format; compress files (#53)

  • Report profiling data in v2.4 intake format

This was tested with both the Ruby (agent and agentless modes) and the PHP profilers.

This also introduces a breaking API change: the ddog_ProfileExporter_build / ProfileExporter::build functions now take two additional arguments -- the profiling library name and version.

Other than that change, using the v2.4 intake format is transparent to the libdatadog users.

Thanks to @morrisonlevi for pairing with me on this.

Note that this does not (yet) include support for including attributes in the reporting data. I'll leave that for a separate PR.

  • Adjust profiler_tags encoding

  • Remove data[] from the name of the files

  • Add lz4 compression to files

  • Don't compress event.json

  • Rename profile_library_[name|version] to profiling_library_[name|version]

  • Test for DD-EVP-ORIGIN*

  • Move profiling_library_{name,version} to constructor

  • Document some intake details

Co-authored-by: Levi Morrison levi.morrison@datadoghq.com

Created at 4 days ago
pull request closed
Report profiling data in v2.4 intake format; compress files

What does this PR do?

This PR changes libdatadog to report profiling data using the v2.4 intake format. It also compresses the files ~including~ excluding the new event.json file using lz4.

This was tested with both the Ruby (agent and agentless modes) and the PHP profilers.

This also introduces two breaking changes:

  1. Breaking API change: the ddog_ProfileExporter_new / ProfileExporter::new functions now take two additional arguments -- the profiling library name and version.
  2. Breaking behavior change: Files are now automatically compressed with lz4 by libdatadog so libraries mustn't do their own compression (Ruby is such an example).

We expect that other than the API change, using the v2.4 intake format and the added compression will be transparent to the libdatadog users.

Thanks to @morrisonlevi for pairing with me on this.

Note that this does not (yet) include support for including attributes in the reporting data. I'll leave that for a separate PR.

Motivation

Get libdatadog users to use intake v2.4, and lay the ground work for including attributes in the reported data.

Also save some bytes over the wire for files. A few stats: - PHP CLI: running composer create-project symfony/symfony-demo resulted in a 40016 byte pprof which compressed to 18139 bytes in 141661 nanoseconds. That's a compression ratio of 2.2061 and a space savings of 54.67%. - Ruby github relenv test app: | | Uncompressed | Gzip (Current Ruby profiler) | LZ4 (libdatadog) | |----------------------|-------------:|-----------------------------:|-----------------:| | code-provenance.json | 49K | 6K | 9.5K | | rubyprofile.pprof | 402K | 48K | 76K |

Additional Notes

(Nothing)

How to test the change?

Validate that data can still be reported to the backend, in both agent as well as agentless modes.

Created at 4 days ago