OpenTelemetry (OTel) is an open source, vendor-neutral observability framework that supplies APIs, SDKs, and tools to instrument, generate, collect, and export telemetry data (metrics, logs, and traces). It has a vibrant ecosystem of ecosystem of components, integrations and vendors.
OpenTelemetry was formed as a CNCF project in 2019 via the combination of OpenCensus and OpenTracing, and Datadog was there from the start. As a member of the OpenTracing Specification Council we provided various OpenTracing-compliant tracers, exporters for OpenCensus and an OpenMetrics integration.
Datadog's contributions
As we do now, Datadog firmly believed in the future of open source telemetry. We joined the community early in the process of collaboratively defining OTel standards, made our auto-instrumentation tracers available to be leveraged as the foundation of OpenTelemetry tracers and provided expertise in their implementation.
Datadog engineers continue to contribute code, maintain features, and participate in contributor roles (triager, approver, maintainer) for OTel repositories including:
- opentelemetry-collector
- opentelemetry-collector-contrib
- opentelemetry-demo
- opentelemetry-dotnet-instrumentation
- opentelemetry-ebpf-profiler
- opentelemetry-go-instrumentation
- opentelemetry-php-contrib
- opentelemetry-profiling
We also believe that contributing to open source doesn't end with just writing code. That's why at Datadog, we encourage our engineers to participate in the various Special Interest Groups (SIGs) crafting the future of OpenTelemetry. This includes co-leading:
What do you think is the most exciting thing about the OpenTelemetry community and ecosystem?
OpenTelemetry is in a unique position to get a wide diversity of viewpoints and contributions: from well-established vendors such as Datadog to more recent participants whose products are entirely based in OTel, all the way through companies, individuals and open source projects from all over the world using OpenTelemetry to instrument their applications and infrastructure.
It is really exciting to get to work with such a diverse set of people; I feel like I am better equipped to understand all of the important problems because of that, and I get many different eyes on everything I develop or propose. This makes the contributing experience a pleasure and a great learning experience.
How did you start contributing to OpenTelemetry?
I started contributing to OTel in late 2020. My first commit on an OpenTelemetry component was released on the OpenTelemetry Collector v0.12.0 in October that year. I was involved in the development of the Collector's Datadog exporter, the plugin that allows sending OTLP data to Datadog, so I naturally decided to pick up some work on the Collector and try to become involved in other components.
I slowly picked up more responsibilities, started attending the SIG meetings and before I knew it I was given the opportunity to become an approver. My first release as release manager was v0.47.0, and just a few versions later I became a maintainer and an approver for the core Collector libraries.
What are you currently working on?
I recently kickstarted the System Semantic Conventions Working Group where I am a co-lead together with an engineer from Red Hat. We want to stabilize and refine system-related semantic conventions to have a standard way to talk about topics such as CPUs, memory, and low-level networking in OTLP. I had contributed to the semantic conventions specification in the past but it's mostly new territory for me!
I am also trying to push forward on getting the core libraries on the OpenTelemetry Collector to 1.0 and make the project maintenance sustainable in the long term. We have a big team now (more than 25 people have some official role in the Collector repositories!), but some parts of the day-to-day maintenance such as releases, documentation and consistency across components are still a bit brittle and I want to improve on that.
Product support
We also see our excitement for a universal, open source telemetry standard reflected in our customers and the community, as use of OTel has increased rapidly. Our goal is for Datadog to be the best observability platform for OpenTelemetry.
To support Datadog customers in their adoption of OpenTelemetry we have made several enhancements to our platform including:
- Datadog Exporter for the OpenTelemetry Collector, which has stable support for OTLP traces and metrics and logs.
- OTLP ingest in the Datadog Agent, enabling Datadog customers to use OTel SDKs while benefiting from the Datadog Agent’s ecosystem of integrations.
- Datadog Connector for the OpenTelemetry Collector, providing accurate APM statistics, in the form of metrics for OTel users.
- W3C trace context support to all Datadog SDKs, allowing Datadog and OpenTelemetry instrumentation to collaborate in a mixed environment.
- Added support for the OpenTelemetry Collector Host Metrics receiver, with plans to continue expanding our infrastructure monitoring support for OTel users.
- Enabled seamless integration between Datadog RUM and OTel-instrumented backends.
- Ingestion and visualization of runtime metrics from OTel-instrumented applications in Java, .NET, and Go.
- Support for the OpenTelemetry API in Datadog SDKs, allowing users to maintain vendor-neutral instrumentation of all your services, while still taking advantage of Datadog’s native implementation, features, and products.
- Support for 128-bit trace IDs, letting OTel and DD spans being correlated seamlessly.
Our future with OTel
Our journey with OpenTelemetry doesn’t stop here and we have several projects underway. Here are just some of the new features we are working on:
- Agent with embedded OTel Collector.
- Support and/or convergence with OTel configuration and semantics, providing consistent in-app user experience.
- Expand support for OTel APIs in Datadog’s SDKs, with Span Events and Baggage support in development.
- Build support for OTel eBPF profiler in Datadog backend.