To reliably diagnose both performance bottlenecks and security vulnerabilities in your applications, it’s crucial to gain fine-grained visibility into the execution of your code—a capability that normally requires code instrumentation. Instrumenting your code manually, however, is highly effort-intensive work, which is why at Datadog we’ve made such extensive efforts to automate code instrumentation wherever possible. Automatic instrumentation relies on techniques like runtime injection or bytecode manipulation, with the unique makeup of each programming language providing different opportunities and challenges for automation.
With respect to Go applications, the challenges with auto-instrumentation have traditionally been thought to outnumber the opportunities. The language compiles directly into native binaries and does not provide a built-in way to hook into the flow of execution transparently, unlike many other programming languages—including interpreted languages like Python and virtual machine–based languages like Java. To gain visibility into the performance of their Go applications, teams before now have needed to use alternative techniques, such as eBPF and binary patching, that present significant drawbacks and limitations. This situation has prevented many of these teams from achieving the level of observability they would otherwise prefer to have.
To solve this problem, we developed Orchestrion, a compile-time instrumentation tool that integrates directly with the Go toolchain. By open-sourcing Orchestrion, we aim to provide the entire Go community with an effective and safe way to gain deep visibility into their applications without needing to modify any code.
Observability as a security enabler
Observability plays a critical role in security, as the lack of visibility into an application’s execution can leave teams blind to critical vulnerabilities. Orchestrion’s compile-time approach to instrumentation provides access to the entire codebase and therefore allows visibility into the entire application, with no blind spots.
Additionally, since Orchestrion operates at the code level, it allows you to implement runtime application self-protection (RASP) features that inject instructions to change how the application runs at critical points. Through this technique, teams can allow applications to intercept function calls at runtime and prevent exploitation of known vulnerabilities—such as OWASP Top-10 threats like SQL injection and local file inclusion (LFI) attacks. For example, an application could identify an LFI attack by tracking how user-provided data is used within file system operations. If a request parameter is later found to be the path of a sensitive file (e.g., /etc/passwd), the application could block the operation before any sensitive data is exposed. Similarly, it could detect SQL injection attempts by observing how tainted user inputs are concatenated into SQL queries and flagging potential risks before they result in data breaches.
In addition, Orchestrion’s ability to analyze an application’s entire execution flow offers distinct security detection advantages. For example, it allows teams to detect anomalies that indicate attacks in real time—before they cause harm. And in the future, it could allow advanced IAST functionality to detect potential vulnerabilities before they are actively exploited.
Why open source?
We envisioned Orchestrion as an open source project from the start. Strong observability benefits the entire Go community, and it should be an accessible feature of the Go ecosystem.
Open-sourcing Orchestrion also provides transparency and auditability for this important feature. Anyone can review the code for security and compliance, which builds trust in the software.
Collaborating with Alibaba on a next-generation solution
We also know that our efforts to auto-instrument Go applications can only be strengthened by working with other contributors on an open source project. In fact, we are now collaborating with Alibaba and other stakeholders to create the next generation of an OpenTelemetry–standard tool inspired by Orchestrion.
Here’s how this collaboration came to be: As we were developing Orchestrion, we learned that Alibaba was also working on its own OpenTelemetry solution for Go and was proposing to donate it to OpenTelemetry. These efforts were distinct enough from ours that we saw an opportunity to combine the best of both and create something together with even broader capabilities. We then approached OTel, who welcomed our proposal for collaboration and facilitated successful discussions with Alibaba. This in turn led to the creation of the OpenTelemetry Go Compile-Time Instrumentation Special Interest Group (SIG), which is focused on developing our next-generation solution.
Since the creation of the SIG, we have begun collaborative development of the new joint solution on GitHub (whose progress you can review at https://github.com/open-telemetry/opentelemetry-go-compile-instrumentation). Through this work with Alibaba and other SIG contributors, we aim to build a robust, industry-wide standard for auto-instrumenting Go applications—one that benefits Go developers everywhere.
Help us continue to improve Go observability
With Orchestrion, we’re advancing the observability of Go applications. By integrating directly into the compiler, Orchestrion delivers a low-effort solution that benefits the entire ecosystem.
We invite the open source community to join us in refining and expanding this approach. Get involved in the OpenTelemetry Go Compile-Time Instrumentation Special Interest Group (SIG) and help shape the future of Go observability—making applications more transparent, performant, and secure for everyone.