Configuring OpenTelemetry Agents to Enrich Data and Reduce Observability Costs

BindPlane OP is a powerful open-source tool that makes it easy to build and manage telemetry pipelines to ship data from IT environments of any kind and size to any analysis tool or storage destination. BindPlane OP installs and configures OpenTelemetry agents, which support a wide variety of sources and can be configured to ship data to multiple destinations while enriching or reducing data simultaneously. The vendor-agnostic toolset is excellent for reducing data costs and getting the most out of your data. OpenTelemetry agent configurations are formatted as YAML files that are always editable without the need to uninstall and reinstall the agent. With BindPlane OP, you can create and edit configurations with a codeless visual interface and step-by-step options for collecting and managing your data. This article covers how to create and edit a configuration in BindPlane OP, add sources and destinations, add processors to reduce or enrich data, and apply that configuration to one or many agents with a few clicks. 

What Are Configs in BindPlane OP?

Configurations in BindPlane OP are instructions for directing OpenTelemetry agents. The Configs tab is where you create, save, and apply agent configurations in BindPlane OP. Existing configurations appear in a sortable, searchable list, and new configurations can be created by clicking the “New Configuration” button at the top. Clicking on a configuration will open a page where you can see details for that configuration, including the sources, destinations, and a topographical map representing the real time data flow and processing of agents using that configuration. 

You can edit the sources and destinations by clicking them, and add new ones by clicking the respective buttons. New configurations created on the Configs page exist as templates that do not affect your agents unless explicitly applied, so don’t worry about affecting your environment while creating or editing new configurations, but if you edit an existing configuration that is already applied to agents in your environment, those changes will be pushed to the appropriate agents when you click “Save” after an edit is made.

At the bottom of any configuration’s page, you can see a searchable list of agents using that configuration, their operating system, and their connection status. You can also apply the configuration to new agents by clicking the “+” button at the top of this section. 

Creating an OpenTelemetry Agent Configuration in BindPlane OP

Creating a configuration is a three-step process.

• Give your configuration a name, select your intended operating system, and write a brief description.
  • It’s good practice to give enough detail in the description so that other users can quickly identify the intended use of the configuration. 
• Add your intended source(s).
  • Sources appear in a searchable list with “log,” “metrics,” and “traces” labels.
  • You can adjust your basic settings for the source, like the hostname and port. You can also adjust advanced settings, such as the collection interval. Advanced settings vary by source.
• Add your destination(s).
  • Destinations appear in a searchable list. Click a destination to select it.
  • Give your destination a name for use in BindPlane OP (the name does not have to match anything outside of BindPlane OP, but that can make organization easier)
  • Add the necessary routing and authentication information for the destination. The specific requirements vary by destination, but typically include a region and an access key or password. Tooltips in BindPlane OP can help you find your authentication information if you don’t have it.
  • You can add as many destinations as you wish by repeating the previous steps. OpenTelemetry agents can ship data to any number of destinations.
  • Once you add a destination to a configuration, it is saved as a component, so you can skip the routing and authentication when you use it in another configuration.

Adding Processors to Enrich Data and Reduce Data Costs

Processors are components in configurations that manipulate data at the source before it’s sent to your destinations. There are many ways data can be manipulated, but the two general categories of data manipulation are enrichment and reduction. Enrichment means you add additional information to data at the source that helps you use the data more effectively at your destination. Reduction means you filter or eliminate data to reduce data flow and save on data costs. Below are the steps for adding a processor, and list of processors and their uses.

1. On the Configs page, click on the configuration you want to add a processor to, then click on the source you want the processor to affect data from.
2. Click “Add Processor” at the bottom of the source details.
3. Select the processor you want from the list, or create a custom processor.
   1. Input the details for how you want the processor to manipulate your data, then click “Save.” The processor will be applied to any agents using the configuration automatically.