OpenTelemetry

Logs that are not samples (eg. access logs) are exceptionally good sources of usage information. Combined with OpenTelemetry open standard for log forwarding, you can easily extract usage information from your logs and forward them to OpenMeter.

Benthos based collector

This is a Benthos-based collector.

#Prerequisites

You will need an OpenTelemetry-compatible log-forwarding solution. The easiest way to get started is to use the OpenTelemetry Collector.

#Configuration

First, create a new YAML file for the collector configuration. You will have to use the otel_log Benthos input:

input:
  otel_log:
    # Point your log forwarder to this address using the OTLP gRPC protocol.
    address: 127.0.0.1:4317

OpenMeter Benthos Collector

This is a custom input plugin that is not part of the official Benthos distribution. You can find the source code of the plugin here.

Next, you need to configure the mapping from your log schema to CloudEvents using bloblang:

pipeline:
  processors:
    - mapping: |
        root = {
          "id": uuid_v4(),
          "specversion": "1.0",
          "type": "api-calls",
          "source": "otlp-log",
          "time": this.record.attributes.time,
          "subject": this.record.attributes.subject,
          "data": {
            "method": this.record.attributes.method,
            "path": this.record.attributes.path,
            "region": this.record.attributes.region,
            "zone": this.record.attributes.zone,
            "duration_ms": this.record.attributes.duration,
          },
        }

About log attributes

this.record.attributes contains the log attributes extracted by the otel_log input plugin.

Finally, you need to configure the OpenMeter output:

output:
  openmeter:
    url: https://openmeter.cloud # optional
    token: '<YOUR OPENMETER CLOUD TOKEN>'

Read more about configuring Benthos in the Benthos-based collector guide.

Example on GitHub

Check out a fully working example on our GitHub.

#Installation

Check out the Benthos-based collector guide for installation instructions.