ob-col/exporter/prometheusexporter

Prometheus Exporter

Status
Stability	beta: metrics
Distributions	core, contrib, aws, grafana, observiq, redhat, sumo
Issues
Code Owners	@Aneurysm9

Exports data in the Prometheus format, which allows it to be scraped by a Prometheus server.

Getting Started

The following settings are required:

• endpoint (no default): the address on which metrics will be exposed, using path /metrics. For full list of HTTPServerSettings refer here.

The following settings can be optionally configured:

• const_labels (no default): key/values that are applied for every exported metric.
• namespace (no default): if set, exports metrics under the provided value.
• send_timestamps (default = false): if true, sends the timestamp of the underlying metric sample in the response.
• metric_expiration (default = 5m): defines how long metrics are exposed without updates
• resource_to_telemetry_conversion
  • enabled (default = false): If enabled is true, all the resource attributes will be converted to metric labels by default.
• enable_open_metrics: (default = false): If true, metrics will be exported using the OpenMetrics format. Exemplars are only exported in the OpenMetrics format, and only for histogram and monotonic sum (i.e. counter) metrics.
• add_metric_suffixes: (default = true): If false, addition of type and unit suffixes is disabled.

Example:

exporters:
  prometheus:
    endpoint: "1.2.3.4:1234"
    tls:
      ca_file: "/path/to/ca.pem"
      cert_file: "/path/to/cert.pem"
      key_file: "/path/to/key.pem"
    namespace: test-space
    const_labels:
      label1: value1
      "another label": spaced value
    send_timestamps: true
    metric_expiration: 180m
    enable_open_metrics: true
    add_metric_suffixes: false
    resource_to_telemetry_conversion:
      enabled: true

Given the example, metrics will be available at https://1.2.3.4:1234/metrics.

Metric names and labels normalization

OpenTelemetry metric names and attributes are normalized to be compliant with Prometheus naming rules. Details on this normalization process are described in the Prometheus translator module.

Setting resource attributes as metric labels

By default, resource attributes are added to a special metric called target_info. To select and group by metrics by resource attributes, you need to do join on target_info. For example, to select metrics with k8s_namespace_name attribute equal to my-namespace:

app_ads_ad_requests_total * on (job, instance) group_left target_info{k8s_namespace_name="my-namespace"}

Or to group by a particular attribute (for ex. k8s_namespace_name):

sum by (k8s_namespace_name) (app_ads_ad_requests_total * on (job, instance) group_left(k8s_namespace_name) target_info)

This is not a common pattern, and we recommend copying the most common resource attributes into metric labels. You can do this through the transform processor:

processor:
  transform:
    metric_statements:
      - context: datapoint
        statements:
        - set(attributes["namespace"], resource.attributes["k8s.namespace.name"])
        - set(attributes["container"], resource.attributes["k8s.container.name"])
        - set(attributes["pod"], resource.attributes["k8s.pod.name"])

After this, grouping or selecting becomes as simple as:

app_ads_ad_requests_total{namespace="my-namespace"}

sum by (namespace) (app_ads_ad_requests_total)