• /
  • EnglishEspañolFrançais日本語한국어Português
  • EntrarComeçar agora

Transform processor

The transform processor modifies, enriches, or parses telemetry data using OTTL (OpenTelemetry Transformation Language). Use it to add context, normalize schemas, parse unstructured data, or obfuscate sensitive information before data leaves your network.

When to use transform processor

Use the transform processor when you need to:

  • Enrich telemetry with organizational metadata: Add environment, region, team, or cost center tags
  • Parse unstructured log messages: Extract structured attributes using regex, Grok patterns, or JSON parsing
  • Normalize attribute names and value schemas: Standardize different naming conventions across services or agents (level → severity.text, env → environment)
  • Hash or redact sensitive data: Remove PII, credentials, or other sensitive information before it leaves your network
  • Extract values from strings: Pull HTTP status codes, durations, or other data from log messages
  • Aggregate or scale metrics: Modify metric values or combine multiple metrics

OTTL contexts

OTTL operates in different contexts depending on the telemetry type:

  • Logs: log context - access log body, attributes, severity
  • Traces: trace context - access span attributes, duration, status
  • Metrics: metric and datapoint contexts - access metric name, value, attributes

Configuration

Add a transform processor to your pipeline:

transform/Logs:
        description: Transform and process logs
        output:
          - nrexporter/newrelic
        config:
          rules:
            - name: add new field to attribute
              description: for otlp-test-service application add newrelic source type field
              conditions:
                - resource.attributes["service.name"] == "otlp-java-test-service"
              statements:
                - set(resource.attributes["source.type"],"otlp")
      transform/Metrics:
        description: Transform and process metrics
        output:
          - nrexporter/newrelic
        config:
          rules:
            - name: adding a new attributes
              description: adding a new field into a attributes
              conditions:
                - resource.attributes["service.name"] == "payments-api"
              statements:
                - set(resource.attributes["application.name"], "compute-application")
      transform/Traces:
        description: Transform and process traces
        output:
          - nrexporter/newrelic
        config:
          rules:
            - name: remove the attribute
              description: remove the attribute when service name is payment-service
              conditions:
                - resource.attributes["service.name"] == "payment-service"
              statements:
                - delete_key(resource.attributes, "service.version")
      nrexporter/newrelic:
        description: Export to New Relic

Config fields:

  • rules: An array of transformation rules.

    • name: A unique identifier for the rule.

    • description: A human-readable explanation of what the transformation does.

    • conditions: A list of OTTL boolean expressions.

    • statements: A list of OTTL actions (such as set or delete_key) to perform on the telemetry data.

Key OTTL functions

set()

Sets an attribute value.

- set(attributes["environment"], "production")
- set(attributes["team"], "platform")
- set(severity.text, "ERROR") where severity.number >= 17

delete_key()

Removes an attribute.

- delete_key(attributes, "internal_debug_info")
- delete_key(attributes, "temp_field")

replace_pattern()

Replaces text matching a regex pattern.

# Redact email addresses
- replace_pattern(attributes["user_email"], "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}", "[REDACTED_EMAIL]")


  # Mask passwords
- replace_pattern(attributes["password"], ".+", "password=***REDACTED***")


  # Obfuscate all non-whitespace (extreme)
- replace_pattern(body, "[^\\s]*(\\s?)", "****")

Hash()

Hashes a value for pseudonymization.

- set(attributes["user_id_hash"], Hash(attributes["user_id"]))
- delete_key(attributes, "user_id")

ParseJSON()

Extracts attributes from JSON strings.

# Parse JSON body into attributes
- merge_maps(attributes, ParseJSON(body), "upsert") where IsString(body)

ExtractGrokPatterns()

Parses structured data using Grok patterns.

# Parse JSON log format
- ExtractGrokPatterns(body, "\\{\"timestamp\":\\s*\"%{TIMESTAMP_ISO8601:extracted_timestamp}\",\\s*\"level\":\\s*\"%{WORD:extracted_level}\",\\s*\"message\":\\s*\"Elapsed time:\\s*%{NUMBER:elapsed_time}ms\"\\}")


  # Parse custom format with custom pattern
- ExtractGrokPatterns(attributes["custom_field"], "%{USERNAME:user.name}:%{PASSWORD:user.password}", true, ["PASSWORD=%{GREEDYDATA}"])

flatten()

Flattens nested map attributes.

# Flatten nested map to top-level attributes
- flatten(attributes["map.attribute"])

limit()

Limits number of attributes, keeping specified priority keys.

# Keep only 3 attributes, prioritizing "array.attribute"
- limit(attributes, 3, ["array.attribute"])

Complete examples

Example 1: Add environment metadata

transform/Logs:
  description: "Enrich logs with environment context"
  config:
    rules:
      - name: enrich-with-environment-metadata
        description: Add environment, region, team, and cost center metadata to all logs
        statements:
          - set(attributes["environment"], "production")
          - set(attributes["region"], "us-east-1")
          - set(attributes["team"], "platform-engineering")
          - set(attributes["cost_center"], "eng-infra")

Example 2: Normalize severity levels

Different services use different severity conventions. Standardize them:

transform/Logs:
  description: "Normalize severity naming"
  config:
    rules:
      - name: convert-level-to-severity
        description: Convert custom level attribute to severity_text
        conditions:
          - attributes["level"] != nil
        statements:
          - set(severity_text, attributes["level"])
      
      - name: delete-level-attribute
        description: Remove the redundant level attribute after conversion
        statements:
          - delete_key(attributes, "level")
      
      - name: normalize-error-case
        description: Normalize error severity to uppercase ERROR
        conditions:
          - severity_text == "error"
        statements:
          - set(severity_text, "ERROR")
      
      - name: normalize-warning-case
        description: Normalize warning severity to uppercase WARN
        conditions:
          - severity_text == "warning"
        statements:
          - set(severity_text, "WARN")
      
      - name: normalize-info-case
        description: Normalize info severity to uppercase INFO
        conditions:
          - severity_text == "info"
        statements:
          - set(severity_text, "INFO")

Example 3: Parse JSON log bodies

Extract structured attributes from JSON-formatted log messages:

transform/Logs:
  description: "Parse JSON logs into attributes"
  config:
    rules:
      - name: parse-json-body-to-attributes
        description: Parse JSON log body and merge into attributes
        conditions:
          - IsString(body)
        statements:
          - merge_maps(attributes, ParseJSON(body), "upsert")

Before: body = '{"timestamp": "2025-03-01T12:12:14Z", "level":"INFO", "message":"Elapsed time: 10ms"}'

After: Attributes extracted: timestamp, level, message

Example 4: Extract HTTP status codes

Pull status codes from log messages:

transform/Logs:
  description: "Extract HTTP status from message"
  config:
    rules:
      - name: extract-http-status-code
        description: Extract HTTP status code from log body using regex pattern
        statements:
          - ExtractPatterns(body, "status=(\\d+)")
          - set(attributes["http.status_code"], body)

Example 5: Redact PII

Remove sensitive information before data leaves your network:

transform/Logs:
  description: "Redact PII for compliance"
  config:
    rules:
      - name: redact-email-addresses
        description: Redact email addresses from user_email attribute
        conditions:
          - attributes["user_email"] != nil
        statements:
          - replace_pattern(attributes["user_email"], "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}", "[REDACTED_EMAIL]")
      
      - name: mask-passwords
        description: Mask password attribute values
        conditions:
          - attributes["password"] != nil
        statements:
          - replace_pattern(attributes["password"], ".+", "***REDACTED***")
      
      - name: hash-user-ids
        description: Hash user IDs and remove original value
        conditions:
          - attributes["user_id"] != nil
        statements:
          - set(attributes["user_id_hash"], SHA256(attributes["user_id"]))
          - delete_key(attributes, "user_id")
      
      - name: mask-credit-cards-in-body
        description: Mask credit card numbers in log body
        statements:
          - replace_pattern(body, "\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}", "****-****-****-****")

Example 6: Parse NGINX access logs

Extract structured fields from NGINX combined log format:

transform/Logs:
  description: "Parse and enrich NGINX access logs"
  config:
    rules:
      - name: extract-nginx-fields
        description: Parse NGINX access log format into structured attributes
        statements:
          - ExtractGrokPatterns(body, "%{IPORHOST:client.ip} - %{USER:client.user} \\[%{HTTPDATE:timestamp}\\] \"%{WORD:http.method} %{URIPATHPARAM:http.path} HTTP/%{NUMBER:http.version}\" %{NUMBER:http.status_code} %{NUMBER:http.response_size}")
      
      - name: set-severity-for-server-errors
        description: Set severity to ERROR for 5xx server errors
        conditions:
          - attributes["http.status_code"] >= "500"
        statements:
          - set(severity_text, "ERROR")
      
      - name: set-severity-for-client-errors
        description: Set severity to WARN for 4xx client errors
        conditions:
          - attributes["http.status_code"] >= "400"
          - attributes["http.status_code"] < "500"
        statements:
          - set(severity_text, "WARN")
      
      - name: set-severity-for-success
        description: Set severity to INFO for successful requests
        conditions:
          - attributes["http.status_code"] >= "200"
          - attributes["http.status_code"] < "400"
        statements:
          - set(severity_text, "INFO")

Example 7: Flatten nested attributes

Convert nested structures to flat attributes:

transform/Logs:
  description: "Flatten nested map attributes"
  config:
    rules:
      - name: flatten-kubernetes-attributes
        description: Flatten nested kubernetes attributes into dot notation
        conditions:
          - attributes["kubernetes"] != nil
        statements:
          - flatten(attributes["kubernetes"])
      
      - name: flatten-cloud-provider-attributes
        description: Flatten nested cloud provider attributes into dot notation
        conditions:
          - attributes["cloud.provider"] != nil
        statements:
          - flatten(attributes["cloud.provider"])

Before: attributes["kubernetes"] = {"pod": {"name": "my-app-123", "uid": "abc-xyz"},"namespace": {"name": "production"}}

After: Attributes flattened: kubernetes.pod.name, kubernetes.pod.uid, kubernetes.namespace.name

Example 8: Conditional transformations

Apply transformations only when conditions are met:

transform/Logs:
  description: "Conditional enrichment"
  config:
    rules:
      - name: tag-critical-services
        description: Add business criticality tag for checkout and payment services
        conditions:
          - resource.attributes["service.name"] == "checkout" or resource.attributes["service.name"] == "payment"
        statements:
          - set(attributes["business_criticality"], "HIGH")
      
      - name: normalize-production-environment
        description: Normalize production environment names to standard format
        conditions:
          - attributes["env"] == "prod" or attributes["environment"] == "prd"
        statements:
          - set(attributes["deployment.environment"], "production")
      
      - name: normalize-staging-environment
        description: Normalize staging environment names to standard format
        conditions:
          - attributes["env"] == "stg" or attributes["environment"] == "stage"
        statements:
          - set(attributes["deployment.environment"], "staging")
      
      - name: cleanup-legacy-env-fields
        description: Remove old environment attribute fields after normalization
        statements:
          - delete_key(attributes, "env")
          - delete_key(attributes, "environment")

Example 9: Data type conversion

Convert attributes to different types:

transform/Logs:
  description: "Convert data types"
  config:
    rules:
      - name: convert-error-flag-to-boolean
        description: Convert string error_flag to boolean is_error attribute
        conditions:
          - attributes["error_flag"] != nil
        statements:
          - set(attributes["is_error"], Bool(attributes["error_flag"]))
      
      - name: set-success-boolean
        description: Set success attribute to boolean true
        statements:
          - set(attributes["success"], Bool("true"))
      
      - name: convert-retry-count-to-int
        description: Convert retry_count_string to integer retry_count
        conditions:
          - attributes["retry_count_string"] != nil
        statements:
          - set(attributes["retry_count"], Int(attributes["retry_count_string"]))

Example 10: Limit cardinality

Reduce attribute cardinality to manage costs:

transform/Logs:
  description: "Limit high-cardinality attributes"
  config:
    rules:
      - name: limit-attribute-cardinality
        description: Keep only the 5 most important attributes
        statements:
          - limit(attributes, 5, ["service.name", "environment", "severity_text"])
      
      - name: generalize-user-api-paths
        description: Replace user ID in path with wildcard to reduce cardinality
        conditions:
          - IsMatch(attributes["http.path"], "/api/users/\\d+")
        statements:
          - set(attributes["http.path"], "/api/users/*")

OTTL function reference

For the complete list of OTTL functions, operators, and syntax:

Next steps

Copyright © 2026 New Relic Inc.
CarreirasTermos de serviçoPolítica do DMCAAviso de privacidadeWebsite TermsAviso de patenteUK Slavery Act

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.