Set up and deploy Pipeline Control gateway

Set up and deploy the Pipeline Control gateway to manage the flow of data from your monitored entities to New Relic.

Considerations for using gateway

As you integrate and use the gateway component, here are some important considerations for you to keep in mind:

Attribute and data management

Attribute support: The gateway uses OpenTelemetry semantic conventions for attributes. Refer to the gateway documentation for details on supported attributes.
Data/attribute handling: The gateway forwards your requests for dropping data or attributes for metric time slice data, sql_trace_data, and trace_sample_data directly to New Relic without modification.

Parsing and data integration

Log parsing: The gateway does not include Grok parsing for logs. Grok parsing is a method used to extract structured data from unstructured log messages using predefined patterns and regular expressions. You may need alternative methods to structure log data for querying and visualization.
Prometheus integration: The gateway does not support Prometheus remote write for metrics, which may affect your integration with Prometheus-based systems.

Licensing and compliance

License key management: The gateway does not accommodate multiple license keys from native OpenTelemetry Protocol (OTLP) sources, such as agent collectors or APM SDKs. Consider this when managing your licenses.
Compliance standards: The gateway isn't FedRAMP-compliant, which may affect your use if you need to meet federal security standards.

Set up Pipeline Control gateway

Screenshot of the button to start Pipeline Control gateway installation

To set up Pipeline Control gateway:

In your New Relic account, in Integrations & Agents, click Set up Pipeline Control, or search for gateway and select Pipeline Control gateway.
Ensure the account selected is the one you want to use for Pipeline Control. If not, click Select an account, select the account of your choice, and then click Continue.
To enter your credentials, provide an existing license key or create a new key. If you create a new key, copy the key. Then, click Continue.
Select or create a gateway fleet:

Select an existing fleet if you want your new gateway to share an existing configuration (sampling, filtering, and transformation rules) with other gateways in your organization.
Enter a unique gateway fleet name if you need a fresh, isolated configuration for a specific environment or business unit.

Select an existing Kubernetes cluster using the drop-down menu, or create a new one.
If you create a new cluster, enter the cluster name and namespace. The default namespace is newrelic.
Download the configuration file, values-newrelic-gateway.yaml to a directory.
Run the installation command shown on screen:
bash
```
$helm repo add newrelic https://helm-charts.newrelic.com && helm repo update && \
>helm upgrade --install agent-control-bootstrap -n newrelic-agent-control \
>newrelic/agent-control-bootstrap --create-namespace --values </path/to/values-newrelic-gateway.yaml>
```
Dica
Replace </path/to/values-newrelic-gateway.yaml> with the path where the configuration file is located.
Click Continue.
Click Test connection and wait for a successful connection.

To view details of your gateway setup, go to New Relic Control and click Gateway.

Why use multiple fleets?

Imagine a corporation (The Corp) has two divisions: OrgA (needs 100% of data for safety) and OrgB (needs 1% sampling to control high-volume costs).

With only one fleet, The Corp is stuck. If OrgB applies 1% sampling to save money, OrgA loses 99% of its flight safety data. One team's cost-control breaks the other team's mission-critical monitoring.

By using multiple fleets, The Corp can separate their "brains":

Mission-critical fleet: OrgA joins this fleet to get 100% data retention.
High-volume fleet: OrgB joins this fleet for 1% sampling and GDPR-compliant redaction for their European sensors.

As a result, separate teams can now manage unique sampling, security, and compliance rules without affecting the rest of the organization.

Installing a gateway is now like hiring a worker. You just have to tell them which team (fleet) to join.

Add more clusters

To add a new cluster to your gateway setup,

Open New Relic Control, and then select Gateway.
In the clusters table, click Add a cluster.
Follow the instructions in the gateway setup.
Come back to the gateway summary page. You should see your newly added cluster.

After you have created your gateway clusters, configure pipeline processing using YAML configuration. Refer to Gateway configuration.