When upgrading your Prometheus agent to v2, you'll need to account for its reliance on Prometheus v3. This means any breaking changes introduced in Prometheus v3 may also be present in your agent's v2. Specifically, Prometheus v3 implements strict validation for the Content-Type header received during scraping. Unlike Prometheus v2, which would default to the standard Prometheus text protocol if the header was missing or invalid, Prometheus v3 will fail the scrape under these conditions.
As a result, you must implement the following changes:
- Both default Kubernetes jobs require
fallback_scrape_protocol: "PrometheusText0.0.4"to be added to your specification. The default jobs forconfig.kubernetes.jobsshould be configured as follows:
- job_name_prefix: default fallback_scrape_protocol: "PrometheusText0.0.4" target_discovery: pod: true endpoints: true filter: annotations: prometheus.io/scrape: true- job_name_prefix: newrelic fallback_scrape_protocol: "PrometheusText0.0.4" integrations_filter: enabled: false target_discovery: pod: true endpoints: true filter: annotations: newrelic.io/scrape: true- If you have any custom
config.static_targets.jobsthat target a Prometheus endpoint with missing or invalid headers, you must fix these to provide a valid header. As a temporary solution, you can addfallback_scrape_protocol: "PrometheusText0.0.4"to your specification. - Similarly, any custom
config.extra_scrape_configsthat target a Prometheus endpoint with missing or invalid headers must be fixed to provide a valid header. A temporary solution is to addfallback_scrape_protocol: "PrometheusText0.0.4"to your specification.
Importante
The --reuse-values flag is not supported if you upgrade from v1 to v2.