Impact

Prometheus and its exporters can be secured by a web.yml file that specifies usernames and hashed passwords for basic authentication.

Passwords are hashed with bcrypt, which means that even if you have access to the hash, it is very hard to find the original password back.

However, a flaw in the way this mechanism was implemented in the exporter toolkit makes it possible with people who know the hashed password to authenticate against Prometheus.

A request can be forged by an attacker to poison the internal cache used to cache the computation of hashes and make subsequent requests successful. This cache is used in both happy and unhappy scenarios in order to limit side channel attacks that could tell an attacker if a user is present in the file or not.

Patches

The exporter-toolkit v0.7.3 and v0.8.2 have been released to address this issue.

Workarounds

There is no workaround but attacker must have access to the hashed password, stored in disk, to bypass the authentication.

Credit

We want to thank Lei Wan reporting this security issue.

Impact

Users who use Azure AD remote write with OAuth authentication are impacted.

The clientsecret field in the Azure AD remote write OAuth configuration (storage/remote/azuread) was typed as string instead of Secret. Prometheus redacts fields of type Secret when serving the configuration via the /-/config HTTP API endpoint. Because the field was a plain string, the Azure OAuth client secret was exposed in plaintext to any user or process with access to that endpoint.

Patches

The problem has been patched by changing ClientSecret in OAuthConfig to Secret. Users should upgrade to 3.11.3 or 3.5.3 LTS.

Workarounds

Users who can not upgrade can switch to Managed Identity or Workload Identity authentication for Azure AD remote write, which do not involve a client secret.

Impact

The remote read endpoint (/api/v1/read) does not validate the declared decoded length in a snappy-compressed request body before allocating memory. An unauthenticated attacker can send a small payload that causes a huge heap allocation per request. Under concurrent load this can exhaust available memory and crash the Prometheus process.

Patches Has the problem been patched? What versions should users upgrade to?

Fixed in 3.11.3 and 3.5.3 LTS. Users should upgrade to these versions or later.

Workarounds User who can not upgrade can place Prometheus behind a reverse proxy or firewall that requires authentication before requests reach /api/v1/read.

A denial of service attack was found in prometheus/clientgolang. This flaw allows an attacker to produce a denial of service attack on an HTTP server by exploiting the InstrumentHandlerCounter function in the version below 1.11.1, resulting in a loss of availability.

DISPUTED blackboxexporter v0.23.0 was discovered to contain an access control issue in its probe interface. This vulnerability allows attackers to detect intranet ports and services, as well as download resources. NOTE: this is disputed by third parties because authentication can be configured.

Impact

An attacker with the permission to perform POST requests on the /api/v1/alerts endpoint could be able to execute arbitrary JavaScript code on the users of Prometheus Alertmanager.

Patches

Users can upgrade to Alertmanager v0.2.51.

Workarounds

Users can setup a reverse proxy in front of the Alertmanager web server to forbid access to the /api/v1/alerts endpoint.

References

N/A

Prometheus is an open-source monitoring system and time series database. Prior to versions 3.5.3 and 3.11.3, the clientsecret field in the Azure AD remote write OAuth configuration (storage/remote/azuread) was typed as string instead of Secret. Prometheus redacts fields of type Secret when serving the configuration via the /-/config HTTP API endpoint. Because the field was a plain string, the Azure OAuth client secret was exposed in plaintext to any user or process with access to that endpoint. This issue has been patched in versions 3.5.3 and 3.11.3.

Impact

In 2.23.0, Prometheus changed its default UI to the New ui. To ensure a seamless transition, the URL's prefixed by /new redirect to /. Due to a bug in the code, it is possible for an attacker to craft an URL that can redirect to any other URL, in the /new endpoint.

If a user visits a prometheus server with a specially crafted address (e.g.: http://127.0.0.1:9090/new/new<url>), they can be redirected to an arbitrary URL.

e.g. if a user visits http://127.0.0.1:9090/new/newhttp://www.google.com/, they will be redirected to http://google.com.

Patches

The issue will be patched in 2.26.1 and 2.27.1 releases. In 2.28.0, the /new endpoint will be removed completely.

Workarounds

The workaround is to disable access to /new via a reverse proxy in front of Prometheus.

Note: Users who use a --web.external-url= flag with a path (e.g. --web.external-url=http://example.com/prometheus) are not affected.

For more information

If you have any questions or comments about this advisory, please use our community channels (https://prometheus.io/community). Our security policy is available at https://prometheus.io/docs/operating/security/

Withdrawn Advisory This advisory has been withdrawn because the vulnerability does not apply to the Prometheus golang package. This link is maintained to preserve external references.

Original Description A stored, DOM based, cross-site scripting (XSS) flaw was found in Prometheus before version 2.7.1. An attacker could exploit this by convincing an authenticated user to visit a crafted URL on a Prometheus server, allowing for the execution and persistent storage of arbitrary scripts.

DISPUTED Prometheus Blackbox Exporter through 0.17.0 allows /probe?target= SSRF. NOTE: follow-on discussion suggests that this might plausibly be interpreted as both intended functionality and also a vulnerability.

Impact

Stored cross-site scripting (XSS) via crafted metric names in the Prometheus web UI:

Old React UI + New Mantine UI: When a user hovers over a chart tooltip on the Graph page, metric names containing HTML/JavaScript are injected into innerHTML without escaping, causing arbitrary script execution in the user's browser. Old React UI only: When a user opens the Metric Explorer (globe icon next to the PromQL expression input field), and a metric name containing HTML/JavaScript is rendered in the fuzzy search results, it is injected into innerHTML without escaping, causing arbitrary script execution in the user's browser. Old React UI only: When a user views a heatmap chart and hovers over a cell, the le label values of the underlying histogram buckets are interpolated into innerHTML without escaping. While le is conventionally a numeric bucket boundary, Prometheus does not enforce this — arbitrary UTF-8 strings are accepted as label values, allowing script injection via a crafted scrape target or remote write.

With Prometheus v3.x defaulting to UTF-8 metric and label name validation, characters like <, >, and " are now valid in metric names and labels, making this exploitable.

An attacker who can inject metrics (via a compromised scrape target, remote write, or OTLP receiver endpoint) can execute JavaScript in the browser of any Prometheus user who views the metric in the Graph UI. From the XSS context, an attacker could for example:

- Read /api/v1/status/config to extract sensitive configuration (although credentials / secrets are redacted by the server) - Call /-/quit to shut down Prometheus (only if --web.enable-lifecycle is set) - Call /api/v1/admin/tsdb/deleteseries to delete data (only if --web.enable-admin-api is set) - Exfiltrate metric data to an external server

Both the new Mantine UI and the old React UI are affected. The vulnerable code paths are:

- web/ui/mantine-ui/src/pages/query/uPlotChartHelpers.ts — tooltip innerHTML with unescaped labels.name - web/ui/react-app/src/pages/graph/GraphHelpers.ts — tooltip content with unescaped labels.name - web/ui/react-app/src/pages/graph/MetricsExplorer.tsx — fuzzy search results rendered via dangerouslySetInnerHTML without sanitization - web/ui/react-app/src/vendor/flot/jquery.flot.heatmap.js — heatmap tooltip with unescaped label values

Patches

A patch has been published in Prometheus 3.5.2 LTS and Prometheus 3.11.2. The fix applies escapeHTML() to all user-controlled values (metric names and label values) before inserting them into innerHTML. This advisory will be updated with the patched version once released.

Workarounds

- If using the remote write receiver (--web.enable-remote-write-receiver), ensure it is not exposed to untrusted sources. - If using the OTLP receiver (--web.enable-otlp-receiver), ensure it is not exposed to untrusted sources. - Ensure scrape targets are trusted and not under attacker control. - Do not enable admin / mutating API endpoints (e.g. --web.enable-admin-api or web.enable-lifecycle) in cases where you cannot prevent untrusted data from being ingested. - Users should avoid clicking untrusted links, especially those containing functions such as labelreplace, as they may generate poisoned label names and values.

Acknowledgements

Thanks to @gladiator9797 (Duc Anh Nguyen from TinyxLab) for reporting this.

Prometheus is an open-source monitoring system and time series database. From 2.49.0 to before 3.5.3 and 3.11.3, in the Prometheus server's legacy web UI (enabled via the command-line flag --enable-feature=old-ui), the histogram heatmap chart view does not escape le label values when inserting them into the HTML for use as axis tick mark labels. An attacker who can inject crafted metrics can execute JavaScript in the browser of any Prometheus user who views the metric in the heatmap chart UI. This vulnerability is fixed in 3.5.3 and 3.11.3.

Dear all,

the Prometheus project[1] has received a public "vulnerability" report[2] against what the reporter called SSRF, but what is the core functionality of blackboxexporter[3]: The ability to trigger network probes over the network to monitor a target's availability. The reporter stated that CVE-2020-16248 has been assigned. From context, it seems to be a paid assessment of our software for an unnamed client which increases motivation to get "results", in particular CVEs for "zero days" - which are then promptly reported publicly with an embargoed CVE.

The reporter has not replied to our statement that this behaviour is core functionality. I could not find out which organization has reserved CVE-2020-16248 so I decided to send email to this list to inform the organization, enabling them to update their records.

Sorry for using this list for that purpose, I could not find a less wrong place to inform the (hopefully) interested parties.

Best, Richard

[1] https://prometheus.io/ [2] https://github.com/prometheus/blackboxexporter/issues/669 [3] https://github.com/prometheus/blackboxexporter