On October 5, 2021, an open-source data visualization platform known as “Grafana” released important security updates, including a patch that addresses a critical security flaw, enlisted as CVE-2021-39226 on NVD.
This vulnerability first appeared in version 2.0.1, allowing users to view and delete snapshots regardless of their credentials, leading to data exposure and loss. More precisely, both unauthenticated and authenticated users are allowed to access snapshot data that should be beyond their intentionally assigned permissions.
Snapshot Authentication Bypass
As a critical vulnerability with a CVSS score of 9.8, the potential impact includes unauthorized disclosure of sensitive information, data modification, and denial of service (DoS) attacks.
It arises due to inadequate access control mechanisms within the snapshot functionalities, therefore being labeled as an “Improper Authentication” weakness, enumerated as CWE-287.
The vulnerability allows users who have not logged in or provided any credentials to view snapshots in Grafana, which are sets of various visualizations on the dashboard captured at a particular moment in time.
This unauthorized access is achieved through literal paths, which are specific URLs that unauthenticated users can use to access snapshots. These literal paths are "/dashboard/snapshot/:key" and "/api/snapshots/:key".
In Grafana, there is a configuration setting called "public_mode" that determines whether snapshots can be accessed by unauthenticated users. If this setting is set to true, they can not only view snapshots but can also delete them.
Even if the "public_mode" setting is set to false, authenticated users (users who have logged in and provided valid credentials) can still exploit the vulnerability to delete snapshots, which is not the intended behavior.
Grafana’s Redemption
Understanding the timeline of the vulnerability's discovery and response provides valuable insights into the promptness of Grafana's actions and the severity of the flaw. The vulnerability was responsibly disclosed to Grafana Labs by Tran Viet Tuan on September 15, 2021. Upon receiving the report, Grafana Labs quickly engaged in verification and initial reproduction of the vulnerability.
The severity level was escalated on the same day, and workarounds were deployed on Grafana Cloud to protect users. Subsequent analysis revealed the full extent of the vulnerability, prompting the declaration of the vulnerability as critical on September 16, 2021. By September 17, 2021, a full audit of Grafana Cloud instances was completed, which found no evidence of exploitation.
Finally, on October 5, 2021, Grafana released the public security fix making it accessible to all users. The responsible disclosure and timely response demonstrated the importance of collaboration between security researchers and vendors to ensure the security of open-source projects.
Patched Versions and Mitigations
As always, it is strongly advised to upgrade to these patched versions to eliminate the risk of exploitation, enforcing the appropriate access control measures. Grafana’s quick reaction to this critical security flaw saved the day, addressing it in versions 8.1.6 and 7.5.11.
Grafana Cloud instances have been protected since September 16, 2021, and Grafana Enterprise customers received updated binaries under embargo on September 28, 2021. This means that if you are using Grafana Cloud or Enterprise, you can rest assured that the issue’s been resolved.
Identifying Exploitation And Attack Auditing
To identify potential exploitation, administrators can perform an audit through reverse proxy/load balancer logs. By searching for specific paths with response status code 200 (OK), like "/dashboard/snapshot/:key", "/api/snapshots/:key", and "/api/snapshots-delete/:deleteKey" they can narrow down possible attacks.
Grafana Enterprise users can also leverage the "Log web requests" feature with “router_logging = true” and look for specific request URIs and actions related to snapshot deletion.
The Workaround
If for any reason immediate upgrading is not an option, administrators can implement a workaround by using a reverse proxy to block access to vulnerable paths.
Disabling access to "/api/snapshots/:key", "/api/snapshots-delete/:deleteKey", "/dashboard/snapshot/:key", and "/api/snapshots/:key" will help mitigate the risk and will buy you some time until a proper update is installed.
However, it is crucial to note that a workaround is called a “workaround” for a reason - because it’s a temporary measure and should not be considered a permanent solution.
Conclusion
Although in this particular case, we haven’t witnessed any major breaches, working in IT requires always keeping your guard up. Having a solid background and a good understanding of these topics doesn’t guarantee an adequate reaction.
Taking care of a complex infrastructure requires that one stays up-to-date with the latest news from the world of cyber-security, otherwise, how would one know when to act?
While keeping your already proven and efficient methods for staying protected, the additional step you can do to prevent any surprises is subscribing to our blog. We are doing our best to stay relevant and cover every major incident.
You might ask yourself how can we keep track of what’s relevant and what’s not? It’s all thanks to the DataGrid Surface’s team of hard-working security engineers who keep making scanners that detect vulnerable systems on the web.
If you want to see what’s the real state of security on a large scale, find out what the real numbers are, or see if your systems are vulnerable, we recommend checking our feed. It won’t hurt to ask for a second opinion 🙂
Subscribe to be updated on the new content!