Underlying processes within Git-based Source Code Management systems (SCMs) cause code to remain accessible even after being deleted or overwritten, continuing to expose previously leaked secrets, new Aqua Security research shows.
Security best practices dictate that developers should never hard-code secrets, and Aqua’s new research shows that a secret – be it a password, token, or passkey – that was hardcoded once may be permanently exposed even after removal, as most secrets scanners are likely to miss it. Aqua is calling them ‘phantom’ secrets.
Conventional scanning methods, most of which only scan repositories accessible via the Git clone command, are likely to miss roughly 18% of the potentially exposed secrets, Aqua discovered after looking at more than 50,000 repositories belonging to the top 100 organizations on GitHub.
“During our research, we uncovered some significant secrets, including gaining access to the complete cloud environments of some of the biggest organizations in the world, infiltrating the internal fuzzing infrastructure of sensitive projects, accessing telemetry platforms, and even obtaining access to network devices, SNMP secrets, and camera footage of Fortune 500 companies,” the cybersecurity firm says.
One of Mozilla’s public GitHub repositories contained an API token for FuzzManager, the internal tool used for collecting, managing, and analyzing fuzzing data, providing access to Firefox fuzzing data.
Configured with high privileges, it allowed both read and write access, potentially allowing an attacker to access information on unpatched vulnerabilities affecting both Firefox and the Tor browser.
Aqua also found the API token of a Mozilla employee that provided access to a telemetry system that collects usage and performance statistics, along with other Firefox data used for performance improvements.
Meraki API tokens of Fortune 500 companies, granting access to the Meraki Dashboard API, which is used for network resource management, were also found exposed.
Additionally, Aqua discovered an Azure service principal token belonging to a large healthcare company, which provided access to many of the organization’s Azure resources, as it had high privileges on Azure AD.
In addition to providing full control over the organization’s Kubernetes clusters, the token could be used to obtain credentials to the internal Azure Container Registry, “which could have led an attacker to perform a supply chain attack, by pushing a malicious container image, impacting the organization, and customers,” Aqua explains.
All these secrets were reported to the impacted organizations and were rotated to prevent compromise.
Despite increased awareness on the risks associated with leaked secrets and the existence of tools and bug bounty programs that help hunt for these exposures, the issue has not been eradicated, Aqua explains, pointing to how secrets scanning tools are engineered.
“The problem of exposed secrets in source code remains a common and significant challenge in the software development life cycle (SDLC). Not all secrets scanning tools are alike, the diverse scanning tools vary in the volume of results, and the level of their accuracy,” Aqua notes.
“When building a secrets scanning tool you need to take some basic engineering decisions, for instance will the detection mechanism be based on pattern recognition, or the detection is entropy-based,” it added.
API key patterns, secrets hardcoded in unpredictable locations, credentials with different entropy levels, the limitations and behavior of SCM platforms affecting the scanner’s accuracy, and the different methods of hardcoding secrets in the code contribute to the complexity of the issue.
In a technical writeup, Aqua details some of the blind spots that secrets scanning tools may miss, how commits remain accessible via cache views even after deletion, and some of the methods that attackers can use to extract secrets from commits that have been removed from repositories.
In addition to implementing security best practices throughout code’s entire lifecycle, developers are advised to consider secrets that have been accidentally pushed into production as compromised and to immediately rotate them, as well as to make all the necessary efforts to remove them from public repositories.
“The findings once again reinforce the best practice that secrets should never be put into code, not even for testing purposes, and security teams must be able to monitor this. The software supply chain is optimized for speed and convenience, but this cannot come at the expense of secure engineering practices,” Aqua Security CTO and co-founder Amir Jerbi said.
Related: Secrets Exposed in Hugging Face Hack
Related: Researchers Discover Dangerous Exposure of Sensitive Kubernetes Secrets
