Guide to secret scanning

According to a 2025 report from IDC, organizations that use secret scanning use it to detect secrets only for about 50% of their applications. The other 50% of their codebases might still have exposed credentials. The theft of these credentials is a leading cause of data breaches, according to IBM’s Cost of a Data Breach report. On average, breaches caused by stolen credentials cost USD 4.67 million.

Exposed secrets can include API keys, access tokens, passwords, private keys, encryption keys and any other credential that human or nonhuman identities might use to authenticate to a system. They are most commonly exposed when developers hardcode them into configuration files or elsewhere for convenience during the software development process. The credentials can then become exposed in several ways. For example, production credentials in environment variable files can be accidentally committed (especially when those files are replicated across the CI/CD pipeline) or left in earlier versions of a Git history.

To detect exposed secrets, organizations use tools that scan source code for strings that are known or likely to be sensitive information. These tools look for patterns that are commonly used by certain service providers, strings that appear excessively random or commonly used strings such as sequential numbers or “password.” While most open-source tools use pattern recognition and entropy analysis, newer tools use machine learning algorithms to predict which strings likely include exposed credentials and permissions.

Secret scanning is part of the broader practice of vulnerability management, the continuous discovery, prioritization and resolution of security vulnerabilities in an organization’s IT infrastructure and software. The benefits of secret scanning include reduced security breaches, reduced response times and improved compliance.

Secret scanning uses mathematical techniques and information from public repositories to analyze text files in code, commit and pull request histories for secrets and credentials that shouldn’t be there.

Secret scanning tools generally follow a defined workflow. First, they scan the source code, logs and other artifacts for relevant strings. Then, they run that data through a series of detection methods, remove false positives, validate the results and remediate exposures.

Secret scanning tools and processes can run perpetually through the continuous integration/continuous delivery (CI/CD) pipeline as developers commit code, periodically on a set schedule or both.

Scanners observe and extract strings from:

• Code repositories and version control systems such as GitHub, including both current code and complete code history.
  
  
• CI/CD tools such as GitHub Actions or Jenkins.
  
  
• IT infrastructure, including containers and infrastructure as code (IaC) architecture.
  
  
• Non-code tools that are integrated into the DevOps pipeline, such as Jira, Slack or Confluence.
  
  
• Observability software and its associated storage.

Secret scanning tools use a combination of techniques, including regular expression (regex), entropy analysis, and keyword scanning, to perform a thorough search of locations that might hold exposed credentials.

A regular expression, or regex, is a sequence of characters that is commonly associated with a particular secret. For example, the AWS access key regex follows the pattern AKIA[0-9A-Z]{16}. Every key begins with AKIA, then uses 16 uppercase alphanumeric characters. A secrets scanner will specify commonly known and used combinations such as these and seek them out in the system through pattern matching.

Entropy analysis measures how random a string is. A string’s randomness can correlate with its likelihood of being a secret. Both excessively random strings—such as “16hFyy8Rt5Gv”—and extremely low-entropy strings—such as “abcde”—have higher chances of being secrets, so both will be flagged.

Dictionary or keyword scanning scans the system for strings that are known to be secrets—for example, anything known to be kept in the vault of a secrets management platform. These tools also look for obvious context clues such as “password,” “secret” or “token.”

Some secret scanning tools use a combination of these secret detection strategies. Emerging tools use machine learning and artificial intelligence trained on datasets of leaked secrets to identify credentials that might elude other detection strategies or that aren’t yet classified as regular expressions.

Validation helps reduce false positives by checking that the secrets detected by scanning tools are truly secrets and truly exposed on the network.

Validation methods include:

• API verification, where the scanning tool makes a simple API call by using the secret in question to find out whether it is truly functional.
  
  
• Checksum authentication, where the tool verifies that the string is valid according to the required internal logic of the credential. For example, in the case of AWS access keys, the tool would check that a potentially exposed key fits the 16-digit regex structure.
  
  
• Extracting metadata from the results of the scan that might contain information about how and where the credential can be used. For example, for an exposed access key, information from its commit history such as authors, timestamps and commit messages might reveal whether the credential is outdated or was added intentionally. 

Many secret scanners can automatically remediate exposed secrets by revoking the credential in question and starting the process of issuing a new one.

First, the credential is revoked so that unauthorized users who already have the credential cannot use it. Secret scanning tools also look at logs and Git history to help ensure that hardcoded secrets are scrubbed from older versions of the code that might still be referenced or reverted.

Then, the owner of the credential is prompted to generate a new one. Instead of being hardcoded into the codebase, this new credential is stored in a secrets management platform, such as HashiCorp Vault.

Secret scanning is distinct from the broader practice of vulnerability management in that it specifically looks for credentials that were accidentally exposed in the normal process of code deployment. Vulnerability management, in contrast, is the overall assessment of all potential vulnerabilities in software.

Exposed secrets are, in the most literal sense, a vulnerability, so secret scanning can be considered a kind of vulnerability management. However, it is only one vulnerability management practice alongside many others in the developer workflows of DevSecOps. Other practices include software composition analysis (SCA) and dynamic application security testing (DAST).

The benefits of secret scanning include preventing costly breaches, strengthening regulatory compliance and reducing the mean time to remediate (MTTR).

Challenges of secret scanning include managing false positives, maintaining performance speed and defining the scope of an organization’s security posture.