Understanding reliability score
After uploading a package SBOM file in CycloneDX format, Concert assesses the package components and fetches a reliability score based on the Open Source Security Foundation (OpenSSF) scorecard.
The OpenSSF is a collaborative forum that focuses on technical and educational initiatives to improve the security of the open source software ecosystem. The foundation created a scorecard tool that assesses several important security "checks" where a score (0-10) is rendered for each check that is applied to a software package or its dependencies. This helps you enforce best practices to guard against malicious actors and vulnerabilities in your build system or source code. Refer to https://scorecard.dev/ to learn more.
Concert applies the OpenSSF scorecard to generate a reliability score, giving you insight to the criticality of security risks within your software packages. Each check returns a score of 0-10 and an associated risk level (critical, high, medium, and low) for each package. There are several standard checks, and each automated check returns a score out of 10, with 10 being the best score. The scorecard is based on an aggregate score, which is a weight-based average of the individual checks that are weighted by risk. The checks with risk level 'Critical' are weighted at 10, while the risk level High is weighted at 7.5. Medium and Low risk checks are weighted at 5 and 2.5. By using a weighted average, the scorecard can effectively prioritize high-risk issues over low-risk ones, helping ensure that the most critical vulnerabilities are addressed first.
Concert assesses the software supply chain to identify weaknesses in various components, such as source code, build processes, dependencies, testing procedures, and post-release maintenance.Concert displays score for each of these checks within a package and also displays a description against each check describing what failed. Reliability score helps quickly identify if best practices are being followed or not.
| Name | Category | Risk Level | What it does |
|---|---|---|---|
| Binary-Artifacts | Cybersecurity | High | Determines whether the package is has binary artifacts or not. The only possible scores for this check are 0 (indicating binaries are present in the project) and 10 (ideal score, indicating no binaries are present). Learn more. |
| Branch-Protection | Secure development | High | Assesses whether the branch is protected in some capacity with GitHub's branch protection or repository rules settings. This allows maintainers to define rules that enforce certain workflow for branches, such as requiring review or passing certain status checks before acceptance to the main branch. A low score indicates that the package is vulnerable to intentional malicious code injection. Learn more. |
| CII-Best-Practices | Cybersecurity | Low | Determines whether the project has earned an OpenSSF (formerly CII) Best Practices badge the passing level or higher. Learn more. |
| Code-Review | Secure development | High | Determines whether the project mandates code review before pull requests are merged. A low score indicates a high risk of unintentional vulnerabilities or possible injection of malicious code. Learn more. |
| Dangerous-Workflow | Cybersecurity | Critical | Determines whether the project's GitHub action workflows have dangerous code patterns, such as untrusted code checkouts, exposed secrets, or use of untrusted inputs in scripts. A low score indicates a critical risk of exploitable code. Learn more. |
| Fuzzing | Cybersecurity | Medium | Determines whether the project uses fuzzing, as in the practice of feeding unexpected or random data into a program to expose bugs. A low score indicates a medium risk of possible vulnerabilities in the code. Learn more. |
| License | Policy | Low | Detects the presence of a published license file and its dependencies. A low score indicates the potential lack of a license file in the top-level directory as well as the associated dependencies. Learn more. |
| Maintained | Secure development | High | Assesses activity within a project to help ensure that it is actively maintained. A low score indicates an archived project or one that has not been updated recently, presenting a high risk of unpatched vulnerabilities. A higher score indicates that there is at least one commit per week in the past 90 days. This check only succeeds if the project is more than 90 days old. Learn more. |
| Packaging | Cybersecurity | Medium | Determines whether a GitHub project is published as a package. A low score indicates a risk of missed security updates. Learn more. |
| Pinned-Dependencies | Cybersecurity | Medium | Determines whether a GitHub repository "pins" dependencies that are used during the build and deploy process. A pinned dependency is a dependency that is explicitly set to a specific hash instead of allowing a mutable version or range of versions. Learn more. |
| SAST | Cybersecurity | Medium | Determine whether the project uses Static Application Security Testing (SAST), also known as static code analysis. A higher score indicates the possible presence of unknown bugs. Learn more. |
| Security-Policy | Cybersecurity | Medium | Determines whether a project has published a security policy (SECURITY.md) and assesses the quality of its contents. A low score indicates a risk of insecure reporting of vulnerabilities. Learn more. |
| Signed-Releases | Secure development | Medium | Determines whether the project is cryptographically signing release artifacts by looking for a digital signature for each release. A low score indicates a risk of installing malicious releases of a package. Learn more. |
| Token-Permissions | Secure development | High | Determines whether the project's automated workflow tokens follow the principle of least privilege. A low score indicates a higher vulnerability to malicious code injection. Learn more. |
| Vulnerabilities | Cybersecurity | High | Determines whether the project has open, unaddressed vulnerabilities in the codebase or dependencies that use the OSV (Open Source Vulnerabilities) service. A low score indicates a higher risk of known vulnerabilities. Learn more. |