So you’ve decided you want to integrate Software Composition Analysis (SCA) into your security tooling? Or perhaps you are revisiting your SCA tool and looking at alternatives and seeing how the market has evolved since your original selection of an SCA tool?

In this article, we look at some key considerations when making a decision around what SCA product to adopt to ensure you find the right fit for your organizational needs.

What is Software Composition Analysis?

You’re probably already familiar with the concept of SCA. Before we dive into the key considerations when buying, it’s helpful to provide a definition because all the considerations in this article will tie back to the definition.

SCA tools identify the third-party components (including open source software, or OSS) used in your application code and the corresponding risks those components might present to your organization. This is relevant and critical for the modern security tech stack because several studies indicate that modern code bases contain significant amounts of OSS and in fact, the majority of the code bases are OSS overall. 

At their most basic, an SCA tool must:

• Generate an accurate inventory of your third-party components.
• Highlight the known vulnerabilities associated with each component (and probably include license risks also).

Really good SCA tools add three capabilities that save time and improve credibility:

• Identify other supply chain risks (even if they aren’t security-specific), like malware, abandoned OSS projects, outdated and unused components, and so on.
• Facilitate risk-based prioritization to determine which findings pose true risk.
• Provide detailed information on the impact of remediation (possible breaking changes, ROI of the upgrade) so you can prioritize what to fix first.

While it’s easy to think SCA is a commodity, a strong SCA program is even more relevant today because we continue to see an increase in software supply chain attacks. Malicious actors target widely-used OSS components such as Log4j, xz utils, and more — all in attempts to impact the massive ecosystem of organizations and businesses relying on these components. Defending against such attacks requires an accurate software inventory that includes the composition of your applications and libraries.

Seven Questions to Ask SCA Vendors

You’ve decided it’s time to implement your organization’s first SCA tool, or you’re unhappy with your current tool and are ready for a change. Fantastic! 

You’ll have lots of questions for potential vendors. We recommend starting with seven questions before you even think about committing to a POV:

• What languages do you support?
• What integrations do you support? (and can you use an API?)
• How do you correlate vulnerabilities to components?
• Can you detect transitive and phantom dependencies?
• What types of risks can you detect?
• How does your reachability analysis work?
• What reporting and metrics do you offer?

What Languages Do You Support?

The most fundamental consideration for SCA tools is the breadth of coverage when it comes to language support. If the SCA tool is limited to one or a couple of programming languages, you may find yourself with gaps in coverage across your enterprise. This is especially problematic if you have a large diverse developer base with a myriad of applications and programming languages in use. 

Language support also requires being forward-thinking and having confidence that your selected SCA tool will not only support the languages currently used in your environment, but also those which are likely to be used in the future. Support for languages your organization might consider adopting in the near future is important, of course. But so is support for “rising” languages, even if you have no plans to adopt them. A vendor supporting a language like Rust, that is rising in popularity but is still not widely adopted, is a good indicator of that vendor’s commitment to promptly developing support rather than waiting for wide adoption. This is a good sign for you, even if you have no plans to adopt that particular language.

What Integrations Do You Support? (And Can You Use an API?)

Modern SCA tools need robust integrations to ensure a seamless flow of information to those in a position to take action. This means integrating with CI/CD platforms, pipelines, vulnerability management aggregation tools, enterprise risk registers, and common ticketing tools.

Just as important is how those integrations work. SCA tools can be clunky and a burden to manage when they don’t have capable API’s for exchanging data. This makes their adoption challenging, rife with risk and less likely to succeed. Out-of-the box integrations with key systems (like GitHub and Jira) can be important to a smooth and rapid rollout of SCA tooling, but it’s equally important for the product to have customer-accessible APIs that are:

• Complete: Access to all of your data and all hosted/server-side features should be possible via API
• Usable: If the API is complex or doesn’t clearly map to the results data model, it can be untenable to use it

Modern tooling should have an API-centric design (ideally API-first, where the API for a capability is developed and then the product’s own tooling is built, consuming that API). This allows you to inexpensively customize integrations to your own needs, and to develop integrations for uncommon (even custom-built) or legacy systems your organization might use; and keep the maintenance and operation of those systems low-cost.

How Do You Correlate Vulnerabilities to Components?

A big part of SCA scanning is tying vulnerabilities to the third-party components. The reliability of your vendor’s vulnerability data can be the difference between safety and a breach. We talked about this consideration extensively in OWASP OSS Risk 1: Known Vulnerabilities, so in this section we’ll summarize: Don’t rely on a vendor that only uses NIST’s National Vulnerability Database (NVD).

While the NVD is the most widely-used vulnerability database, it also has some challenges that make it inadequate as a standalone database:

• Not Up-to-Date— The program has drastically slowed down when it comes to enriching CVE’s with data such as CPE’s, CVSS, and more. This is due to challenges with the program's funding and logistics, which has garnered widespread media and industry attention, including letters to Congress and public explanations from folks at NIST who help lead the NVD program. While those challenges seem to be getting addressed now, it does raise long term questions and concerns. What if the NIST NVD has funding or resource issues in the future? Will they actually meet their commitments to address the backlog of unenriched CVE’s? and Will they be making improvements to address some of the longstanding deficiencies of the NVD? 
• No PURL Support— NVD doesn’t currently provide support for Package URLs, or PURLs, which are often used by the OSS and package manager ecosystem for identifying software components. Lack of support for this standard makes it difficult to accurately match an OSS component to NVD entries. There have been efforts to petition for PURL support, but so far that hasn’t materialized and is likely further impeded due to the funding and logistic challenges.

To address gaps with NVD, SCA tools and organizations have increasingly sought secondary sources for vulnerability intelligence. Some of the most notable examples include Github’s Advisory (GHSA) Database, OSS Index, and OSV, each of which have more of a core emphasis and focus on the OSS ecosystem and are likely more relevant for organizations looking to secure third party components.

Proprietary Vulnerability Databases

The sad reality is that all public databases have problems that can result in biased scoring, incomplete affected version data, and inaccurate or incomplete remediation data. A strong SCA tool will supplement public information with their research. 

• The Endor Labs Vulnerability Database is based on NVD, GHSA, and OSV data along with a manually-annotated, function-level database for vulnerabilities going back to 2014, that can be used to determine if the vulnerable function (as opposed to the vulnerable dependency) is reachable (more on that below). This database is updated every 12 hours and analytics are re-run automatically for every project at least once every 24 hours (or sooner based on project configuration), which means that new vulnerabilities are automatically reported, without customers needing to rescan their projects.

Can You Detect Transitive and Phantom Dependencies?

Continuing on the theme of establishing full visibility of the third-party dependencies and corresponding risks, we shift to the types of dependencies the tool can detect. A strong SCA tool doesn’t stop at direct dependencies, but also includes transitive and phantom dependencies. In other words, the tool needs to be able to find all your dependencies, including ones that weren’t explicitly selected by the developer or listed in the manifest.

There is some debate amongst SCA vendors on whether this capability matters so we’ll point to research: As documented in the 2022 State of Dependency Management Report, a startling 95% of vulnerable dependencies were transitive dependencies. 

Your adversaries don’t care if a vulnerability is in a transitive dependency or not, they only care if they can manage to exploit it. Further, your auditors expect you to provide an accurate software inventory; if you leave out dependencies, you will lose credibility and may fail an audit.

What Types of Risks Can You Detect?

While known vulnerabilities are a great metric, they simply aren’t the only metric that needs to be accounted for when looking at third-party components and OSS. Known vulnerabilities are lagging indicators of risk, meaning the risk is already known and potentially more likely to be exploited. 

You also want to account for leading indicators of risk, such as unmaintained software, projects trending toward low activity or low quality, outdated software, name confusion attacks and even the compromise of a legitimate package. These leading indicators of risk are captured in the Top 10 OSS Risks, which was originally created by organizations such as Endor Labs prior to being adopted and run by OWASP. 

How Does Your Reachability Analysis Work?

Now we’ll shift from visibility into your risks to the capabilities that can help you prioritize remediation. While logically it makes sense that you’d want the SCA tool to help with prioritization, the industry has been drowning in SCA noise, or alerts associated with findings which simply don’t pose significant risk to the organization. This leads to massive vulnerability lists being dumped on developers, slowing down development velocity, impeding business outcomes and fostering resentment between security and development teams. 

Noise is a symptom for an underlying problem that prevents prioritization: Visibility without context. In the case of SCA, context means “does this vulnerability pose risk to my unique environment?” Afterall, a CVE is simply a warning that something could be exploited.

Reachability analysis — determining whether vulnerable code can be “reached” by malicious actors — has become the gold standard for adding context to SCA findings. There are several types of reachability analysis, so it can get confusing (dare we say noisy) when researching tools with the capability. Look for function-level reachability, which tells you whether something is exploitable at the function level. This gets the closest to determining whether a specific vulnerability is applicable to your environment. While no tool can promise a 100% false positive reduction, SCA with function-level reachability comes the closest.

Figuring out which parts of a package are vulnerable is an important piece of context, but don’t stop there! In addition to reachability, an SCA tool should provide other contextual filtering such as whether the dependency is in production, whether there is a fix available, and the exploitability of the vulnerability (measured by EPSS). 

You might notice we haven’t talked about CVSS yet. While prioritizing by CVSS is a popular approach, the industry has accepted that it has flaws. Researchers are incentivized to assign high scores because there is more value in their work if what they find is seen as more critical, while vendors and maintainers are incentivized to report low scores so that their software is seen as less risky. This results in bias even when reporters are acting in absolute good faith (the usual case), but can sometimes produce bad-faith scoring. We aren’t saying “don’t use CVSS”: It is useful in deciding what to remediate after you’ve evaluated the exploitability of the risk.

Three Reachability Traps

When using reachability as a risk management strategy, especially for compliance, you’ll need to defend how it works to your developers and auditors. There are three reachability traps that will harm the credibility of your SCA tool.

• Trap 1: It trusts the import statements
  As we discussed above, one of the most important functions of an SCA tool is to generate an inventory of your third-party components. Many tools rely solely on the package manifest to determine what’s being used, and specifically the import statements, which essentially tell you “the developer said, in the code, that they intend to use package X.” This isn’t a guarantee that the package is actually in use, or that it contains all the packages, so you’re likely to run into false negatives and false positives.
• Trap #2: The call graphs that stop at direct dependencies
  While function level reachability is becoming the standard for reachability, not all tools will provide it for your transitive dependencies. The consequence in this scenario is you’ll have a high false negative rate.
• Trap #3: Relies on fix commits for vulnerable functions
  Some tools will look at fix commits and draw the conclusion that if there was a function change in the commit, then that must be where the vulnerability lies. But in reality, commits often make several changes to several functions (not just the one with the vulnerability in it). In this case, the tool incorrectly thinks a collection of functions are vulnerable, when it may only be one function, so you have a higher false positive rate.  

What Reporting and Metrics Do You Offer?

In addition to many of the items discussed above, a key consideration is reporting and metrics. Can the tool go beyond surface level information (such as CVSS scores) and actually show further insight that make the findings actionable?

Valuable information may include whether a vulnerability is known to be exploited, likely to be exploited, reachable, has a proof-of-concept exploit code available, is reachable, is impacting a business critical asset and more. 

Information needs to be presented in an easy to digest format that has a desirable user and developer experience, empowering developers to actively address known risks and not dread using the tool. Mature organizations will often have their own reporting and management infrastructure, whether that’s something like a SIEM, incident aggregator, GRC, or BI toolchain. This means your SCA solutions should also provide convenient ways to export data to simple formats and to access all relevant data via a standardized API.

Other key considerations include regulatory and compliance needs when it comes to reporting. Increasingly we’re seeing compliance frameworks account for software asset inventory including third-party components, including examples such as FedRAMP, CISA’s Secure Software Self Attestation Form, FDA’s Pre-Market requirements and more. Being able to show auditors insights such as components being used, their vulnerabilities, and risks to customers of your products or even internal applications and software.

You also need a tool that can facilitate activities such as incident response. For example, the next time a highly visible open source compromise occurs, being able to understand where the component exists in your enterprise, what applications are impacted, and downstream implications for internal and external consumers of those applications is key.

SCA with Endor Labs

Endor Labs was founded with the mission to make SCA a more effective tool for software supply chain security. With Endor Labs Open Source, you can automate OSS selection and approval, identify applicable risks in OSS libraries and container images, reduce SCA noise by 80%, and implement accurate SBOMs.

• SCA with Function-Level Reachability: Prioritize the handful of vulnerabilities that actually matter, and help developers manage the security and health of their direct and transitive open source packages.
• AI-Assisted OSS Selection: Enable developers to select “good” OSS packages using natural language.
• Container Image Scanning: Consolidate SCA findings with findings for OS base level image risks and application-level risks.
• SBOM Generation: Export software inventories for applications, including VEX documents to annotate vulnerability information.
• Artifact Signing: Add metadata to your artifacts, including code and containers, to provide assurance that you’re following secure development practices.

Put our SCA (and everything else) to the test with a free, 30-day trial.