In the world of application security testing, the terms “code coverage” and “vulnerability coverage” are frequently used. But what do they really mean? Code coverage is the amount of code that is scanned to identify potential vulnerabilities in a software application. Vulnerability coverage refers to the number of specific defects or system misconfigurations found in the software code that could pose potential threats.
Should your AppSec team aim for 100 percent code and vulnerability coverage? Is that realistic? The real answer is, “It depends.” Let’s look at the various considerations and how you can get the best coverage possible.
Code and vulnerability coverage: SAST, DAST, and more
Software developers and penetration testers use sophisticated software assurance testing tools to find vulnerabilities in their software. The challenge is that no one tool (or type of tool) provides adequate coverage of an entire target application. The application security testing tools on the market today find specific weaknesses in applications. Although they may be excellent in identifying specific vulnerabilities, no one solution can do it all.
Each tool specializes in different languages and different weakness classes (e.g., buffer handling, file handling, initialization and shutdown, and number handling). While you may be ecstatic with the results of a test that identifies certain types of weaknesses, if you are only testing a small portion of the code, the results don’t provide a true representation.
Static Application Security Testing (SAST) tools, also known as white-box testing tools, are used early in the software development process to test the application from the inside out. They go line by line to test the source code, byte code, or the binaries.
With SAST tools, 100 percent code coverage is possible since they have access to the internals of an application.
However, even static analysis tools that have access to all of the code do not provide full vulnerability coverage. In fact, experts say that the average tool only covers 14 percent of the vulnerabilities in your code. Therefore, it is an industry best practice to leverage multiple tools and types of tools that complement each other.
Black-box testing tools, also known as Dynamic Application Security Testing (DAST) tools, often have limited code coverage, based on how much of the attack surface they are able to identify and the input they fuzz into the application to cause different types of vulnerabilities. DAST tools test from the outside looking in. They test the application when it is running, attempting to penetrate it to find potential vulnerabilities, including those outside the code and in third-party interfaces.
Other types of tools, such as Interactive Application Security Testing (IAST) tools, threat modeling tools, and even manual testing are also important pieces in the AppSec puzzle if you want to achieve comprehensive code and vulnerability coverage. Even then, we recommend you go one step further, using tools that will help you make sense of these AppSec tools and truly see how complete your code coverage is.
Beyond AppSec tools: Improve visibility with code coverage tools
Although there are multiple advantages to using more than one application security testing tool, there are also hurdles to overcome. With each additional tool comes an additional cost, more time to implement and run it, and the challenge of comparing the differing sets of results (e.g., naming conventions and severity ratings). This is where code coverage tools, such as an application vulnerability manager and a visualization tool, provide unparalleled benefits.
An application vulnerability management tool correlates and normalizes the results from commercial and open-source tools. It delivers a consolidated set of results that provides greater coverage of potential vulnerabilities in the source code and a better assessment of an organization’s overall enterprise risk.
This one tool handles deduplication, remediation management, reporting, and compliance checks. Workflow integration options allow your developers to stay in their preferred environment—Eclipse, Jira, Jenkins, and others—while addressing application vulnerability issues.
The Code Dx application vulnerability management tool also provides Application Vulnerability Correlation (AVC), commonly referred to as Hybrid Analysis. This refers to the combination of SAST results (which identify potential vulnerabilities) with DAST results (which identify which threats are actually exploitable). This allows you to determine which threats exist in your code and can be exploited by an outside attacker, so you can address them first.
The Code Dx team has also developed a free OWASP solution, called Code Pulse. This open-source penetration testing visualization tool provides insight into real-time code coverage analysis testing. It helps your testing team evaluate the performance and coverage of each tool being used for application security testing.
It does this through a visual illustration of your application’s attack surface and how your penetration testing interacts with it. Because it functions in real time when your application is active, you can tell exactly which parts of your code are covered by the penetration test—and which parts aren’t.
Code Pulse also shows you which parts of the application are covered by each tool, so you can see where there are overlaps—and, more importantly, where there are gaps. This helps you assess your code and vulnerability coverage and determine whether or not you need to add different tools to your testing process. You can compare coverage across all of your tools quickly, see any code that hasn’t been tested, and view the results of scan setting adjustments immediately.
In summary, SAST tools can provide 100 percent code coverage (unlike DAST tools), but they do not provide 100 percent vulnerability coverage. To get as close to 100 percent vulnerability coverage as possible, we recommend using a combination of SAST, DAST, and other AppSec testing tools to obtain comprehensive coverage. While this may seem like a daunting task, an application vulnerability manager helps you make sense of the results of these tools. A visualization tool such as Code Pulse makes it easy to see where you’re covered and where you need to amp up your protection.