Going Agile with Pen Testing: Testing the Agility of Offensive Security
Agility. In many domains, you just can’t win without it. Martial arts, jazz, comedy and business all require the ability to think on your feet and pivot, fast and gracefully, when necessary. But let’s not forget the most prominent example of a sector transformed by agility—software product development.
While agile development doesn’t bring overall efficiency, it increases release frequency, shortens development cycles and permits developers to watch users closely and adjust on the fly. Agility gets organizations to “value” quicker. It has profoundly redefined what is achievable through software and technological innovation. As more and more of our business and cultural life goes digital, the consumer demand drives additional adoption of agile methodologies. This revved-up cycle created a massive technology boom, one that echoes throughout everyone’s daily activities.
As the digital footprint around us increases, so does our attack surface and risk. Things that once were islanded or safe are suddenly interconnected and vulnerable. Medical devices, payment systems and smart cars are all being targeted by malicious attackers. The hurtling pace of innovation in the digital space has outstripped our ability to secure these environments effectively.
The question is: How can an organization enforce security controls and still be agile? Most product companies operate in a stop-and-go mode, moving from development to penetration test—when everyone waits four weeks—and then, if all goes well, to release. If you’re lucky.
Yes, the modern Software Development Lifecycle (SDLC) and Continuous Integration/Continuous Deployment (CI/CD) pipeline have security built in—or at least they should, according to current standards and best practices—but what about offensive security and pen testing? How agile can the pen testing process be? And can an agile pen testing approach effectively address the security problem?
The first step in any agile process analysis is to break down large units of work into smaller ones that can be managed at a more granular level. In traditional waterfall software development, the requirements phase is a lengthy process, from concept to definition through final review and approval. Development work would then be performed in strict adherence to approved requirements, which would finally be tested and accepted. The end-result could vary from a perfectly functioning system to a completely un-usable system, due to any piece of this process breaking down. By contrast, an agile approach would start with a broadly defined end-result, supported by a highly iterative development process designed to seek out and find failures quickly, enabling the developers to shift mid-cycle and reach a suitable outcome in a much shorter timeframe.
This breakdown of unit of work may be logically applied to threat modeling in the agile world. First, your agile backlog must be designed into a threat model. You must identify your core asset impact, and test accordingly. Don’t lose sight of your core, but you can still do partial tests. How do you overlap? How do you rotate? How do you prevent predictability? All issues we need to resolve, but all solvable. Once you’ve done this, you can start agile pen testing.
The typical, deep-dive manual pen test follows a similar waterfall approach. The scope is defined up front, and fieldwork follows afterwards, closely aligning to the approved scope and rules of engagement. The pen testing team applies creative approaches to find security vulnerabilities, which are aggregated and compiled into a comprehensive final report, which gets presented to the client. The process results in a list of meaningful security findings which the client will remediate, and the pen tester will re-test to validate. The desired end-result is a material reduction of risk of system breach. Once again, this waterfall-type of approach typically takes several weeks to months to complete. What does this process look like when broken down into more granular steps?
Start with the desired end result—identify relevant system vulnerabilities and remediate them as quickly and efficiently as possible. From here, the process requires two significant deviations from the traditional approach:
(a) an iterative, dynamic process to feed the test objects into the pen testing scope;
—and—
(b) an equally iterative and dynamic process to present pen testing results back out to the client.
Again, both the input test objects and output testing results should be small and managed at a granular level. Once a portion of code or module goes through the pen testing process, the development and app teams work directly off of the iterative reporting output to confirm and remediate these findings. Then this is fed back into the testing process for final validation. Since each unit of work is compact, the process can scale and flex to accommodate various types of test subjects. For instance, complex and highly dynamic systems, as well as ecosystems containing many interconnected systems, can be tested quickly. They can also be aligned to the development-and-release cycle, thanks the ability to parallelize and independently process multiple small units of work. This should include, of course, automation, which means adding in DevSecOps and automating Static Application Security Testing (SAST), Dynamic Application Security Testing (DAST) and Software Composition Analysis (SCA) processes, and designing security into your key management strategy.
With some reasonable re-tooling, the contemporary pen testing process could be redesigned to align more closely with agile methodologies. When integrated into an organization’s development process, the agile pen testing approach could allow for companies to identify, remediate, and validate key security fixes more efficiently than before. Time will tell if this method can solve the security problems we face in the midst of the digital revolution. At the very least, this could be the first (small and iterative) step towards finding that solution.
