Stop Pentesting. Start Programming.

As I mentioned in my blog about joining ProjectDiscovery, I came from mostly a DevOps background - I was just getting into software development as a career when the term "DevOps" was coined in 2009, almost 15 years ago. And so I got to see the problems that existed in getting code written and deployed to production in reliable, efficient, and repeatable ways.

DevOps purports to solve a lot of those classic problems - by allowing developers and operators to work more closely together and work on systems to get things done, rather than just throwing code and bugs over a wall at each other. And 15 years later, we've seen countless different kinds of implementations of "DevOps'' in quite a large range - all the way from a single team with everyone on it to creating specialized teams with DevOps in their names. The right solution could be debated until the heat death of the universe. However, I think there are still lessons to be learned from this history as we look at the next term-of-art that is getting used (and abused): DevSecOps.

The biggest takeaways we learned from the DevOps revolution are that repeatability and automation are the keys to enabling teams to move quickly without breaking too many things. Gone are the days (hopefully) where developers are deploying directly from their laptops or operators are deploying by SCP-ing JAR files to various servers and hopefully remembering all the steps to get it right (ask me sometime about the group I worked with who had an 11-page single-spaced Word document on how to deploy the system).

DevOps brought with it practices that made the "Ops" side of the world more programmable: just look at all the tools Hashicorp has brought to market since they were founded in the early days of DevOps (2012). Terraform allows for programmable infrastructure, Packer automates build management, Nomad allows programmable scaling, and the list goes on. These tools and many others (like various CI tools) are the foundational ones that allowed us to move from a manual error-prone environment to one of repeatable builds, repeatable deployments, and continuous delivery.

The next turning: DevSecOps

While DevSecOps as a term is already being overused by countless folks - even faster than we learned to overuse "DevOps" in fact - the foundation of this word can give us some key insight into why it even was a term to begin with.

Security has become the forefront of basically every discussion about software today. Just as 15 years ago as the second wave of the internet revolution and the app revolution took hold, businesses realized that to compete they needed to be able to build and deploy software faster, today they understand that the competitive advantage (or biggest threat) comes from security issues. Where speed was key before, security is now at the forefront.

In a barely-post-pandemic world, we've only come to rely even more heavily on technology to connect us, protect us, and surround us. With this comes an ever-increasing threat to the security of our online data and applications. Businesses must find a way to respond, and that is by looking for the same qualities that we looked for in DevOps: reliability, efficiency, and repeatability.

The "current" state of security

Unfortunately, we often have just the opposite of reliability, efficiency, and repeatability. Many security programs rely heavily on humans or human processes as the primary method of understanding the security attack surface and vulnerability of that attack surface.

If we look just at what makes someone "compliant" with security, we find things like penetration tests. But how often are those actually run? Once a year? Once a quarter? Maybe even once a month if we think of ourselves as very advanced? On the other hand, how often are attackers looking to gain a foothold in your organization - the real threat you're trying to prevent? I bet it is much more often, and more than likely is "daily."

More mature security organizations may employ more active techniques like bug bounty programs or internal offensive security teams. And these are essential pieces of the puzzle - but the problem is still relying on humans and their set knowledge of the environment that they are operating in. When's the last time you reassessed the scope of either of these programs? What percentage of your external attack surface do you think they cover? 80, 90, 95%? Again, a hacker doesn't care about what percentage of your attack surface is covered...they only need to be right once.

What is needed

So what is needed here is an additional layer of protection - and an additional set of tools that allow us to better leverage the human time we spend on addressing these problems. Just as DevOps didn't replace the SRE or software developer but instead gave them the tooling they needed to be efficient and effective, so too must security engineers, offensive security teams and CISOs look to automation to extend their offensive security programs.

And when looking at this extension, we have to think through all aspects of our offensive security program and bring to bear tooling that addresses each of the areas of concern:

• Asset Discovery
• Asset Enrichment
• Exploitable Vulnerability Detection
• Remediation & Regression Protection

Let's look at how creating an automated program around each of these aspects is critical for the next generation of security engineers and CISOs.

Asset Discovery

The first question we have to ask ourselves when we want to secure our external attack surface is: What is our external attack surface? Often times this involves a good amount of guessing - we might know what domains we own, maybe we have access to cloud DNS and services records. But we have to combine that data with the data that can be found in the various corners of the internet where an attacker may look.

Public databases of DNS and previous DNS records, archived information about our attack surface, internet-wide search engine results, and public database of certificates that we have issues are all places that we should look to ensure we have a complete picture of our external attack surface. To do so manually - or even to write countless bash scripts to do so semi-automatically, just won't cut it. We have to leverage the same tooling that attackers and pentesters do to do complete asset discovery.

Asset Enrichment

Once we have a good idea of what external attack surface exists, we need to enhance and enrich that data with information about what cloud assets we have, what technologies each of these assets is hosting and where they are hosting it.

This includes identifying HTTP/S endpoints - both ones that respond positively and negatively, as well as looking for other open ports and spidering around our web properties to understand what other APIs and services they are calling.

This is yet another great case for automation - the more that we can repeatably iterate on what is in our attack surface and what is running there, the more likely we are to be able to respond quickly to a changing threat environment.

Additionally, understanding when NEW assets or technologies enter our scope is critical to seeing emergent places for attackers to gain a foothold into our organization.

Exploitable Vulnerability Detection

Perhaps the most crowded part of this over all security tooling space would be vulnerability detection. There are dozens if not hundreds if not thousands of tools that will happily report back dozens if not hundreds if not thousands of "vulnerabilities" to you once you have your attack surface mapped.

But if you've been in security even for a few days, you already know that not all vulnerabilities are created equal. Some things flagged as "vulnerabilities" are really just warnings or flags that are important to an auditor, but may not represent an actual exploitable vulnerability in one of your systems.

Doing automated, repeatable, customizable scanning for exploitable vulnerabilities from the outside looking in is the best way to help your team shift through the noise to find the signal of the vulnerabilities you really care about. And that's a struggle when using a tool that might be mostly a "black box" where you put assets in one side and get a pile of "vulnerabilities" out the other.

That's where ProjectDiscovery's most popular open source tool - Nuclei - is helpful. Nuclei is powered by a massive open source repository of community generated templates. That means these templates are used by thousands of engineers to run millions of scans a month and thus are being fine-tuned by both ProjectDiscovery AND our community to look for the most critical, most exploitable of vulnerabilities.

And it doesn't stop there. You can also make your OWN nuclei templates - they are written in a simple YAML-based DSL - and thus focus on the things your team really cares about. Building bespoke Nuclei templates puts your team in the driver's seat and allows you to own your own offensive security automation pipeline without having to rely on an unknown "black box."

Remediation & Regression Protection

For many folks - especially those looking to attack you - the problem ends there. Once an exploitable vulnerability is found, then it is "someone else's problem" right?

Wrong. Most security teams I've spoken to speak about the struggle of integrating their findings and feedback into the development lifecycle, and then understanding how they can track and confirm the remediation of those various findings. This is where automation again can be a huge time saver - and stop things from going through the cracks.

Once our automated systems know that X vulnerability exists on Y asset and can be reproduced with steps A, B, and C, then that system should track that vulnerability until it is remediated. In addition, just because we find the vulnerability in one system doesn't mean our work is done: we need to be able to quickly and efficiently test those steps against ALL relevant systems...yet another argument for a fully automated offensive security program.

And finally, once the fix is implemented, we want to put in place regression testing to ensure we aren't "bitten" by the same bug again. Yet another place that automation can shine...by building it into our program, we "never forget" about exploits we've already solved for.

And this is what ProjectDiscovery is currently working on. We're currently in the beta of Nuclei Cloud, a platform which brings together all the tools we've discussed so far as well as automated retesting, regression testing, and integration with the systems your development teams use to track issues.

Start Programming

We've seen the wide range of options there are around building an automated offensive security program. Armed with these tools and knowledge, I'm confident we can all work to democratize security by leveraging automation to make us all as security engineers more efficient and effective. All the while keeping in mind repeatability and our ability to iterate on the plan as the attack surface and attack methods change.

Pentesting shouldn’t be a once-a-year event, it should be a continuously improving automated process.

Vulnerability discovery shouldn’t be based on protecting you from an audit, but from an attacker.

Your offensive security program should be able to scale far beyond how many red team members you have and your bug bounty budget.

If you decide to stop pentesting and start programming, you’ll find that building an automated offensive security program just might be the best investment you’ve ever made.