A key component of DevSecOps and identical to running unit tests to validate code after a build, running automated security testing after an application has been deployed (such as automated penetration tests) can provide us with a tool that identifies security risks. As we’ve seen recently, there’s been a growth of many companies experiencing information being compromised and with the development culture of “move fast and break things”, I expect this trend of successful attacks will continue. Before we look at our options for automating this testing, we want to be aware of its limits, evaluate the requirements, and consider common designs that are useful.
This Does Not Solve Zero Days
Before we look at how to add this to our build and deployment process, it is worth noting that automated security testing will not solve or prevent zero days. Zero days are unknown vulnerabilities that attackers exploit because they are aware of them, either through their own discovery or through an interaction (such as a financial interaction where a zero day is purchased). In the same manner, new attack techniques like Duqu 2.0’s attack from operative memory may not be detected either even if it’s not a zero day because the technique is unknown before the attack. New techniques and zero days are discovered after the fact sometimes because of a compromise. While this doesn’t mean we should avoid automated security testing, it does mean that we should be aware of the limits that our testing will have – we cannot know what we don’t know.
Requirements
Before we can run automated security testing, we have to design a build process across our environments that allow us to execute automated security testing our application and database schema – in other words, our application design must follow from our lower environments of development through production. For an example, if our web application uses 20 procedure for CRUD operations and we want to run penetration tests against these stored procedures, if they’re completely different across environments, we’d have to run these tests in each environment. Will customers or clients enjoy a drastically slowed experience if we’re running penetration tests? More than likely not. However, there may be situations in which we will have to run automated security testing in an environment like production.
When we consider running these tests in our environment, we want to account for some of the following:
- Ceteris Paribus Design. We want to ensure that our design (application and database schema) match from one environment to another. There may be architectural reasons that they don’t, but if they don’t in anyway, we must note these because our automated security tests may give us inaccurate information
- Lifecycle Order. Order matters when we run our automated security testing. Using an example of penetration tests attempting SQL injection, we want to run these tests on a live development site after we develop and complete our new code. For example, if we ran the tests before we deployed new code, we would be testing old code. We’ll first build our web application to pick up the new code, run our unit tests to validate our code, deploy our web application, and then begin our security tests to check for SQL injection
- Scheduling. Development windows also matter since some penetration tests can consume huge amounts of resources in some cases. We may want to run these tests at the end of the development day, but we have to be careful because unit tests may fail and require fixes that don’t allow us time. The schedule of these tests can affect other developers in our environment, so we have to consider this as a major factor
- Limitations. When we run automated security tests to discover weaknesses, we should be aware of what we are testing along with what we are not testing for. Measuring the security of a website as an example runs deeper than simply testing for web or SQL injection, as attackers can use social engineering – something we can’t automate a test for
- Reporting and Action Plan. When our automated security testing completes, we should report the findings in a manner for development teams to review. In addition, we must also have a plan of action if our testing identifies a weakness in our application. Some companies may find it appropriate to have a plan of action on the basis of the risk level of the findings, while others may plan for any security risks
Common Design Models
Once we know what we’ll be measuring, we can add automated security testing to our development and release flow. In the below image of a web application, we see a design where we have developers working in different branches that must be merged prior to a build. After the merge and build, our next step involves running unit tests that will ensure our code works correctly and when our code is validated, we deploy our application. Now that our web application is live, we’ll run our penetration tests and generate a report about the risks that we found. This design is only one model of running these tests because in some contexts, we may be able to measure security before we deploy the final product.
In some cases, we may be able to combine our security testing with our unit testing. In this model we’re looking at our code specifically, not looking at our final product. This is an important point because the underlying assumption of this model is that we can evaluate the code to find weaknesses instead of testing the final product. Both may come with strengths and weaknesses and some stricter companies may prefer to use both of these techniques – running a code analyzer and running penetration tests after the final product is produced. Due to software library complexity, such as various software libraries used in software, it can be difficult to detect possible interactions ahead of time where penetration testing may be able to identify. However, this is only a concern in some situations: some software applications use few libraries or limit how their libraries are used within their application, making the below model appropriate.
I’ve seen appropriate situations where both models were combined: security code tests would run during the unit test step and following the deployment, penetrations tests would run. This adds cost in terms of time and provided we have the development window for this, it can add a more robust development build and release flow.
Warning: Keep Testing Secure
As the we know the answer to “what is the first rule of battle?” is “never reveal your position” we should consider how this applies to the security of our automated security testing. If our testing was compromised or discovered by the wrong actor, it would immediately reveal our position. Attackers would love to know what you automatically test for because for every strategy there is a counter strategy and this will never cease to be true. I would be extremely cautious about where these tests are run. We want to be careful about allowing even internal talent to have access to what is being tested – even if they can see a report of the possible weaknesses.
Conclusion
Although a key component to DevSecOps, automated security testing does not answer possible security compromises like zero days or unknown development techniques, it can reduce the risks of some attacks which we see frequently that involve code injection, weak authentication, misconfiguration, etc. For an attacker, we have to realize that these are “easy attacks” and the purpose of our tests is to give attackers a barrier to jump over. In some cases, the work to jump over these won’t be worth the results and they’ll find other more valuable targets.
Table of contents
| DevSecOps: Developing with Automated Security Testing |
| DevSecOps: Security Testing Around Builds and Shared Information |
He has spent a decade working in FinTech, along with a few years in BioTech and Energy Tech.He hosts the West Texas SQL Server Users' Group, as well as teaches courses and writes articles on SQL Server, ETL, and PowerShell.
In his free time, he is a contributor to the decentralized financial industry.
View all posts by Timothy Smith
- Data Masking or Altering Behavioral Information - June 26, 2020
- Security Testing with extreme data volume ranges - June 19, 2020
- SQL Server performance tuning – RESOURCE_SEMAPHORE waits - June 16, 2020