10 Reasons You’re Not Getting the Most Value Out of Your Penetration Testing Programs

Over the last 20 years, there has been speculation that penetration tests are dead. “They don’t work!” A big issue with this statement is that there is no clear definition of what a penetration test is. Therefore, how can anyone deem them useful or not useful when people might be talking about different tests? We invited Vice President of Breach, Detection and Response, Laurent Desaulnier, from GoSecure to dive deeper into penetration testing (pentest) and its value. He helped us identify 10 reasons you’re not getting the most value out of your penetration testing programs. We will review them but before we do that, let’s define a penetration test. (Watch the entire 50-minute workshop here.)

“Penetration Testing programs identify exploits, flaws, issues with policies, along with policies that allow poor passwords, and other common security concerns to help improve security posture and close gaps. Code review is also available on request.” (GoSecure)

Now let’s get into why you might not be getting the most value out of your penetration tests.  

‍

1. The Penetration Test is Under Scoped

Let’s review a penetration test request from previous clients. They requested to perform an external pentest on their perimeter with the following restrictions:

A) Exchange, SharePoint and ALL servers are hosted on Azure

B) Our VPN portals, main website and customer portal are OUT OF SCOPE

C) More specifically, here are 3 IP addresses you are authorized to test

‍

It’s absolutely valid to remove items from your scope and sometimes you need to do it. Laurent from GoSecure was telling us about how they were conducting a pentest for a bank and they had a Swift server – used to exchange money between banks with about $1.8 billion going through that server every hour. Obviously, they didn’t want to risk taking down that server and asked for it to be out of scope. That's understandable.  

‍

However, in real life, hackers don’t care what’s vulnerable. They won’t say, “Oh I don’t want to upset this business. I can see that this is vulnerable. I just won’t touch it.” That doesn’t happen. Removing too many items from the scope may secure a “good” pentest outcome but only because the penetration test itself was under scoped. The “good” result might tick off a compliance checkbox but it’s not very useful. Under scoping may be a reason you’re not seeing enough value from your penetration tests.

2. The Penetration Test is Over Scoped

This is an example of an over scoped program that previous clients have requested, “I would like external penetration testing on our 2000 IPs and all our web applications. Our budget is three days of work, including a full report.”

‍

It takes about 4 minutes to test an IP. That’s 8000 minutes testing their IPs or 133 hours which is 5 and a half full days. That request is clearly over scoped and we haven’t even addressed testing the web applications. Based on this request, it is obvious that this company is not looking to improve their vulnerabilities. As we could guess, it’s probably for checking off a compliance check box. By requesting minimal effort, you won’t learn very much from the experience. And when businesses over scope, the value of the penetration tests becomes less valuable.  

‍

To further analyze this situation, there isn’t much value in an over scoped test because the time constraints are not realistic. Hackers do not have any time restraints! Therefore, limiting an external test is not a good representation of what can actually happen to your business. If a penetration test requires more time, it’s actually in your company’s best interest to take the time it needs.  

‍

There's a saying among pentesters that if you burn money, at least you warn yourself of potential harm to your business. In this over scoped scenario, the client is just spending money for no valuable reason. Over scoping might be the reason you are not obtaining as much value as you would like from your penetration tests.  

‍

‍

3. The Penetration Test is Not a Surprise

Often times, when a company conducts a penetration test for compliance, they want a clean test. In other words, they want the employees to pass any phishing tests. As a result, the company tells their employees when the penetration test is taking place. They will literally say, “Hey, tomorrow we are performing a phishing test at noon.”

‍

And guess what happens? The company gets very good results on the test! Warning employees proves nothing. The company gains no value from the test. You WANT to see who will click the phishing links. You want to see if you need security awareness training or to see if a previous security awareness training worked. Hackers are not going to warn your employees when they’re about to send a phishing email.  

‍

On a similar note, I want to emphasize that employees clicking a link does not mean they are a weak link. It does not mean they should be fired. It also does not mean they should be shamed for clicking the link. An employee clicking a phishing link represents a training deficiency. It does not mean they should be fired. The problem is with the training, not the employee. Without training, employees will keep clicking the link and they will keep getting fired. The problem will not be resolved. However, training instilled in the culture will foster better results for a surprise phishing attack.  

‍

In sum, you want to surprise your employees with a phishing attack. If not, you may not see the full value of it.

‍

(Fun fact, most people who “fail” a surprise phishing attack are C-Level executives).

‍

4. The Penetration Test Should Not Be Automated

According to Laurent from GoSecure, an automated test is not a Penetration Test. He said, “for many people, pen tests are basically what we call the Nessus scan or a vulnerability scan, where you have an automated tool and you press next and you have this report saying, oh, these are the findings you have. And for me, that's not a pen test because you have no intelligence out of it. Ultimately, if tools were any good, we would be out of jobs.”

‍

Laurent continues to explain that penetration tests need to be evaluated and looked over by humans. Pentesters are a very valuable part of pen testing. It’s important that a tester links different flaws. For example, you might have two small flaws by themselves that are not dangerous, but when they are linked together, they have a much bigger impact. Only a human is able to assess the risk and deliver that important part of a pentest.

‍

Automated tools don't have this intelligence and thus are not able to provide full value of a pentest. Unfortunately, it's still quite common that companies that offer an “automated” pentest.  As a result ,there is a disconnect between what people understand as a pen test versus what is delivered. That causes many problems.

‍

Note: Vulnerability Assessments (VAs) are still useful. However, it’s recommended to do them in-house. Considering they are fully automated, there is no reason to spend $400 an hour to get another company to do them. Look into tools like Nessus, Nexspose, and Qualys to do them in-house.

‍

5. People Have A Warped Perception of Penetration Tests

Speaking of expectations, Laurent from GoSecure goes into detail about how many people think penetration testing is like what they see in movies or television series. More specifically, they imagine Ferraris, fancy technology, maybe some jets. In reality, not many technology jobs are like that. Hacking especially has been glamorized in the media. Usually, hackers are people in hoodies in front of a computer at a desk. There’s nothing fancy about it. Penetration tests are a form of hacking but definitely not like in movies because the movies are just not accurate. (Note that penetration tests are considered ethical hacking.)

‍

Based on the movies, people also have a warped perception of how much time it takes to hack. In the movies, you see the time running out before a bomb is about to explode and seconds before it goes off the “hacker” is able to save the day. In reality, penetration tests can take weeks. Real-life cyber criminals spend A LOT of time trying to get into IT environments. So in the end, when people think hacking is like in movies, sometimes they get disappointed at what we actually offer. They think it’s not valuable if it takes so much time.

‍

6. When Companies Get a Penetration Test Compliance Reasons Only

Companies getting penetration tests for compliance reasons don’t see a lot of value in the test for two reasons. First, they want to find the least number of problems in the test so they can pass compliance. They will either under scope the program or warn their employees of a phishing test or both. As a result, they spend thousands of dollars to cheat the system. Of course, they’re not going to see the value. The second reason they see less value in a penetration test is because by nature the test is supposed to find as many vulnerabilities as possible when they actually want the least number of vulnerabilities to be found. Their expectations are the complete opposite of what the test was intended to do. As a result, more times than not, when companies get penetration test for compliance reasons, they see the test as a burden.

‍

‍

7. Asking For Approvals for Every Test Is Not Realistic

Some clients have overly controlled requests. An example: “Testers are required to ask permission before running any scans or before exploiting anything. Each vulnerability must be reported before any action is undertaken. Testers will be informed if they can proceed with exploitation after consideration from the Board of Directors.”

‍

Pentesters have to go through hundreds of tests to find the few vulnerabilities that exist. Can you imagine asking for permission for every single test? The goal at GoSecure is to perform the penetration test as quickly as possible while also finding as many vulnerabilities as possible. Asking for an approval for every test is unrealistic. We understand that they want to know what’s going on. However, it’s too costly and unrealistic. When companies ask for overly controlled requests, we know they might be disappointed in the end.  

‍

8. Penetration Tests aren’t as Valuable if You Exclude Known Vulnerabilities  

A reason companies may not be getting the most value out of their penetration tests is if they exclude known vulnerabilities. An example is “Don’t use Pass The Hash, we know we are vulnerable since we are reusing our local administrator account passwords on all endpoints and servers.” You might be thinking, “Well, obviously if we know about a vulnerability, we don’t want to pay for pentesters to find it.”  

‍

But let's not forget the value of a pen test. Penetration tests are to identify vulnerabilities, linking the flaws if there are any and making recommendations on how to make updates. By not including those vulnerabilities as part of the scope, you are not sure if it can be linked to a bigger problem AND you’re not getting any recommendations. At the very least, you want to include known vulnerabilities in the report of the penetration test to make it visible to your management.  

‍

9. Penetration Tests Are the Most Valuable When You Fix the Vulnerabilities After the Test

The goal of a Penetration Test is to identify vulnerabilities and do what is necessary to fix them. The ideal outcome is to use the report of the pentest and start improving the flaws listed from highest priority to lowest priority. Ideally, your company gets a penetration test one year later to assess the changes and potentially find new vulnerabilities. It is recommended to do a pentest every year so you remain proactive in detecting flaws. In some cases, companies get the penetration test and don’t make any changes. As a result, they don’t see the value of the test because they are still just as vulnerable as before they conducted the test. To get the most value out of a pen test, you need to make the necessary changes to fix the vulnerabilities.  

‍

10. Not Patching Before a Pen Test

Finally, before a Penetration Test, you would patch the software that requires patching. A good portion of ransomware attacks are done through software that has not been patched. A Penetration Test is not going to be seen as valuable if it’s going to tell you something you already know is a risk.

‍

Conclusion

Penetration Tests are still alive. However, without a formal definition, everyone doesn’t get value from it. And that is why there are debates on whether or not penetration tests are worth the investment. This blog is a list of 10 reasons companies don’t get the most value of their penetration tests. Now, you are equipped to set the right expectations should you get a penetration test. Our recommendation is to schedule a 15-minute call with us to determine if a penetration test is right for you. Talk to you soon!

‍