Check out the on-demand sessions from the Low-Code/No-Code Summit to learn how to successfully innovate and gain efficiencies by improving and scaling citizen developers. look now.
Cyber attacks succeed by using social engineering and spear phishing to find and exploit vulnerabilities in IT environments, endpoints and corporate identities. They often launch persistent threats immediately, then steal credentials to move laterally through networks undetected. MITER chose this breach sequence for its first book closed “MITER ATT&CK Assessments for Security Service Providers.”
The purpose of the ATT&CK assessment is to test suppliers cyber security efficiency. How ready, capable, and accurate are these solutions to identify and stop an attempted breach? without know when and how it will happen?
MITER Engenuity ATT&CK assessments are based on a knowledge base of tactics, techniques and sub-techniques to keep assessments open and fair. MITER AT&CK Matrix for Company is the most commonly used framework for evaluating the security of enterprise systems and software.
Stress tests of managed services and MDR
Historically, MITER ATT&CK assessments have informed security vendors upfront – prior to active testing – what intrusion attempts and breaches they will be tested against and why. With this advance information, providers are notorious for game ratings, leading to inaccurate results.
In a closed-book assessment, vendors do not have prior knowledge of the threats they will face during the test. MITER ATT&CK Assessments for Security Service Providers are the first closed-book assessment designed to test the technical effectiveness and real-world capabilities of vendors’ Managed Services or Managed Detection and Response (MDR) solutions.
>> Don’t miss our new special issue: Zero trust: the new security paradigm.<
Closed-book reviews provide the most realistic reflection of a security vendor’s performance in a customer environment. “The closed-book test provides the opportunity to show how security platforms perform against adversary trades in a real environment, as vendors have no prior knowledge to guide their actions,” said Michael. Sentonas, Chief Technology Officer at CrowdStrike.
MITER’s assessment of MDRs is particularly relevant, given that chronic shortages of cybersecurity skills put organizations at higher risk of breaches. According to (ISC)² Cybersecurity Workforce Study“An additional 3.4 million cybersecurity workers are needed to effectively secure assets.” Managed detection and response (MDR) provides organizations with an effective way to close the skills gap and improve business resilience.
The MITER Security Service Provider assessment took five days, with a 24-hour reporting window. Sixteen MDR vendors participating in the program had no prior knowledge of the adversary or its tactics, techniques, and procedures (TTPs). They were each ranked over 10 stages comprising 76 events, including 10 unique ATT&CK tactics and 48 unique ATT&CK techniques.
“We selected OilRig based on their defense evasion and persistence techniques, complexity, and relevance to industry verticals,” writes Ashwin Radhakrishnan of MITER Engenuity. The first round of MITER ATT&CK assessments tested vendors by emulating the TTPs of Oil rig (also known as HELIX KITTEN), the adversary group with operations aligned with the strategic objectives of the Iranian government.
The attack scenario began with a spear phishing attack against a national organization using malware associated with the HELIX KITTEN campaigns. Then, the simulated attack threat initiated a lateral movement through networks to identify and collect critical information, with the end goal of data exfiltration.
Combining human intelligence with AI and ML yields the best results
MDR vendors with multiple generations of platform products and managed services experience, using a combination of artificial intelligence/machine learning (AI/ML) and real-time human intelligence, have done the best in the MITER assessment. The top four vendors, those that detected the most of the 76 adversary techniques, were CrowdStrike Falcon Complete, Microsoft, SentinelOne, and Palo Alto Networks.
These MDR providers draw on the knowledge and intelligence of experienced security analysts who use AI/ML applications and techniques designed to analyze telemetry captured from endpoints, networks, and the cloud infrastructure. The result: AI-assisted threat hunting expertise that enables their solutions to identify and thwart vulnerabilities.
MITER Engenuity summarizes its test results in ATT&CK® Ratings: Managed Services – OilRig (2022) and the Top 10 Ways to Interpret the Results. This document provides an overview of the methodology and the interpretation of the results. MITER also makes the layer file graph available for further analysis in its ATT&CK Browserindicated below.
The results of the 16 vendors who participated in the MITER ATT&CK assessments for security service providers showed the factors that enabled vendors to do well. The vendors that have done the best are experienced operators of their own security technologies. They offer a holistic range of features across their security portfolios. These vendors consistently produced the best security results with the highest detection coverage in the study.
CrowdStrike topped all vendors in this category by reporting 75 out of 76 counseling techniques used in the MITER ATT&CK evaluation. Additionally, in line with the fact that the most successful vendors built real-time threat intelligence into their platforms and managed services, CrowdStrike was able to internally identify the nation-state emulated adversary in less 13 minutes.
For an MDR, AI-assisted threat intelligence is essential
Achieve the convergence of AI, ML and human intelligence in an integrated MDR solution is the future of cybersecurity. Therefore, product lifecycles for cybersecurity platforms must be tightly integrated with MDR workflows. This way, valuable features, such as native first-party threat intelligence, become truly actionable.
The evaluation showed how MDR solutions capable of generating or creating and then verifying threat intelligence are successful in identifying the most events. CrowdStrike’s reliance on Indicators of Compromise (IOC) and other strategic intelligence integrated into all of their products show how threat intelligence can be scaled in an MDR solution. Identifying the nuanced aspects of MDR solutions and what companies should look for in a solution is why MITER ATT&CK Assessments for Security Service Providers are so valuable to organizations that look to these benchmarks for guidance.
VentureBeat’s mission is to be a digital public square for technical decision makers to learn about transformative enterprise technology and conduct transactions. Discover our Briefings.