On Tuesday June 27, 2023, Forrester released the first Forrester Wave on Network Analysis and Visibility (NAV). This is Forrester’s second attempt at assessing the NAV vendor landscape as the initial NAV Wave was cancelled due to analyst turnover and the resulting lack of expertise. Based on our assessment of the research, expertise in modern threat detection continues to be lacking.
Forrester remains stuck in a vendor-driven mindset defined by rigid tech requirements based on a fascinating blend of Zero Trust ivory-tower idealism and legacy technology understanding which is completely disconnected from customers and the market at large. Nowhere in the assessment does Forrester take into consideration customers’ definition of modern threat detection and response that spans network, identity, and cloud.
The most glaring flaw is the fact that Forrester’s methodology puts more weight on integrated decryption than any other criterion, while completely ignoring any relevant aspect of ML/AI use or actual threat coverage. The result is that a pure signature-based IDS with decryption would significantly outscore the most advanced AI system without integrated decryption. (Maybe NAV actually stands for Network Anti-Virus?)
NAV is flawed thinking and the Forrester Wave flawed methodology when defining modern threat detection and assessing technology offerings, has the potential to lead customers badly astray. Case in point, when raising factual inaccuracies in their assessment, Forrester limited vendors to 5 objections, and any vendor misrepresentation beyond the 5 would not be discussed, nor corrected by the analyst. We highly encourage any prospective buyer to test vendors using a red team – the more advanced, the better – to see for themselves what really matters.
We could list numerous flaws in their methodology, but will focus on the three most glaring when it comes to modern threat detection:
- Flaw #1: Forrester believes all traffic inside of enterprises is encrypted
- Flaw #2: Forrester contends NAV needs to decrypt to find threats
- Flaw #3: Forrester thinks time-to-baseline is more important than threat coverage for ML/AI
Together, these flaws place a heavy operational burden on customers for little-to-no value while badly skewing the characterization of the NAV landscape towards legacy tech.
Flaw #1: Forrester believes all traffic inside of enterprises is encrypted
Forrester’s heavy focus on decryption is based on their view that all enterprises are or soon will be encrypting all traffic for Zero Trust alignment.
That’s simply not true. Vectra AI has monitored encryption rates across our installed base, which includes some of the most security-conscious enterprises in the world, for years. Only 1% of the protocols where payload visibility is required to detect modern threat techniques (DCERPC, Kerberos, SMB, and DNS) are encrypted and that is not increasing. Yes, 1%! Even adoption of SMBv3 has not meaningfully changed this.
Why? Customers are weighing the benefits vs. the downsides of using encryption on internal traffic and choosing not to encrypt. These are reasoned choices. Managing decryption is hard, even with integrated capabilities. It means loading every key from every internal server you want to decrypt traffic to, and then updating every time a key rotates.
As one of our enterprise customers interviewed by Forrester said, “The reality is that trying to decrypt every protocol and communication in a modern enterprise is naïve at best.” Or another, also interviewed by Forrester, said that even with integrated capabilities – decryption is “not worth the time, effort, and additional overhead”.
It seems that Zero Trust dogma is more important to Forrester than the customer perspective.
Flaw #2: Forrester contends NAV needs to decrypt to find threats
So, high-value protocols aren’t encrypted. What is encrypted is web traffic, both internal (about 60%) and external (90%+). Forrester contends that it’s critical to decrypt this to detect threats. Again, flat out wrong.
Modern threat detection technology works by understanding attack methods and movement of data using a mix of ML/AI and heuristics augmented by threat intel. Decrypting is neither necessary nor even helpful in this. Even if the traffic is decrypted, the attacker payload remains encrypted in a way that is not decryptable by the enterprise.
The only thing that you get for decryption is the ability to run IDS signatures. That comes with a high operational cost. Due to TLS 1.3 with PFS, passive decryption requires every single system to run an agent in order to forward session keys. Yet another agent, and the headaches that come with it just to run some IDS signatures!
This is why customers do decryption of outbound traffic at the SASE/proxy for content inspection and policy enforcement and inbound DMZ traffic in an inline NGFW for IDPS coverage of known exploits. NAV complements these by providing behavioral detection and threat intel to find modern and unknown threats.
Vectra AI accurately detects attacks in encrypted traffic—including C2, exfil, data staging, brute force, scans/sweeps, suspicious use of admin protocols, etc. or our customers would not have the value outcomes they have, nor the business outcomes we enjoy as a result. As one of our customers puts it, “You don't need to look inside to know that the car is driving erratically."
Flaw #3: Forrester thinks time-to-baseline is more important than threat coverage for ML/AI
Forrester’s obsession with decryption in service of Zero Trust completely distracts from the reality of what is most important in this market: getting clear threat signal that the SOC can understand with rapid time-to-value and low operational overhead. While there’s a lot of marketing BS around ML and AI that can make heads spin, no sane person believes a modern threat detection system can be built with just signatures. Great ML/AI is a must to keep pace with evolving and emerging attacker methods.
To the extent that ML/AI is considered, Forrester scores it entirely by how long it takes to baseline. Shorter baseline equals higher score. Again, this is a completely broken and misguided way to evaluate ML/AI. Consider the use of admin and service accounts, which are core to most modern attacks. These need to be learned over at least a 30-day time horizon to get accurate baselines, due to the nature of admin work. If you can only remember a couple of days — usually due to limitations in your system — the result will be noise and hassle for users.
Meanwhile, the Forrester methodology completely fails to take into account how well AI/ML models cover the various attack methods at the heart of modern attacks. Literally not a single question or scoring criteria in their methodology related to AI/ML attack detection. Mind boggling! Talk to a buyer or user and this is what they care about first and foremost. Vectra AI has 35 patents in this area, the most patents referenced in MITRE D3FEND countermeasures – more than Crowdstrike, Microsoft, and every other vendor in this Wave combined.
Our advice: Red Team Vectra AI.
Don’t just take our word for it, and please don’t take Forrester’s. The true test of modern threat detection effectiveness is to Red Team vendors, and this Forrester NAV “strong performer” will take on any of the “leaders” to prove our point.
To see why 4 in 5 customers choose Vectra AI over competitors (including those named in The Forrester Wave), visit vectra.ai/products/ndr.