Enterprises are increasingly required to open and extend their network boundaries to suppliers, partners and customers to support innovative value chains and information collaboration. This scenario, along with more corporate applications being accessed over the cloud & mobile devices, makes firms vulnerable to sophisticated security threats. Big Data Analytics (BDA) applied to enterprise security promises to bring a new level of intelligence to network forensics and risk management. Information security will be more intelligence-driven, contextual and risk aware in real-time. Gaining insights into what big data is telling us about security threats is the hard part. Collecting the data is the easy part. BDA frameworks, along with reductions in infrastructure costs for data warehouses generate massive clusters of computers that can be managed efficiently, and with fewer people. These economics will disrupt traditional monitoring, SIEM (Security Incident & Event Manager), identity management and governance, risk & compliance (GRC) in the field. Contemporary SIEM devices do aggregation & correlation at roughly thousands of events per second. More sophisticated security management platforms that are BigData-enabled should be able to process millions of incident events per second with the same hardware footprint.
Historically you’ve had to do significant filtering, factoring and reduction of security data to get to a manageable size that allows security professionals to perform analysis and make decisions. Now, being able to mine petabytes of operational and security risk data from diverse data sources will provide actionable intelligence in real time. It is expected this “mining” can be done with industry standard 3rd party applications through open-source methods. BDA enables highly efficient batch processing to analyze historical data, find out when the attack started, how initial probing went undetected and how the attacker breached your system. The use of big data analytics in an enterprise security context provides for situational awareness, automates the threat detection processes, improves reaction times and will ultimately help with prevention. Watch for startups innovating in this space.