The Future of Big Data in Security
Currently there are problems with traditional security infrastructure such as network level security measures including firewalls, IPS, WAF and endpoint security measures such as antivirus that are not able to detect current emerging threats. Since endpoint security is signature based, it lacks in APT (Advanced Persistent Threat), malware detection and mitigation.
Solutions like SIEM (Security Information and Event Management) for monitoring device logs and network traffic also lacks the tools to detect long-term planned attacks. Another big challenge is processing data from different sources and types (unstructured data).
Tools which can analyze intranet traffic are not yet developed or implemented to detect anomalous activity during attacks, especially in the case of lateral shifts. Attackers are able to use this to understand the network.
Traditional DLP (Data Loss Prevention) is also unable to prevent sophisticated bypass techniques of data leakage. Techniques like segregating critical data into multiple attachments or segregating it into multiple emails to a particular destination are ineffective against data leakage .
There are also no tools for monitoring Social Engineering or Physical Security. The data for analysis will be from different sources such as audio, video and sensors.
So the challenge is that we are lacking huge processing, correlating and analytics for more data. We need a security solution with the capability to predict, detect and prevent long term attacks as well as solutions capable of correlating unstructured data from different data sources.
All of these drawbacks from traditional security solutions can be overcome by implementing a Big Data platform. Big data analytics can help in predicting attacks, proactive detection of breaches and quick containment.
Features of a big data security platform
Integration – Manage all data on one platform.
Analytics – Perform deep analytics and store data.
Visualization – Bring big data to the users with tools such as dashboards and spreadsheets.
Development – Develop tools for engines and analytic applications with ease.
Workload Optimization – Improve upon efficient processing and storage.
Security and Governance – Protect sensitive data with suitable retention policies.
With these features, the future of security infrastructure can be enhanced with big data.
Developments in big data
Big Data in SIEM
A Big Data platform which features more correlation on more data can be used to predict and understand long term planned attacks. We can analyze real time logs with old logs ( 1 yr old or more) to understand such attacks.
Big Data integration with security and networking devices can provide insight like network health and link utilization management.
Big Data in Endpoint /Network Management
Big Data analytics on the endpoint/network infrastructure can detect anomalous activity in the network. The real time intranet activities can be correlated with the old reference traffic to detect any malicious activity within the network.
Compromised endpoints can be detected and isolated from the network before the infection spreads to other systems within the network.
Big Data in Data Loss Prevention (DLP)
Big Data in DLP can have wide multi-thread correlating engines to identify segregated critical data like data in different files and different formats to prevent data leakage.
DLP can correlate and monitor all the data sent or received from a particular IP/Email ID/AD ID to predict data leakage. Even planned data leakage can be prevented.
Big Data in Engineering/Physical Security solution
Tools with facial recognition can be implemented with Big Data as a platform.
These solutions can be integrated with an access card and facial recognition for analyzing and preventing tailgating and piggybacking.
The detection logic can compare the swipe of an access card and the person standing in front of the door. In case of tailgating/piggybacking, there will be a difference between the card swiped and the individual using it. This solution will involve more fine-tuning and can be an automated monitoring solution.