NEbraskaCERT 2005:
Security Information and Event Management (SIEM)
Matt Stevens Chief Technology Officer Network Intelligence Corporation 8-10-05
Security Information/Events = Logs
Logs are audit records generated by any software component running on your IT infrastructure Log records cover:
Normal activity Error conditions Configuration changes Policy changes User access to assets Incident alerts Unauthorized use of resources Non-privileged access to files User behavior patterns Clearing of sensitive data Access to audit trails
Logs provide feedback on the status of IT resources and all activity going through them
Example Logs
Sample Operating System Logs Windows2K Server
2005/05/17 12:59:12.387 EDT192.168.1.52%NICWIN-4Security_529_Security: Security,91350077,Tue May 17 12:58:43 2005 , 529,Security,NT AUTHORITY/SYSTEM,Failure Audit,WA1-MASTERFDC,Logon/Logoff ,,Logon Failure: Reason: Unknown user name or bad password User Name: PIQA Domain: Ntoss Logon Type: 3 Logon Process: NtLmSsp Authentication Package: NTLM Workstation Name: UpTime-HA 2005/05/17 12:59:29.793 EDT192.168.1.24%NICWIN-4Security_560_Security: Security,69561800,Tue May 17 12:58:29 2005 , 560,Security,NTOSS/ashtylla,Failure Audit,WA1-MAS90-DC,Object Access ,,Object Open: Object Server: SC Manager Object Type: SC_MANAGER OBJECT Object Name: ServicesActive New Handle ID: - Operation ID: {0,261811266} Process ID: 784 Primary User Name: C:\WINNT\system32\services.exe Primary Domain: WA1MAS90-DC$ Primary Logon ID: NTOSS Client User Name: (0x0,0x3E7) Client Domain: ashtylla Client Logon ID: NTOSS Accesses (0x0,0xF8EAAF4) Privileges READ_CONTROL Connect to service controller Enumerate services Query service database lock state
Traditional Interest in Event Logs
Point security solutions provide log messages about critical network events Main focus on firewalls and IDS/IPS devices Correlation of events from multiple security points reduces false positives
Insider Threat Study
Paper: Insider Threat Study: Computer System Sabotage in Critical Infrastructure Sectors Published by: U.S Secret Service and CERT Coordination Center/SEI Date: May 2005
From Section3 Detecting the attack: In general, 75% of the insiders were identified through manual procedures only, and 19% were identified using a combination of automated and manual procedures. The various mechanisms used to identify the perpetrators included
system logs (70%) insiders own source IP address (33%) phone records (28%) username (24%) auditing procedures (13%)
In those cases in which system logs were used to identify the insider as the perpetrator, the following logs were used
remote access logs (73%) file access logs (37%) system file change logs (37%) database/application logs (30%) email logs (13%)
But That Is Just The Beginning
Event log data is the single most
underutilized source of information within the organization.
Capacity Planning
Compute Resources Network Bandwidth
LAN & WAN
Disk space consumption
Servers Clients
Performance & Uptime
Where events happen When they occur Who is affected What sub-systems are involved Identify common elements
Legal & Human Resources
Accurate, detailed audit trail Enforce acceptable use policies
A report sitting on your chair Monday AM is a powerful deterrent to further abuse
Provides supporting evidence Can link human assets to IT assets
Incident Investigation & Forensics
A strong historical record is your best friend What seems benign today can turn out to be harmful tomorrow Logs can quickly narrow down the search Similar incidents become easier to resolve
Help Limit Corporate Liability
Determined abuse is hard to stop An effective policy that is actively monitored proves corporate responsibility Its hard to intimidate an event log
Detect & Prevent I.P. Theft
Makes spotting unusual patterns easier Proper resource access can be monitored An effective logging policy can serve as a strong deterrent to casual I.P. theft Supports efficient prosecution
Audit & Enforce Employee Productivity I.T. resources are expensive and budgets are tight Maintaining peak competitive stature is key to corporate entity survival 1% increase in information worker productivity can net nearly a 5% increase in corporate profits*
*Source: 2003 McKinsey report on global competition
Troubleshoot System and Network Problems The original reason for logging
Over 30% of the code in 1969 version of UNIX was dedicated to logging support*
Can be extended to support internal application development Logs tell the story that other debugging techniques miss
*Source: Ken Thompson and Dennis Ritchie, Bell Labs
Support Compliance Regulations
Applies to both Govt and industry regs All regulations are based upon similar principles:
Establish controls Monitor the controls Report on the trends and monitoring efforts
We all know todays list of regulations
An effective event log platform prepares you for tomorrow
Audit & Enforce IT Security Policy
Apply risk metrics to IT processes Finding breakdowns in IT security policy faster reduces IT risk Only effective way to validate point source security technologies
Event Log Data Creators
Web server activity VA Scan logs Windows logs Client & file server logs Wireless access logs Windows domain logins DHCP logs NAS Access Logs Switch logs Web cache & proxy logs Content management logs Router logs Firewall logs VPN logs
IDS/IDP logs
Linux, Unix, Windows OS logs
VLAN Access & Control logs Oracle Financial Logs Database Logs
Mainframe logs
Event Log Information Consumers
Finance Marketing
Sales
Customer Legal Service Human Resources Operations Engineering
Event Log Information
Mapping Consumers to Use Cases
Capacity Perform. Legal/HR Incidents Corp. Planning & Uptime Action Forensics Liability Customer Service Marketing I.P. Theft Regulatory Employee Trouble- I.T. Prod. Shooting Security Compliance
X X X X X X X X X X
X X X X X X X X
X X X X X X X X X X X X X X
Legal Sales Finance Human Resources Operations Engineering
X X X
X X X
X X X X
Many Consumers & Use Cases
Silos of Redundant Information Management
How to Avoid Silos?
Deploy an Enterprise-class SIEM Solution
Collect All the Data. Broad device support: network, security, infrastructure, & applications Agent-less, multi-protocol, non-normalized (no filtering) data capture 100% raw data capture Deep source device coverage. Not a subset of events, all of the known events into a Scalable Enterprise Platform Modular growth to expand with business initiatives Price/performance for enterprise-class deployments Efficient storage and personnel utilization that Provides Powerful Analysis for Compliance Violations and Security Threats Multiple views into the data Targeted reports for security, SOX, HIPAA, etc Correlation results between device types Baseline of workflows Detailed forensic analysis Guaranteed, real-time alert performance under load
Security Information and Event Management (SIEM)
Data Analysis
Data Management
Data Collection
Collection: Of Strategic Importance
It All Starts Here Goal: Capture 100% of the Data
But still be able to make use of it
Requirements:
Scalable system
Must be able to meet the accumulating collection rates
Wide device support
Analysis capabilities for many device types
No filtering or normalization of data
All data is important - normal activity included.
Robust data management tools
Raw data collection High data compression rates Encryption of stored data Authentication of stored data
Agents vs. Agent-less
Collection: Implementing the Strategy
Roll the device collection by types of devices or by departments Focus on the most critical assets first Turn on auditing features on your critical assets Leverage SIEM to transform the data into information that in turns drives knowledge
Collection: Source Device Protocols
Syslog Syslog-NG SNMP Windows event logging API CheckPoint LEA FTP Formatted log files
comma/tab/space delimited, other
ODBC connection to remote databases Push/pull XML files via HTTP Cisco IDS POP/RDEP/SDEE
Collection: Open Device Support
Architecture
should be open and permit in-field addition and updates of source device(s). Uses existing source device collection protocols Should not require changes to core product Treat devices as added content that can be distributed without interruption to production systems Automatically identify unknown events, yet still permit intelligent analysis later if required
Analysis: Real-time Correlation
Easy to Use GUI Rule-based Correlation Anomaly-based Correlation In-line Analysis Variable Decay-time Taxonomy-Driven Baselines 3D Asset Awareness Scalable Performance All the Data
Delivers Efficient and Effective Security Operations. Correlation Logic can be Managed Online or Offline. Advanced Boolean Logic Driven Correlation Enables Real-time Evaluation Against Corporate Policies Detects and Alerts on Variations from Automatically Computed Baselines of both Events and Alerts. REAL Real-Time Analysis with Consistent High Performance Alerting, Independent of Incoming EPS. Permits Sophisticated Correlation Logic to Detect and Alert on Multi-Vector Fast Attack & Low and Slow Auto Calculation of Normal for All Events, Devices and Alerts. 100% Mapped via Extensible Taxonomy Intelligent Alert Ranking Via Automatic Gathering Of Asset Vulnerability, Value, & Type. 500-300K Sustained EPS from up to 30K Source Devices with 100% Data Capture Provides Best TCO Accuracy Assured Regardless of Event Format Changes. Huge Device Breadth & Depth. No Agents.
Analysis: Vulnerability Data
VA tools provides a known list of hosts and detected vulnerabilities. Analysis can leverage this data to score threats based on asset vulnerabilities All rules evaluate vulnerability data of all the target assets. Higher vulnerability values of attacked assets escalate the severity level Customized rules should be able to evaluate individual assets or asset groups and alert when their vulnerabilities exceed a certain level
Analysis: Threat Scoring
Alerts are grouped into alert categories All alert categories have (5) alert severity levels that default to US Homeland Security levels Internal scoring algorithm automatically computes alert severity levels based upon event contents, rates, baselines and asset values Incorporates asset attributes, including frequency of asset in event payload, importance and vulnerability Automatic ranking of all alerts contained in a view to focus security administrators on most critical incidents first
Analysis: Event Classification
All events should be classified using a global taxonomy structure thus providing a standard to the myriad of non-standard log events from all source device vendors Leveraging taxonomy to evaluate events by category, regardless of source device permits correlation to stay current and relevant far easier Event classification should be fully exposed to the user. Users should be able to create new categories and assign new messages to any level in the taxonomy tree
Analysis: Baseline Data
Baselines should be created automatically learned from the actual network activity Minute, hour, day and week baselines permit tracking of spikes as well as low and slow patterns Baselines are aware of normal activity pattern changes over the course of the day, week and month. Correlation engines can use baseline data to detect anomalies based on activity percent change from normal behavior
Analysis: Baselines
So o c re ver tim e
10 0 9 0 8 0 9 0 7 0 6 0 5 0 4 0 3 0
61 55 60 57 58 54 57 54 59
8 0 S o V lu c re a e
7 0
6 0
2 0 1 0 0 -1 0
5 0
48
52
50
4 0 1 2 3 4 5 6 7 8 9 1 0 1 1 1 2 Tim (in M utes e in )
C urrent S core S core B aseline %D ifference
% D re c iffe n e
Analysis: Baselines
Score over time
100
98
90 80
90 70 60
78
80 Score Value
50 70
64 62 68 67 65
40 30
61 58 57 54 54 59
60
53
57 55
60
55 58
57
20 10 0 -10
50
48 48
52
50
40 1 2 3 4 5 6 7 8 9 10 11 12 Time (in Minutes)
Current Score Score Baseline % Difference
Analysis: Baselines
Score over time
100
98 81%
90
80 70
90
60
70
64 62
68 67 28% 61 55 58 57 57 19% 54 7% 3% -5% 10% 18% 54 59 65
40 30
58
60
53
57 55 14% 16%
60
20 10
50
48 48 0
52 2%
50
0 -10
40 1 2 3 4 5 6 7 8 9 10 11 12 Time (in Minutes) Current Score Score Baseline % Difference
% Difference
Score Value
Severity Levels Calculation
80
78
50
Analysis: Baselines
Score over time
100
98 81%
90
80 70
90
60
70
64 62
68 67 28% 61 55 58 57 57 19% 54 7% 3% -5% 10% 18% 54 59 65
40 30
58
60
53
57 55 14% 16%
60
20 10
50
48 48 0
52 2%
50
0 -10
40 1 2 3 4 5 < or = 10% Score Baseline = Low 6 >10% higher than Score Baseline = Guarded >25% higher than Score Baseline = Elevated >50% higher than Score Baseline = High >75% higher than Score Baseline = Severe 7 8 9 Time (in Minutes) Current Score Score Baseline % Difference 10 11 12
% Difference
Score Value
Severity Levels Calculation
80
78
50
Analysis: Correlation Example Worm Detection
Correlation Rule Name: W32.Blaster Worm The goal of this rule is to detect Blaster worm variants as well as other malicious code by analyzing network traffic patterns.
13
Analysis: Correlation Example Website Attack
Correlation Rule Name: SQL Injection Attack The goal of this rule is to detect information theft from E-Commerce websites through the exploitation of the trusted connection between the web server and the database.
14
Analysis: Real-time Threat Analysis
27
Analysis: Real-time Threat Analysis
28
Analysis: Reporting
User Activity from External Domains
Analysis: Reporting
Operational Change Control
Analysis: Reporting
Password Changes and Expirations
Analysis: Reporting
Top 20 Denied Inbound
31
Analysis: Reporting
Top 20 Firewall Categories (Taxonomy)
32
Analysis: Reporting
Scheduled Reports
30
Analysis: Real-time Event Viewer
Data Mining in Real-time or Historically
29
An Enterprise Platform for Compliance and Security
SIEM
Example SIEM Architecture
Patent-pending scaleable, distributed architecture for enterprises and global organizations. Local collection and storage of event data with true global analysis across multiple DBs. Leveraging local and remote collectors and a distributed DB, global organizations can collect and process over 300,000 EPS from up to 30,000 devices. Capture, analyze, and manage >26 Billion events per day per distributed DB.
ISO 17799: A Content Framework for IT Compliance
SIEM SIEM
16
Best Practices
Dont Try to filter the logs at the source Predicting what is useful or not is like playing Russian Roulette Its much easier to purge information you dont need vs. never having it Good event logging systems will capture 100% and let you purge later Determine Reporting Time Periods 1 week, 1 month, 90 days - more? Reporting Periods will drive event data retention policies. Plan to store data at least 2 complete reporting intervals If you purge old data be sure you have proper archives Archive Key Logs to Long Life Media CD-ROM, DVD-RW, etc Use a centralized, standard time source When event logs are time aligned life is much easier Be cautious of sensitive event log content Many logs are sent in the clear leverage a VPN for WANs Be sure that centralized logging facility is secure
Best Practices
Dont Alert on Everything Take it Slow Prioritize on what You REALLY want to be alerted on Leverage Correlation to Weed Out False Positives Rules-based correlation techniques can reduce the chatter Correlated reporting will let get a more holistic view of the network Test Your Logging Facility Are you REALLY capturing all the logs? REALLY? Encourage Your Teams to Analyze the Data Determine your standard reports develop baselines look for exceptions If You Didnt Log It, Then It Never Happened
