ACSE AlienVault Certified Security Engineer 1 Thursday, May 3, 12
Nov 14, 2015
ACSEAlienVault Certified Security Engineer
1
Thursday, May 3, 12
2Thursday, May 3, 12
About this document ACSE (AlienVault Certified Security Engineer)
Author: AlienVault Training Team ([email protected])
Document Version 1.0
Last revision: 01/2012
Product version used: 3.1
Copyright Alienvault 2012All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying,
recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and publisher.
Any trademarks referenced herein are the property of their respective holders
3
Thursday, May 3, 12
Contents Installation
Updates
CLI
Event Collection
Data Sources
Policies & Actions
Logical Correlation Directives
Logger
IDM
HIDS
Secure Connection
Snort
Dimensioning and Deployment
4
Thursday, May 3, 12
Bubba Throughout the document Bubba will give you useful hints and
links for further documentation
We come in peace and security!
5
Thursday, May 3, 12
AlienVault InstallationGetting to speed
6
Thursday, May 3, 12
Products AlienVault Installations
Appliances- Sensors (X1000, X2000, X3000, X4000)
- Loggers (L1000, L2000, L3000)
- SIEM (S1000, S2000, S3000)
Software- analog to the Appliances range, installable on custom hardware
custom restrictions, special purpose environments, etc.
Preferred: AlienVault Appliances- optimum performance and compatibility
7
Thursday, May 3, 12
Installation Guide Find the Installation Guide here
http://www.alienvault.com/docs/Installation_Guide.pdf
8
Thursday, May 3, 12
Hardware recommendations For a production system:
At least 4GB Ram 64 Processor DUAL Core Processor
Depending on the amount of traffic being monitored and the amount of data captured RAM has to be increased, always avoiding SWAP memory usage.
If we dont have the appropriate hardware:
"Divide et vinces"
9
Thursday, May 3, 12
Network hardware Requires Intel E1000 cards for capturing
performance performance performance
Administration interface can be any card with no known problems
10
Thursday, May 3, 12
Best practice Always use the latest installation image
If you need performance you cant use any hardware
disable unused data sources
for the time of installation and customization, try to stick to english for faster support
dont use Sniffers (ntop, snort, p0f) on interfaces without tap or port/mirror
Performance is crucial when sniffing is involved. Get rid of unneeded CPU hoggers!11
Thursday, May 3, 12
Best practice: VmWare VMware installations are popular but
minimal memory: 8GB minimal number of CPUS: 4 take care that all resources are bound singularly to the AlienVault
guest system
12
Thursday, May 3, 12
Best practice: Partitioning SIEM only
use 50-100 GB of disk space for /var/lib/mysql use the maximum available remaining space for /var
- REASON: /var/lib/mysql keeps all the configuration
Logger only
use at least twice the space required per log interval for /var- 50 GB expected logs/day => /var => 100GB
use the remaining rest for /var/ossim/logs- REASON: if /var runs out of space /var/ossim/logs remains intact
13
Thursday, May 3, 12
Best practice: Partitioning Use a small amount of space for the operating system
50-100 GB are absolutely sufficient be sure to configure enough swap space
golden rule: 2x amount of RAM additional swap can be configured later with a loopback mount
the storage killer: /var use at least 80% or more for this partition depending on the usage you may sub-partition /var further
14
Thursday, May 3, 12
Best practice: Partitioning Sensor only
50-100GB for / 2x physical memory (RAM) for swap remaining space: /var
Example: SIEM - S1000
15
path filesystem size partition
/boot ext2 2GB #1
/ ext3 100GB #2
swap swap 16GB #3
/var ext3 834GB #4
/var/lib/mysql ext3 100GB #5
Thursday, May 3, 12
Installation profiles Selecting the server role
Sensor Server (includes SIEM & Logger) Database Framework
can be changed during installation or anytime after installation
16
Thursday, May 3, 12
Profile: Sensor enables Sensor functionality
needs access to all the networks being monitored receives all the network traffic
Port Span- needs to be configured separately
Tap device Hub
17
Thursday, May 3, 12
Profile: Sensor out of the box enabled data sources
Snort (Network Intrusion Detection System)
Ntop (Network and usage Monitor)
OpenVAS (Vulnerability Scanning)
P0f (Passive operative system detection)
Pads (Passive Asset Detection System)
Arpwatch (Ethernet/Ip address parings monitor)
OSSEC (Host Intrusion Detection System)
Nagios (Availability Monitoring)
OCS (Inventory)
18
Thursday, May 3, 12
Profile: Server combines SIEM functionality
correlation risk assessment etc.
with Logger functionality
forensic long-term storage digitally signed
The server installation profile also comes with a Sensor with limited functionality to monitor the Server itself
19
Thursday, May 3, 12
Profile: Database enables a mysql server for usage with Server components
is required in most installations
SIEM only- event and alarm storage
- metadata
- framework
Logger- configuration
- metadata
- framework
20
Thursday, May 3, 12
Profile: Framework enables the Web front-end for the server
is installed on most appliances
low overhead useful in emergency situations
- power outage on central console
- connection to central console lost
- usage as a per department or per location console
21
Thursday, May 3, 12
Profile: All-In-One Enables all AlienVault components on a single appliance
Server (SIEM + Logger) Database (Configuration and event storage + correlation) Sensor Framework (Web-Interface)
Useful for:
Testing Evaluation Small deployments
activated on every automated install
22
Thursday, May 3, 12
Installation methodsAutomated installation
1. Boot the installation system
2. Configure networking
3. Create and mount the partitions on which AlienVault will be installed
4. Watch the automatic download/install/setup/update of the base system.
5. Set up users and passwords
6. Load the newly installed system for the first time
Custom installation
1. Boot the installation system
2. Select the installation language
3. Configure keyboard
4. Configure location
5. Select the installation AlienVault profiles for this installation
6. Configure networking
7. Create and mount the partitions on which AlienVault will be installed
8. Enter the professional license
9. Watch the automatic download/install/setup/update of the base system.
10.Set up users and passwords
23
Thursday, May 3, 12
Installation checklist Rack Space
Power
Network Configuration
Port mirroring IP addresses
Professional Key
Internet Access (Required when installing the professional version)
Role of the device to install
24
Thursday, May 3, 12
INSTALL
25
Thursday, May 3, 12
Installation: Next steps Point your web-browser to https://siem_ip/
username/password = admin Use ssh to login to the appliance
same password as in installation dialog
Be sure to note down the root password you enter in the installation process.26
Thursday, May 3, 12
Hands-On: Installation Fill in the following tables with the partitioning data for the given
profiles
Logger, 4TB of overall disk capacityLogger, 8TB of disk capacityLogger, 8TB of disk capacityLogger, 8TB of disk capacity
Mountpoint Capacity Comment
SIEM, 4 TB of disk capacitySIEM, 4 TB of disk capacitySIEM, 4 TB of disk capacity
Mountpoint Capacity Comment
27
Thursday, May 3, 12
Hands-On: Installation Given the following SIEM design, which profiles need to be
installed on which machines?
SIEM & ConsoleProfiles:__________________________________________________________________________________________________________________
Sensor:Profiles:__________________________________________________________________________________________________________________
Logger:Profiles:__________________________________________________________________________________________________________________
28
Thursday, May 3, 12
AlienVault UpdatesKeeping your system up to date
29
Thursday, May 3, 12
AlienVault: Update channels AlienVault uses the Advanced Packaging Tool (apt) for software
maintenance reliable
tested
Every AlienVault Appliance is configured to retrieve updates
Base System: Debian repositories AlienVault Software: AlienVault repositories
- Binary and package updates
/etc/apt/sources.list.d/alienvault-pro.list
- AlienVault professional feed
/etc/apt/sources.list.d/alienvault-etpro-pro.list
Get more info on how APT works here: http://en.wikipedia.org/wiki/Advanced_Packaging_Tool30
Thursday, May 3, 12
Updating Process Important files (They should never be modified)
/etc/apt/sources.list- Contains the different software repositories
/etc/apt/preferences- Contains the priority configuration between the different repositories
The update system in AlienVault:
Updates the AlienVault components as well as the Debian base system
Allows the AlienVault development team preventing software packages from being upgraded (Unstable versions, software causing troubles to other users.
31
Thursday, May 3, 12
Update AlienVault The system notifies in the Web interface the availability of new
versions of the AlienVault components
The system notifies the management console on the availability of new updates to the components of AlienVault
If the update procedure requires any manual change, it will be explained in the updates notification system
To update the whole system, use the following command:
# alienvault-update
debugging: alienvault-update -v
32
Thursday, May 3, 12
Update AlienVault During the update process:
When prompted always select installing the latest configuration files available
Once the system has been updated, if something is not working run:- # alienvault-reconfig
Snort rules, OpenVas scripts, directives and plugins will be updated automatically using software packages
33
Thursday, May 3, 12
Update AlienVault In case new Snort or OpenVas rules are included manually, you will
need to run the following scripts to update the information in the database
Snort:- # perl /usr/share/ossim/scripts/create_sidmap.pl /etc/snort/rules
OpenVas:- # perl /usr/share/ossim/scripts/update_nessus_ids.pl
And restart the AlienVault Server
# /etc/init.d/ossim-server restart
34
Thursday, May 3, 12
Package Management Following apts principle several tools are available to install
packages and monitor installed software
apt-cache- search for packages (apt-cache search )
- show package requirements, versions etc.
apt-get- install individual packages
dpkg- show installed packages
The Debian APT howtos:http://www.debian.org/doc/manuals/apt-howto/35
Thursday, May 3, 12
Hands-On: Updates Update the system
review the software sources in /etc/apt/sources.list.d
Which versions of the following software is installed:
ossim-server ___________________________ ossim-agent ___________________________ mysql-server-core ___________________________ ossim-framework ___________________________
36
Thursday, May 3, 12
AlienVault CLIpower to the user
37
Thursday, May 3, 12
AlienVault: Base System AlienVault Appliances are built around 64 bit Debian 5 Linux
Open Reliable Secure Innovative
AlienVault uses many packages from Debian
where possible AlienVault repositories
for AlienVault software packages customized Debian packages
38
Thursday, May 3, 12
AlienVault: Filesystem Current standard installation is based on ext3 filesystem
recommended filesystem for usage of the AlienVault software journaled stable
39
Thursday, May 3, 12
AlienVault: system access Get access to the platform
SSH access with superuser root- use the password supplied by AlienVault or the one from the installation
process
HTTPS access with administrative user admin- default password is admin
- can be switched to normal HTTP
40
Thursday, May 3, 12
Configuration Files AlienVault
OSSIM Server: /etc/ossim/server/config.xml OSSIM Agent: /etc/ossim/agent/config.cfg Frameworkd: /etc/ossim/framework/ossim.conf
Snort /etc/snort/snort.ethN.conf
OpenVas /etc/openvas/openvasd.conf
Nagios /etc/nagios3/
Database
/etc/mysql/my.cnf
41
Thursday, May 3, 12
Configuration Files System startup
/etc/rc* Logrotate
/etc/logrotate.d Network configuration
/etc/network/interfaces DNS configuration
/etc/resolv.conf Rsyslog
/etc/rsyslog.conf Monit
/etc/monit/monitrc
42
Thursday, May 3, 12
AlienVault: Services Stop a service
# /etc/init.d/ stop
# service stop
Start a service # /etc/init.d/ start
# service start
Restart a service # /etc/init.d/ restart
# service restart
The parameters that the services will use when starting are usually configured in the following path:
# /etc/default/ 43
Thursday, May 3, 12
alienvault-setup /alienvault-reconfig
ossim-reconfig
AlienVault Components
Database
Integrated Tools
OS Components
ossim_setup.conf
44
Thursday, May 3, 12
ossim-setup.conf interface: Network management interface (eth0, eth1...
language: Language used within AlienVault (en, es, fr...)
profile: Profile or Profiles enabled in the system
version: Version of AlienVault in use
hostname: Name of the system
admin_ip: IP address to manage the system
first_init: Variable to check whether is the first boot or not
email_notify: e-mail to receive notifications
45
Thursday, May 3, 12
alienvault-setup (Database) acl_db, event_db, ossim_db, osvdb_db, ocs_db: Name for the
different databases
db_ip: IP address of the Database
db_port: Listening port of the Database
pass: Password of the Database
type: Type of Database
user: User in the Database
create: If i is set to yes, database will be deleted and created again when running alienvault-reconfig
46
Thursday, May 3, 12
ossim-setup.conf (Sensor) detectors: Enabled detector plugins (Separated by comma and
using the same name that the plugin configuration file has)
interfaces: Listening interfaces (Separated by comma)
ip: IP address that the sensor will use to connect to the AlienVault Server
monitors: Enabled monitor plugins (Separated by comma and using the same name that the plugin configuration file has)
name: Name of the sensor
networks: Local networks that will be monitored from that sensor (In CIDR format and separated by comma)
47
Thursday, May 3, 12
ossim-setup.conf FRAMEWORK
framework_ip: IP address of the Web interface framework_port: Listening port of the frameworkd daemon
SERVER
server_ip: Listening IP address of the AlienVault Server server_port: Listening port of the AlienVault Server server_license: Professional license code server_plugins: Enabled plugins in the Server profile
48
Thursday, May 3, 12
ossim-setup.conf SNMP
snmpd: Enable the Snmp daemon snmptrap: Enable Snmp traps collection community: Snmp community
FIREWALL
active (firewall): Enable or disable iptables VPN
vpn_infrastructure: Enable or disable the VPN between the OSSIM components
vpn_net: VPN Network vpn_port: VPN Port
49
Thursday, May 3, 12
Agent Configuration /etc/ossim/agent/config.cfg [daemon]
daemon: Daemon mode (True or False)
pid: Path to the PID file (Process identifier)
[event-consolidation]
Enable events consolidation at Sensor level
by_plugin: List of plugins that will be consolidated
enable: Enable or disable (True or False)
time: Wait n seconds to consolidate the events before sending them
50
Thursday, May 3, 12
Agent Configuration /etc/ossim/agent/config.cfg [log]
Configures the verbose level and the path to the different log files error: File in which the error events will be stored file: File in which all the agent logs will be stored stats: File in which the agent stats will be stored (Every 5 minutes) verbose: Configures the verbose level (Debug, Info, Warning, Error or Critical)
[output-plain]
Writes in a log file what is being sent to the AlienVault Server (Useful for debugging and developing purposes)
enable: Enable or disable (True or False) file: File in which the output-plain will be stored
51
Thursday, May 3, 12
Agent Configuration /etc/ossim/agent/config.cfg [output-server]
Configures the server to which events are sent enable: Enable or disable sending events to the server (True or
False)
ip: IP address of the AlienVault Server (Logger or SIEM) port: Listening port of the AlienVault Server (Logger or SIEM)
[plugin-defaults]
In this category variables can be defined to be used in the plugins configuration.
52
Thursday, May 3, 12
Agent Configuration /etc/ossim/agent/config.cfg
[plugins]- Defines which Data Source Connectors (detectors and monitors) are
enabled
- name_of_the_plugin=path_to_the_plugin_config_file- device= /etc/ossim/agent/plugins/device.cfg
[watchdog]- Monitor the process associated to each plugin (In case it is running in
the same machine)
- enable: Enable or disable (True or False)- interval: Wait X seconds between checks- restart_interval: Restart the process every X seconds (This has to be
enabled in each plugin)
53
Thursday, May 3, 12
Agent Configuration /etc/ossim/agent/aliases.cfg
Contains predefined regular expressions that can be used when creating new plugins
Data Source Connectors configuration files
/etc/ossim/agent/plugins/*.cfg
54
Thursday, May 3, 12
Data Source Connectors Detector plugin configuration (/etc/ossim/agent/plugins/*.cfg)
[DEFAULT] Any var defined inside this category will be sent to the AlienVault
Server plugin_id: Numerical identifier of the plugin within the AllienVault
system (Data Source ID) [config]
type: detector enable: Enable or Disable plugin (It must be enabled in config.cfg) source: Source of the events (log, database, wmi) location: File in which logs can be found create_file: Create the log file in case it does not exist
55
Thursday, May 3, 12
Data Source Connectors Detector plugin configuration (/etc/ossim/agent/plugins/*.cfg)
[config]- process: Name of the process generating logs (If the process is
running in the same system)
- start: Start the process when the agent starts (yes/no)- stop: Stop the process when the agent stops (yes/no)- startup: Command that starts the process- shutdown: Command that stops the process
The next part of the configuration files includes the regular expressions that collect and normalize the events.
56
Thursday, May 3, 12
Agent Configuration The different configuration variables defined in the config file can be used with the
following syntax to help defining new variables:
%()s
When the variable has been defined in the same file: process=pads
shutdown=killall -9 %(process)s
\_CFG()
When the variable has been defined in the main configuration file(config.cfg)
In /etc/ossim/agent/config.cfg file:restart_interval=3600 ; seconds between plugin process restart
In the Data Source Connector configuration file:restart_interval=\_CFG(watchdog,restart_interval)
57
Thursday, May 3, 12
Server Configuration /etc/ossim/server/config.xml
Path to the AlienVault Server log file
Configuration to access the SQL Database
Configuration to access the Snort Database
Configuration to access the OSVDB Database
58
Thursday, May 3, 12
Server Configuration /etc/ossim/server/config.xml
Path to the correlation directives
Waiting time between each execution of AlienVault Server
scheduled jobs
Listening port, name and listening IP address of the AlienVault
Server
59
Thursday, May 3, 12
Web Interface Configuration Executive panel configuration
/etc/ossim/framework/panel/
/etc/ossim/framework/ossim.conf
Paths to applications and libraries Configure access to the different databases Some tools use this file to get the configuration parameters to
access the database (ossim-db, create_sidmap.pl...)
60
Thursday, May 3, 12
Monit Monit
Process monitors all the important services in the AlienVault machines and restarts services in case of a process crash
Configuration file /etc/monit/monitrc Different configuration based on the profile in use When Stopping any process monit must be stopped first
# Framework check process ossim-framework with pidfile /var/run/ossim-framework.pid group framework start program = "/etc/init.d/ossim-framework start" stop program = "/etc/init.d/ossim-framework stop" if 5 restarts within 5 cycles then timeout
61
When debugging, be sure to turn monit OFF!More Monit infos:http://mmonit.com/monit/
Thursday, May 3, 12
AlienVault: System logging All the AlienVault components offer logging
This can and should be used for
Debugging- errors may not be seen directly in the Web interface but can be spotted in the system
logs
- extra information for the AlienVault Support Team
Diagnosis- are all the services up to date
- did system update generate errors
Verification of Success or Failures- is my data source connector generating events
- are idm-events collected
62
Thursday, May 3, 12
AlienVault: Log files OSSIM Server
/var/log/ossim/server.log
OSSIM Agent
/var/log/ossim/agent.log
OSSIM Frameworkd
/var/log/ossim/frameworkd.log
Snort
/var/log/syslog
/var/log/snort (Binary Format)
Other applications
Check the variable location in the plugin configuration file
63
Thursday, May 3, 12
Debug Mode OSSIM Server
# ossim-server D 6 -d Logfiles in /var/log/ossim/server.log
This does not show information on the terminal, much more info will be logged in the server log file
OSSIM Agent
# ossim-agent vv OSSIM Frameworkd
# ossim-framework vv Never leave an application running in Debug mode in a production
64
Thursday, May 3, 12
Networks Card Information ethtool/mii-tool
Network card stats and link status set link speed and type for not autonegoiating switch ports
iptraf measure throughput and TCP sessions on a network interface
tcpdump Check whether the port mirroring is well configured or not do configured devices really send syslog to the Sensor
65
Thursday, May 3, 12
Network Configuration Rename network interfaces
# apt-get install ifrename
Edit the file /etc/iftab
Insert a line for each network interface with the following format :
eth0 mac 00:17:31:56:BC:2D eth1 mac 00:16:3E:2F:0E:9C
Network cards with more than one interface usually have consecutives MAC addresses
# ifconfig -a | grep HWaddr
66
Thursday, May 3, 12
Network Configuration Additionally rename it with udev
# ifconfig -a |grep HWaddr
Create the file /etc/udev/rules.d/010_netinterfaces.rules
Reboot & check udev entries
# udevinfo -a -p /sys/class/net/eth0
67
Thursday, May 3, 12
Network Configuration The network configuration in Debian is stored in the following file:
/etc/network/interfaces
After modifying the previous file, it is required restarting networking with the following command:
# /etc/init.d/networking restart
68
Thursday, May 3, 12
Network Configuration Each interface is configured in /etc/network/interfaces with the
following template
be sure that the interface is also in the auto section to enable automatic startup on system boot
auto lo0 eth0 ... ethallow-hotplug eth0iface eth0 inet static address 192.168.1.133 netmask 255.255.0.0 network 192.168.0.0 broadcast 192.168.255.255 gateway 192.168.1.1 dns-nameservers 192.168.1.100
69
Thursday, May 3, 12
Network Configuration address: IP address given to the interface (In the example eth0).
netmask: Network Mask
network: It is the part of the IP address which is common between all the IP addresses in the network.
broadcast: Broadcast IP of the network.
gateway: IP address of gateway in our network
dns-nameservers: IP addresses of the DNS servers used in our corporation. More than one DNS server can be used (Separated by comma). In there is a local DNS running in your network, t should be placed in first place.
70
Thursday, May 3, 12
Network Configuration Those interfaces in promiscuous mode used to collect the network
traffic (Port mirroring) should have an entry in the network configuration file with the following format:
up ifconfig eth0 0.0.0.0 promisc -arp
71
Thursday, May 3, 12
Network Recommendations The collector does a lot of queries to the DNS server to normalize
events, so, the local DNS should always be configured in any AlienVault box.
In case there is not a local DNS in your network, the different hostnames and their associated IP addresses should be defined in the the /etc/hosts file in your collectors.
Interfaces in promiscuous mode should only be used to collect network traffic, those interfaces should never have an assigned IP address
72
Thursday, May 3, 12
Disk Space The disk space left in the AlienVault machines should be monitored
The folder /var/ will use the biggest amount of disk space in AlienVault
Databases Log files
The /var/ folder should be separated into another partition with the highest amount of disk available (>80%).
73
Thursday, May 3, 12
Swap Memory Swap memory usage slows down the system so we need to make
sure the system is not using frequently the swap partition
If the system is always using the Swap partition it is required to increase the amount of RAM memory installed in the system.
74
Thursday, May 3, 12
Munin Munin can monitor the status of various parameters within the
operative system such as Interrupts, Load, Memory, network, and processes
Munin is really useful to monitor that our hardware is working properly
Munin can be used distributed since 3.1
75
Thursday, May 3, 12
AlienVault: system recovery Backup & Recovery
simple backup script is provided easy recovery after new installation
Backup script: see next page
76
Thursday, May 3, 12
AlienVault: system recovery Backup script: (Caution!!! - /var/ossim/logs is not included)
#!/bin/bashcd /etc/init.d/
process="monit ossim-agent ossim-server ossim-framework apache2 arpwatch exim4 fprobe nagios3 nessus dnfdump nfsen ntop munin-node openvas-scanner openvassd openvas-server osirisd osirismd pads rsyslog postfix samba snort* snmpd tomcat nfdump"
for i in $processdo /etc/init.d/$i stop > /dev/null;done
chmod 000 /etc/cron.hourly/*chmod 000 /etc/cron.daily/*
d1r="/var/ossim/backup/db/`date +%F-%H_%M_%S`"dbs=`echo "show databases" | ossim-db | grep -v "Database" | grep -v "information_schema"`p4ss=`grep -i pass= /etc/ossim/ossim_setup.conf |awk -F'=' '{print$2}'`h0st=`grep -i db_ip /etc/ossim/ossim_setup.conf |awk -F'=' '{print$2}'`test -z $h0st && h0st="localhost"
for db in $dbs; do test -d $d1r/$db/struct || mkdir -p $d1r/$db/struct mysqldump -d -u root -h $h0st -p$p4ss $db > $d1r/$db/struct/$db-struct-`date +%F-%H_%M_%S`.sql mysqldump -u root -h $h0st -p$p4ss $db > $d1r/$db/$db-`date +%F-%H_%M_%S`.sqldone
tar --preserve -czvf $d1r/../../complete_backup_`date +%F-%H_%M`.tgz $d1r > /dev/nullecho "Generated /var/ossim/backup/complete_backup_`date +%F-%H_%M`.tgz file"
tar --preserve -czvf $d1r/../../config_files_`date +%F-%H_%M`.tgz /etc/* > /dev/nullecho "Generated /var/ossim/backup/config_files_`date +%F-%H_%M`.tgz file"
dpkg -l >> $d1r/../../packages_list_`date +%F-%H_%M`
chmod 755 /etc/cron.hourly/*chmod 755 /etc/cron.daily/*
alienvault-reconfig -c -v
exit 0
77
Thursday, May 3, 12
Hands-On: CLI alienvault-setup
change default admin email address configure monitored networks enable detector plugins: foo bar and baz change the time zone, but if ethx to nonpromiscous mode change the system name
delete and rebuild the complete alienvault-database
WARNING: all your data and configuration will be LOST
78
Thursday, May 3, 12
AlienVault Event Collection
Get the data from the network
79
Thursday, May 3, 12
Syslog Using Syslog is usually the easiest way to forward events to the
AlienVault Sensor
All Linux, BSD and MacOSX include different Syslog implementations by default.
It's simple to configure event forwarding policies using Syslog
A large number of devices and applications allow event logging using the Syslog protocol. If it doesnt the log files can be logged into Syslog using logger:
# tail f /path/to/file | logger t application
80
Thursday, May 3, 12
Snare Agent Snare forwards Windows EventLog events to a remote Syslog
server.
When installing Snare, it can also configure a logging policy in the Windows System. Take care if you have already defined a logging policy.
Once Snare has been installed (Configuration -> Collection -> Downloads) download the .reg file to configure it to forward events to the remote Syslog server
81
Thursday, May 3, 12
WMI WMI (Windows Management Instrumentation) provides an
operating system interface through which instrumented components provide information and notification.
WMI allows scripting languages like VBScript or Windows PowerShell to manage Microsoft Windows personal computers and servers, both locally and remotely.
WMI is preinstalled in Windows 2000 and newer OSs. It is available as a download for Windows 95 and Windows 98.
AlienVault includes two data source connectors that allow collecting information using WMI:
Detector: wmi-system-logger.cfg Monitor: wmi-monitor.cfg
82
Thursday, May 3, 12
Fw1-Loggraber Fw1-Loggrabber allows collecting events from Checkpoint FW-1
devices using the Checkpoints LEA (Log Export Api) protocol
Fw1-Loggraber has to be installed along with the detector. It will stablish a connection to get the event from the Checkpoint FW-1 device
83
Thursday, May 3, 12
Cisco SDEE The AlienVault Sensor can collect events from Cisco devices using
the SDEE protocol. The detector allows collecting event from the following devices:
Cisco Network Detection Systems (IPS) Cisco Switch IDS Cisco IOS routers with Inline Intrusion Prevention
System (IPS) functions Cisco IDS modules for routers Cisco PIX Firewalls Cisco Catalyst 6500 Series firewall services modules (FWSMs) Cisco Management Center for Cisco security agents CiscoWorks
Monitoring Center for Security servers
Data Source Connector configuration file: cisco-ips.cfg
84
Thursday, May 3, 12
Rsyslog Rsyslog is the Syslog implementation shipped with AlienVault
Rsyslog is extremely configurable and allows configuring filtering and forwarding in a really easy way
Rsyslog must allow remote connections to collect logs coming from other Syslog servers. This feature has to be enabled in the Rsyslog configuration file (/etc/rsyslog.conf) including the following lines:
$ModLoad imudp
$UDPServerRun 514
$ModLoad imtcp
$InputTCPServerRun 514
85
Thursday, May 3, 12
rsyslog filters Filter using Rsyslog ( /etc/rsyslog.d/)
Forward certain events to a local file
if $msg contains 'error' then /var/log/error if $syslogfacility-text == 'local0' and $msg startswith 'DEVNAME'
and ($msg contains 'error1' or $msg contains 'error0') then /var/log/somelog
Stop processing some events
if $msg contains 'error' then ~ Regex in Rsyslog
http://www.rsyslog.com/user-regex.php
86
Thursday, May 3, 12
rsyslog customization Separate incoming logs (syslog)
find a phrase in syslog to classify the logs (hostname, ip-address,...)
Send logs to a different logfile Create a file with file extension conf in /etc/rsyslog.d
e.g. /etc/rsyslog.d/customize.conf
possible commands# sends logs with to cisco.log:source, isequal, "HOSTNAME" /var/log/cisco.log& ~if $msg contains STRING then /var/log/xyz.logif $msg contains STRING ~
# ip-address examplesif $fromhost == 'IP_ADDRESS' then -/var/log/ossim/device.log&~if $fromhost-ip isequal 'IP_ADDRESS' then -/var/log/cisco-fw.log& ~:fromhost-ip, isequal, "IP_ADDRESS" -/var/log/cisco-fw.log& ~
The rsyslog webpage provides more useful examples, tricks and howtos:http://www.rsyslog.com/87
Thursday, May 3, 12
Log rotation When creating new log forwarding rules (Rsyslog), it is important to
ensure that the logs will not grow indefinitely
To do this we must create new entries in the logrotate configuration
/etc/logrotate.d/
/var/log/ossim/agent.log /var/log/ossim/agent-plain.log /var/log/ossim/agent_error.log /var/log/ossim/agent_stats.log { daily firstaction test -f /var/log/snort/alert || touch /var/log/snort/alert > /dev/null 2>&1 endscript prerotate /etc/init.d/ossim-agent stop > /dev/null 2>&1 endscript postrotate /etc/init.d/ossim-agent start > /dev/null 2>&1 endscript}
88
Thursday, May 3, 12
Log rotation example If you have new separated logfiles just take this example and add
your new logfiles to it!
/etc/logrotate.d/alienvault
/var/log/xyz.log/var/log/foo.log...{ rotate 7 # Save the last 7 logs daily # rotate daily missingok # if file doesnt exist continue notifempty # if log is empty, the log dont rotate delaycompress # postpone compression of previous log-file to next cycle compress # Compress the log postrotate
invoke-rc.d rsyslog reload > /dev/null
}
89see also > man logrotate
Thursday, May 3, 12
Hands-On: CLI Define a syslog source and filter to /var/log/foo...
enable logrotation on the source you just configured
alienvault-setup
change default admin email address configure monitored networks enable detector plugins: foo bar and baz change the time zone change the system name
delete and rebuild the complete alienvault-database
WARNING: all your data and configuration will be LOST
90
Thursday, May 3, 12
Hands-On: CLI Logfiles
send some example logs to /var/log/syslog- use the following script:
filter the log and send it to a separated log file and be sure that this log is not filling up your disk space (rotate the file daily with compression enabled)
while truedo
cat /var/log/firewall.log | logger -t sleep 10done
91
Thursday, May 3, 12
AlienVault Data SourcesAdapt collection to your organization
92
Thursday, May 3, 12
Types of DS Connectors Two types of Data Source Connectors
Detectors: They offer events (Snort, Firewalls, Antivirus, Web servers, OS events..)
Monitors: They offer indicators (Ntop, Tcptrack, Nmap, Webs, Compromise & Attack...)
93
Thursday, May 3, 12
Files Each DS Connector (monitors and detectors) is built on two files:
Plugin.cfgContains the configuration parameters of the plugins and the rules that an event has to match in order to be collected and normalized.
Plugin.sqlContains the description of every possible event that can be collected using the plugin (Plugin_id, Plugin_sid, Name given to the event, priority and reliability)
94
Thursday, May 3, 12
[DEFAULT]plugin_id=4003
# default values for dst_ip and dst_port# they can be overwritten in each ruledst_ip=\_CFG(plugin-defaults,sensor)dst_port=22
[config]type=detectorenable=yes
source=loglocation=/var/log/auth.log
create_file=false
process=sshdstart=nostop=nostartup=/etc/init.d/ssh startshutdown=/etc/init.d/ssh stop
[ssh - Failed password]# Feb 8 10:09:06 golgotha sshd[24472]: Failed password for dgil from 192.168.6.69 port 33992 ssh2event_type=eventregexp="(\SYSLOG_DATE)\s+(?P[^\s]*).*?ssh.*?Failed password for (?P\S+)\s+from\s+.*?(?P\IPV4).*?port\s+(?P\PORT)"plugin_sid=1sensor={resolv($sensor)}date={normalize_date($1)}src_ip={$src}dst_ip={resolv($sensor)}src_port={$sport}username={$user}
Numerical identifier of the plugin
Type of plugin: Detector
Source of the events (log, mssql,mysql or wmi)
Associated process and start/stop options
Type of event
Regular expressions
Fields that will be sent to the AlienVault Server
Default fields for every event
95
Ds Connector: Detector
Thursday, May 3, 12
Ds Connector: Detector plugin_id
Data Source ID. User reserved range: 9000-10000 E.g.: plugin_id=3000
source log: Text file (E.g: SSH, Sudo, Apache...) mssql: Mssql Database (E.g: panda-se) mysql: Mysql Database (E.g: moodle) wmi: Windows Management Instrumentation (wmi-system-logger)
96
Thursday, May 3, 12
Ds Connector: Detector location
- Files in which the applications store the events
- E.g.: location=/var/log/file.log
create_file- Create the file in case it does not exist
- false/true
process / start / stop / startup / shutdown- Only if the process is running in the same machine that the detector
- If the process is not running in the machine, is there a process helping us to collect those logs? syslog? fw1-loggrabber?
97
Thursday, May 3, 12
Ds Connector: Detector Rules
Rules define the format of each event and how they are normalized It is composed by a regular expression and the list of fields that the
event will include when once it is sent to the AlienVault SIEM or Logger
In some cases only one regular expression will collect every event coming from one application, in some other cases more than one rule will be required
98
Thursday, May 3, 12
DS Connector: Detector Rules
Rules are loading in alphabetical order based on the name given to each rule
Once the log matches one the regex of one rule the ossim agent stops processing the event
Generic rules must be the last loaded in memory as they will probably match all the events
The name of the rule is mandatory
99
Thursday, May 3, 12
DS Connector: Detector The rule must include the event type:
event_type=event
The following fields can be used to normalize the event:
Values in bold are mandatory
Fields in red include values that always have to be defined in the plugin
Fields in green can will be filled by the AlienVault Agent in case they can not be found in the original log (Dont include that line when creating the plugin)
Fields in grey are optional
plugin_id plugin_sid date sensor interface protocol
src_ip src_port dst_ip dst_port username password
filename userdata1 userdata2 userdata3 userdata4 userdata5
userdata6 userdata7 userdata8 userdata9
100
Thursday, May 3, 12
DS Connector: Detector Regexp
The regexp field contains the regular expression that defines the format of the events, and extracts the information to normalize the event.
The regular expressions are written using the Python regular expression syntax:
http://docs.python.org/library/re.html
regexp="(\SYSLOG_DATE)\s+(?P[^\s]*).*?ssh.*?Failed password for (?P\S+)\s+.*?(?P\IPV4).*?port\s+(?P\PORT)"
regexp=(\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d)\S+ (\S+) (\S+) (\S+) (\d+) (\w+) (\S+) \S+ (\d+)
101
Thursday, May 3, 12
Operator Meaningc A non special character matches with itself
\c Removes the special meaning of the character c; The RE \$ matches with $^ Indicates located at the beginning of the line$ Indicates located at the end of the line. Any individual character
[] One or any of the characters ; accepts intervals of the type a-z, 0-9, A-Z[^] A char different from ; Accepts intervals of the type a-z, 0-9, A-Z
102
Regular expressions
Useful tools for testing regular expressions:Online: http://www.regexpal.comWindows: RegEx TesterLinux: http://kodos.sf.net/Regexhibit (OSX): http://homepage.mac.com/roger_jolly/software/
Thursday, May 3, 12
Regular expression Matches with
a.b axb aab abb aSb a#b ...
a..b axxb aaab abbb a4$b ...
[abc] a b c (one character srtings)
[aA] a A (one character srtings)
[aA][bB] ab Ab aB AB (two character srtings)
[0123456789] 0 1 2 3 4 5 6 7 8 9
[0-9] 0 1 2 3 4 5 6 7 8 9
[A-Za-z] A B C ... Z a b c ... Z
[0-9][0-9][0-9] 000 001 .. 009 010 .. 019 100 .. 999
[0-9]* empty_chain 0 1 9 00 99 123 456 999 9999 ...
[0-9][0-9]* 0 1 9 00 99 123 456 999 9999 99999 99999999 ...
^.*$ A full line
103
Regular expressions
Thursday, May 3, 12
Operator Meaningr* 0 or more occurrences of the RE rr+ 1 or more occurrences of the RE rr? 0 or an occurrences of the RE r, and no more
r{n} No occurrences of the RE rr{,m} 0 or at most m occurrences of the RE rr{n,m} N or more occurrences of the RE r, but at most mr1|r2 The RE r1 or the RE r2
Regular expression Matches with
[0-9]+ 0 1 9 00 99 123 456 999 9999 99999 99999999 ..[0-9]? empty_string 0 1 2 .. 9(ab)* empty_string ab ababab abababababab
([0-9]+ab)* empty_string 1234ab 9ab9ab9ab 9876543210ab 99ab99ab ...
104
Regular expressions
Thursday, May 3, 12
Regular expression Matches with Equals
\d Any decimal character [0-9]\D Any non decimal character [^0-9]\s Any space character [ \t\n\r\f\v]\S Any non space character [^ \t\n\r\f\v]
\w Any alphanumeric characterand _ [a-zA-Z0-9_]
\W Any non alphanumeric character [^a-zA-Z0-9_]\Z End of line
105
Regular expressions
Thursday, May 3, 12
Pattern Descriptionb,c,X,8 Ordinary characters just match themselves exactly. The meta-characters which do not match themselves because they
have special meanings are: . ^ $ * + ? { [ ] \ | ( )
. Matches any single character except newline (\n).\w Lowercase w matches a "word" character: a letter or digit or under-bar [a-zA-Z0-9_]. It only matches a single word char,
not a whole word.
\W Uppercase w matches any non-word character.\s Lowercase s matches a single whitespace character -- space, newline, return, tab, form [ \n\r\t\f].\S Upper case s matches any non-whitespace character.\d Lowercase d matches a single Decimal digit [0-9]\D Uppercase d matches any non decimal character\t Matches a tab character\n Matches a newline character\r Matches a return character\Z Matches only at the end of the string.\ Escapes special characters. If you are unsure if a character has special meaning, such as '@', you can put a slash in front
of it, \@, to make sure it is treated just as a character.
106
Regular expressions
Thursday, May 3, 12
Regex aliases /etc/ossim/agent/aliases.cfg
/etc/ossim/agent/aliases.local (For user custom aliases)
This file contains predefined regular expressions that can be used to simplify the process of writing new plugins
Usage Example:
\SYSLOG_DATE\s+\IPV4\s+\IPV4
107
Thursday, May 3, 12
Regular Expressions The information extracted by the regular expression from the log
can be accessed by:
Position: (\d\d):(\d\d):(\d\d)- hour={$1}
- minutes ={$2}
- seconds={$3}
Tags: (?P\d\d):(?P\d\d)(?P\d\d)- hour={$hour}
- minutes ={$minutes}
- seconds={$seconds}
108
Thursday, May 3, 12
Functions The AlienVault SIEM and Logger must receive normalized events, as an
example the addresses have to use IPV4 format and the date has to use the following format YYYY-MM-DD HH:MM:SS (2010-12-31 22:57:00)
To simplify the process of normalizing events some functions can be used
resolv()
Translate hostnames into IPV4 addresess (DNS queries)
normalize_date()
The normalize_date function translate many format dates into the format accepted by the SIEM or Logger
- YYYY-MM-DD hh:mm:ss
More functions can be found and defined in ParserUtils.py
109
A very useful tool for testing plugin is located in /usr/share/ossim/scripts/regexp.py
Thursday, May 3, 12
Translation Tables Translations can be configured to be done once the event has
been collected
E.g.: When the event id is not numeric, but plugin_sid has to be numeric
Translations have to be defined inside a category called [translation]
Translate using the function translate().
110
Even more info can be found here:http://www.alienvault.com/docs/AlienVault%20Building%20Collector%20Plugins.pdf
Thursday, May 3, 12
Testing your plugin Useful tool: regexp.py
will analyze a set of logs to a set of regular expressions or a plugin will give you statistics on matched lines and rules
111
# /usr/share/ossim/scripts/regexp.py /var/log/netscreen.log netscreen-firewall.cfg qMultiple regexp mode used, parsing netscreen-firewall.cfg -----------------------------------------------------------------------------Rule: netscreen-firewall-BA-traffic Matched 5586 timesRule: netscreen-firewall-BB-traffic Matched 0 timesRule: netscreen-firewall-EF-system Matched 0 timesRule: netscreen-firewall-ZZZ-generic Matched 50 timesCounted 5636 lines.Matched 5636 lines.
Thursday, May 3, 12
Hands-On: plugins
firewall logs are sent to /var/log/firewall.log
while truedo
cat /var/log/firewall.log | logger -t sleep 10done
112
Thursday, May 3, 12
Hands-On: plugins write a firewall plugin
copy existing similar plugin: plugin_id number start on custom range write your regex rule to match the loglines
write a firewall sql file
copy existing similar sql file: change the fields to your custom plugin rules
activate your plugin on the CLI (alienvault-setup)
write your sql file to the database
113
Thursday, May 3, 12
AlienVault Policies & Actions
Controlling flow of information between AlienVault Components
114
Thursday, May 3, 12
Event Filtering Levels Policy rules
AlienVault Sensor Configuration
Data Source Connector Configuration
Collection method used
Data Source Configuration
115
Thursday, May 3, 12
SIEM & Logger
Database storage (MySQL) The number of events that can be stored will depend on the
hardware in use As we increase the number of stored events, the analysis gets
slower Max number of stored events: 20 millions Certain events don't offer useful information for correlation
and analysis Need to filter useless events
SIEM
Disk storage No more storage limits apart from the available disk space Everything should be stored in the Logger without
exceptionLogger
116
Thursday, May 3, 12
Filter examples Filter all events generated by a tool
Filter certain events from a tool
Filter all events from one host or network
Filter events generated between two different hosts
Filter some events during the night
Filter some events every weekend
Filter all events from one service
117
Thursday, May 3, 12
Filter in Policy 1. Create a DS group with the types of events that have to be filtered
2. Select source and destination Assets (Hosts, Networks, ANY...)
3. Select the ports
4. Select the DS group
5. Select time period in which the policy applies
6. Select policy consequences
1. Process in the SIEM?2. Process in the Logger?3. Store the event?4. Correlate only?
118
Thursday, May 3, 12
Agent Configuration If we dont want to collect any event from one of the plugins
remove the plugin from the file:
/etc/ossim/ossim_setup.conf
in the variable named:
detectors
And run the following command:
# alienvault-reconfig
119
Thursday, May 3, 12
Filtering in DS Connectors Some events can be filtered during the collection process editing
the configuration file for each plugin:
Using the option exclude_sids (E.g.: Apache DS Connector)
exclude_sids=404,200,403
Modifying the regular expressions to avoid matching certain events
120
Thursday, May 3, 12
Hands-on: Policies Write a policy for a Nagios event / service down
use external command and write to /tmp/alert disable ssh on the device in question does your external command write to /tmp/alert
Send email after a prioritized event
Ticket creation (e.g. if only one user is allowed/responsible to subscribe new tickets)
121
Thursday, May 3, 12
Logical Correlation Directives
Put intelligence into your SIEM
122
Thursday, May 3, 12
Logical Correlation Directives Logical Correlation uses Correlation directives to detect attacks
and problems in the monitored networks
By default AlienVault includes a set of Correlation directives
Users can create custom correlation rules in both Unified and Open Source edition
AlienVault Professional Feed provides more than 500 extra correlation rules
123
Thursday, May 3, 12
Logical Correlation Directives Logical Correlation is implemented in the AlienVault SIEM
Correlation Directives are stored in the following path
/etc/ossim/server/ Web-based Correlation Directives Editor
Intelligence -> Correlation Directives
124
Thursday, May 3, 12
Rules Directives are composed by rules
Each rules defines the conditions that event must match to be correlated by the directive
Whenever the conditions defined by a rule are matched, a new event is generated
Some of the values of the rule will be used to generate the new event (In the event of a successful correlation of that rule)
It is possible to have multiple rules in each correlation level (Except in the first correlation level)
125
Thursday, May 3, 12
Example: Brute Force
Root Login RefusedInvalid Password
Invalid UserUser not allowed
User in DenyUsersIllegal User
Brute-Force Attack against SSH Server
SIEM SENSOR
Normalized Events
ALARM: Brute-Force Attack Against 192.168.1.1Total events captured 12.392
First Event 10-02-2011 23:30:23Last event 10-02-2011 23:56:19
Login AttemptLogin AttemptLogin AttemptLogin Attempt
126
Thursday, May 3, 12
Example: Brute-Force Think about all possible events that may be generated
Root Login not allowed (SSH Server Configuration)
Event generated: Root Login Refused
Root Login allowed but incorrect password
Event generated: Invalid Password
Root account locked
Event generated: User not allowed because account is locked
User: root Password: root
127
Thursday, May 3, 12
Example: Brute-Force Global properties of the correlation directive
Name of the directive- This is the name that will take all events generated within the correlation
of this directive
ID of the directive - All events generated within this directive will use 1505 as plugin_id
(Data Source ID) and the ID of the directive as plugin_sid (Event type) 500000-100000 is the reserver range for user-created directives
Priority of the directive (Impact of this Attack or problem in your network)
- All events generated within this directive will have as their priority the global priority value of the correlation directive
128
Thursday, May 3, 12
Example: Brute-Force Global properties of the directive:
plugin_id=1505 plugin_sid=500000 name=SSH Brute Force Attack Against 192.168.2.2 priority=4
Same plugin_id in all events during Logical Correlation: Plugin_id 1505 The Plugin_sid of the new
event is the ID of the directive that generated the event
The name (Signature) of the new event is given by the name of the directive
The priority of the new event is given by the global priority of
the directive
SIEM
SQL Storage
Correlation
Risk Assessment
Policy
Collection
EVEN
TS
Even
t Gen
erat
ed d
uring
Log
ical C
orre
lation
129
Thursday, May 3, 12
Example: Brute Force Events will first try to get in those directive whose correlation has
started and then they will try to match with the first correlation level of the correlation rules that are enabled while the SIEM runs
The same event can be correlated in multiple correlation directives
SIEMLogical Correlation
Directives in the correlation engine Directives waiting to be correlated
130
Thursday, May 3, 12
Example: Brute-Force First correlation level
Special Conditions
Only detector rules can be used in the first correlation level Only one rule This rule doesnt have time out Only one occurrence (Correlation of the directive will start with the
first event matching the conditions of the rule
131
Thursday, May 3, 12
Example: Brute-Force First correlation level
How many events?- Always a single event in the first correlation level
What type of events should match the rule?- Plugin_id and Plugin_sid of the events
What sources and destinations should match the rule?- Hosts, Host Groups or Networks
- Source and Destination Ports
Any other special condition?- E.g.: Avoid events with a certain username from matching the rule
132
Thursday, May 3, 12
Example: Brute-Force First correlation level
What type of events should match the rule?- Check at Configuration -> Collection -> Data Sources
- Only one Data Source in each rule (One plugin_id with one or multiple plugin_sid)
What sources and destinations should match the rule?- The first rule is usually generic (We do not know yet where the attack is
coming from)
- ANY / HOME_NET (Assets in the AlienVault inventory)
133
Thursday, May 3, 12
Example: Brute-Force First correlation level
type: detector (First rule is always detector) and we will collect the events using a detector DS Connector
reliability: When calculating the risk, it will also be 0 - risk= (Asset Value * Priority * Reliability)/25
from/to: Both are set to any because we don't know yet who will be the attacker and his target
port_from/port_to: Set to ANY, not important in this type of directive
plugin_id/plugin_id: List of the events in the SSHD DS Connector that refer to fail authentications
134
Thursday, May 3, 12
Example: Brute-Force This directive would be useless with only one level
This directive with a single level will never generate an event becoming alarm (Reliability is set to 0)
Directives with one level can be used to rename events
List of the IP Addresses of the Web Servers
WIth a reliability of 0 this event will never become alarm, but it will be stored in the
SIEM SQL (SIEM -> Analysis)
135
Thursday, May 3, 12
Example: Brute-Force Second correlation level
- Authentication successful
Same Source and Same destination that matched the first correlation level
Time out before this possibility is discarded?
Would this indicate a big problem? (Authentication successful after 1 failed)
- 10 more authentication failed
Same Source and Same destination that matched the first correlation level
Time out before this possibility is discarded?
Would this indicate a big problem? (A total of 11 Authentication failed in X seconds)
136
Thursday, May 3, 12
Logical Correlation
137
Thursday, May 3, 12
Logical rule example
138
Thursday, May 3, 12
Logical rule example In the directive with id 27 every time the condition given by a rule is
met an event with priority 2 is generated. The reliability will be taken from the rule that has matched:
Maximum risk will be in the fifth rule Priority 2, Reliability 10 With this directive the event will get as much a risk of 4 (When one
of the assets involved has a value of 5)
(2*10*5) / 25 =4
139
Thursday, May 3, 12
Example: WormSIEM
140
Thursday, May 3, 12
Example: WormAlarm
SIEM
140
Thursday, May 3, 12
Example: WormAlarm
Alarm
SIEM
140
Thursday, May 3, 12
Example: WormAlarm
Alarm
Alarm
SIEM
140
Thursday, May 3, 12
Example: Intrusion
Hi!
At your command!
Persistence
Behaviour
1.1.
2.
3.
4.
F
141
Thursday, May 3, 12
Writing Correlation Rules All directives have a unique numeric identifier, a name and a global
priority that will be used in every event generated during the correlation of the directive.
The events generated when correlation the directive will always have 1505 as plugin_id and the id of the directive as plugin_sid.
Writing Correlation Rules The initial rule does not have a time_out value, and it can be
matched by any event while the SIEM is running
The conditions defined by the first rule are usually so generic because we dont know the values of the event that it is going to start the correlation:
In the following rules in many cases we will have to make sure that some fields have the same value than the event that started the correlation of the directive
Writing Correlation Rules To open a second correlation level the following tags have to be
used: and
More than one rule can be included in each correlation level (Not in the first one)
When one of the multiple rules of each correlation level is matched, the rest of the rules of that level are discarded
144
Thursday, May 3, 12
Writing Correlation Rules Two types of rules can be used when writing directives:
Detector rules: Detector rules for incoming events from the different detector plugins (SSH, Snort, Firewalls...)
Monitor rules: Monitor rules request for information to different applications and devices through the collector (Session information, permissions, availability...)
145
Thursday, May 3, 12
Writing Correlation Rules Common fields (Monitor and detector rules):
plugin_id: Numerical identifier of the tool that provides the information (Events in detector rules and indicators in monitor rules)
plugin_sid: Numerical identifier of the type of even within the tool defined by plugin_id or request or query that has to be executed (In Monitor rules)
reliability: Reliability value of every event generated within the directive. It can be an absolute value 0-10 or incremental +2, +6
time_out: Waiting time before the rule expires and the directive process defined in that rule is discarded. The first rule doesnt have a time_out value.
146
Thursday, May 3, 12
Writing Correlation Rules Detector rule: sensor: ANY
Any IP address Address IP (x.x.x.x)
An IP address (E.g: 192.168.1.9) Several IP addresses
IP address list separated by commas (E.g: 192.168.1.2,192.168.2.3) Sensor name
Sensor name (As configured in Configuration -> SIEM Components) Relative values
It is possible to relate variable values to previous correlation levels. 1:SENSOR refers to the sensor IP address in the first correlation level.
Denied values It is possible to include denied elements as value of the target field: "!192.168.2.203,INTERNAL_NETWORK
147
Thursday, May 3, 12
Detector rule: from/to: ANY
Any IP address Address IP (x.x.x.x)
An IP address (E.g: 192.168.1.9) Several IP addresses
IP address list separated by commas (E.g: 192.168.1.2,192.168.2.3) Network name
Network name (Defined in the paragraph Policy in the Web interface) Relative values
It is possible to relate variable values to previous correlation levels:- 1:SRC_IP refers to the origin IP address of the event in the first correlation level.
- 2:DST_IP refers to the target IP address in the second correlation level.
Denied values It is possible to include denied elements as a value of the origin field:
All networks in the inventory HOME_NET
Writing Correlation Rules
148
Thursday, May 3, 12
Writing Correlation Rules Detector rule: port_to/port_from:
This field can adopt an only port or a list of ports separated by commas as value. The keyword ANY is a list of all the ports.
It is possible to use relative values of ports referring to other correlation values:
1:DST_PORT refers to the values of the target port of the first correlation value.
3:DST_PORT refers to the value of the target port of the third correlation value.
To deny ports we put the symbol ! before the port we want to deny:
port="!22,25,110,!21"
149
Thursday, May 3, 12
Writing Correlation Rules Detector rule: protocol:
The field Protocol refers to the protocol in which the communication was established where the event took place.
Protocol can adopt the following values:
TCP
UDP
ICMP
Host_ARP_Event
Host_OS_Event
Host_Service_Event
Host_IDS_Event
Information_Event
It is also possible to specify the protocol with the protocol number.
150
Thursday, May 3, 12
Writing Correlation Rules Sticky
When the events arrive to the correlation engine they will try to be correlated inside directives whose correlation has been started
Using sticky we avoid those events to start the correlation of the same directive again, as they may also meet the conditions given by the same directive.
Sticky Different
This variable can be associated to any field in rules with more than one occurrence, to make all the occurrences have a different value in one of the fields
E.g.: sticky_different=DST_PORT All the events matching the rule must have a different destination
port (Port scanning detection)
151
Thursday, May 3, 12
Writing Correlation Rules Monitor rules define a condition that has to be checked using monitor
plugins
The following variables to, from, port_to, port_from, protocol, sensor, username, filename, password and userdata1-9 will have the values that will be sent to the collector to be used in the monitor plugin. Those values don not have to be met, they will only be used to build the monitor request.
Monitor rules specific fields:
value condition interval absolute
152
Thursday, May 3, 12
Writing Correlation Rules Monitor rule: condition
Establishes a logical relation between the value field and the value returned in the monitor plugin request.
eq equalne non equallt less thangt greater thanle less or equal thange greater or equal than
153
Thursday, May 3, 12
Writing Correlation Rules Monitor rule: value
This field sets the value that has to be compared with the value returned by the collector after doing the monitor request.
Value must be an integer.
Monitor rule: interval
This value of this field sets the waiting time between each monitor request before the rule is discarded because the time defined by time_out is over.
Monitor rule: absolute
This value sets if the value that has to be compared is relative or absolute.
Absolute true: If the host has more than 1000 bytes sent during the next 60 seconds. There will be an answer if in 60 seconds this value is reached.
Absolute false: If the host shows an increase of more than 1000 bytes sent. There will be an answer if the host shows this increase in 60 seconds.
154
Thursday, May 3, 12
Recommendations The objective should not always be generating alarms
Not all directives should generate alarms with a high risk value (Rate the importance of the attack or problem detected by the directive)
In some cases the last rule level should be used to keep collecting events to avoid having the same directive in the correlation engine so often
A lot of directives can be created using the same directive template (Brute force, Worms, Policy Violation...)
Use sticky to avoid directives from being generated exponentally
155
Thursday, May 3, 12
AlienVault LoggerForensic Storage
156
Thursday, May 3, 12
Hands-On: Logger create the indexed Logger
install logger package, called:- alienvault-logger
check if the indexing process is running after this you should see the indexed search possibility in the UI
157
Future versions of AlienVault will only have one Query button combining Raw and Indexed queries
Thursday, May 3, 12
Hands-On: Logger create a remote logger
ssh-keygen ssh-copy-id -i [pub_identity_file] user@remotelogger_IP example:
try to connect without password create the remote logger under AlienVault Components test the remote logger
158
Thursday, May 3, 12
AlienVault IDMIdentity Monitoring & User awareness
159
Thursday, May 3, 12
AlienVault IDM IDM - Identity Monitoring
Introduced in AlienVault 3.1
every source or destination IP can be enriched with user data
Available user information
username domain hostname IP address MAC address
160
Thursday, May 3, 12
IDM: information flow
ip=192.168.2.1 username=alien
hostname=alienshomedomain=alienvault.com
ip=192.168.24.2 username=Administrator
hostname=dc01domain=alienvault.com
SIEMSensors
Sensors
Sensors
Sensors
Send IDM events
Send IDM events
Send IDM events
Send IDM events
161
Thursday, May 3, 12
IDM: how does it work User information on the network
logins ActiveDirectory LDAP servers VPN access
Information can be retrieved by plugins
event=idm-event Rule will set username, hostname, domain etc.
162
Thursday, May 3, 12
IDM: how does it work This information will then be sent to SIEM and associated with IPs
every Event can have IDM information attached
163
Thursday, May 3, 12
Installing IDM Install alienvault-idm on all the sensors and servers for IDM
apt-get install alienvault-idm On the sensor, tell the agent to send IDM events to the SIEM
sensor: /etc/ossim/agent/config.cfg- section [output-idm] for sending to a single server
- section [idm-server-list] for sending to several SIEM/IDM servers
-[output-idm]enable=Trueip=The_IP_of_the_SIEM_server port=40002
[idm-server-list]FQDN1=The_IP_of_the_Second_SIEM_server;40002FQDN2=The_IP_of_the_Third_SIEM_server;40002
164
Thursday, May 3, 12
Installing IDM on SIEM server
add IDM in /etc/ossim/server/config.xml-
- configure IDM for the GUI
echo update config set value=1 where conf=idm_enable | ossim-db
165
Thursday, May 3, 12
Installing IDM Check if the agent already sends IDM events
works by default in some plugins- OSSEC
start integrating your own IDM data
166
Thursday, May 3, 12
Writing IDM plugins Start with a normal plugin skeleton
Insert rules for log messages generated with IDM relevant data
event-type = idm-event
Event attribute associated IDM value
username IDM username
domain IDM domain-name
hostname IDM hostname
mac IDM mac address
ip IDM associated IP
167
Thursday, May 3, 12
IDM: example plugin Example: Ossec-IDM data source
event-type = idm-event
extracted fields from regex: username, hostname, domain, ip
168
Thursday, May 3, 12
Hands-On: IDM install the required packages on the infrastructure to support IDM
do all the necessary configuration to enable the flow of IDM information across the infrastructure
from these two loglines, develop a IDM data source that reports username, hostname, domainname and IP to the SIEM
User [email protected] logged in via 192.168.2.1. Assigned IP address: 10.10.1.15
Login from trusted source 10.10.2.123, user alice, domain aliceslair.com, hostname aliceslaptop
- do you receive IDM events
- do you see IDM events in the console
169
Thursday, May 3, 12
AlienVault HIDSExtending client security
170
Thursday, May 3, 12
Hands-On: OSSEC Configuration on the server side (AlienVault machine), two
possibilities:
1) creating the agent in the UI
171
Thursday, May 3, 12
Hands-On: OSSEC 2) creating the agent on the terminal
important scripts in: /var/ossec/bin: manage_agents (add agent / extract key / list agents / remove agents)
list_agents (all / connected / not connected)
agent_control (more options than list_agents)
172
Thursday, May 3, 12
Hands-On: OSSEC Install ossec-agent on a windows machine
download ossec-agent from the AlienVault UI install it configure it:
- OSSEC Server IP:
- Authentication key:
173
Thursday, May 3, 12
Hands-On: OSSEC After finishing the OSSEC connection between server and agent:
enable OSSEC plugin (ossec.cfg) in alienvault-setup check if you get the events in the UI or agent.log check if the OSSEC events arrive in the SIEM
If not check the troubleshooting page
next page
174
Thursday, May 3, 12
Hands-On: OSSEC Troubleshoot
To check if events are arriving on the server- /var/ossec/logs/alerts/alerts.log
If not, check if the agent is properly connected- view ossec-agent log file on the windows machine
- check ossec-agent configuration file on the windows machine
- check if the OSSEC udp port 1514 is open
175
Thursday, May 3, 12
AlienVault Secure Connection
Aliens love encrypted transports
176
Thursday, May 3, 12
Hands-on: OpenVPN Connecting a sensor to a SIEM with OpenVPN
SIEM (OpenVPN server) side #alienvault-reconfig --add_vpnnode=
cd /etc/openvpn/nodes
restart/start OpenVPN process: /etc/init.d/openvpn restart
alienvault-vm:/etc/openvpn/nodes# ifconfig tun0+ ifconfig tun0tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:10.67.68.1 P-t-P:10.67.68.2 Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:65 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:0 (0.0 B) TX bytes:4340 (4.2 KiB)
177
Thursday, May 3, 12
Hands-on: OpenVPN Connecting a sensor to a SIEM with OpenVPN
send .tar.gz that includes all data we need on the sensor via scp to user@:/etc/openvpn
uncompress on the sensor side: tar -xvzf .tar.gz
restart OpenVPN: /etc/init.d/openvpn restart
alienvault-sensor:/etc/openvpn/# tar -xvzf .tar.gz + tar -xvzf .tar.gz//keys//keys/.csr/keys/.key/keys/ca.crt/keys/.crt.confopenssl.cnf
178
Thursday, May 3, 12
Hands-on: OpenVPN restart OpenVPN: /etc/init.d/openvpn restart tunnel should be established troubleshoot:
- tail -f /var/log/syslog |grep ovpn --color=auto
Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1871]: TUN/TAP device tun0 openedFeb 2 23:46:24 alienvault ovpn-AVinfraestructure[1871]: TUN/TAP TX queue length set to 100...Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: TCPv4_SERVER link local (bound): [undef]:33800Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: TCPv4_SERVER link remote: [undef]Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: MULTI: multi_init called, r=256 v=256Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: IFCONFIG POOL: base=10.67.68.2 size=62Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: MULTI: TCP INIT maxclients=1024 maxevents=1028Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: Initialization Sequence Completed
179
Thursday, May 3, 12
Hands-on: OpenVPN Adapt sensor configuration to tunnel IP addresses
edit ossim-setup.conf on the sensor change all to mostly 10.67.68.1 run alienvault-reconfig
Check events arriving over the tunnel
Security Events (SIEM) / Logger or tcpdump -i tunX
180
Thursday, May 3, 12
SnortNetwork IDS
181
Thursday, May 3, 12
What is Snort? Snort is an NIDS (Network Intrusion Detection System)
It is Open Source (GPL v2) Snort combines signature, protocol and anomaly-based inspection, It has an active development and new rules are added daily Snort is the most widely deployed IDS technology worldwide
182
Thursday, May 3, 12
Brief History Snort was released as an Open Source product in 1998. Since
then its source code has been distributed using the GPL license
Snort is developed by Sourcefire, company founded by the original creator of Snort (Martin Roesch).
New Snort versions are released often, including new functionalities and fixing bugs and errors.
183
Thursday, May 3, 12
IDS Intrusion Detection System, security tool in charge of monitorizing
events on a system in search of traces of an intrusion.
An intrusion attempt is an attempt to compromise the confidentiality, integrity or availability of a computer system or to circumvent its security measures.
184
Thursday, May 3, 12
NIDS Network level IDS
Monitor network traffic
Advantages
Monitor the entire network traffic No impact on the network
Disadvantages
Unable to analyze encrypted information Requires continuous rule update in order to catch new attacks Requires specific network configuration (Port mirroring or Network
Tap)
185
Thursday, May 3, 12
NIDS: Network Traffic If the network traffic is not forwarded to the NIDS machine, only
the traffic generated that has as source or destination that machine will be analyzed.
Network traffic can be forwarded using the following methods:
Network taps Hubs [+ listen only network cables] Switches with a mirroring or span port
186
Thursday, May 3, 12
Why a NIDS? Some attacks and problems can not be detected using a firewall
Attacks tunneling the network traffic Attacks using vulnerabilities in the applications Attacks from our internal network against the internal or external
network.
187
Thursday, May 3, 12
Snort modes Snort can be used in the following modes:
Sniffer: Monitor the activity of the network Packet logger: All the network activity is stored in capture files IDS : The network traffic is compared with predefined patterns in
search of anomalies, attacks or policy violations
In AlienVault only the IDS mode of Snort is used
188
Thursday, May 3, 12
Snort and AlienVault Snort is a very important tool within AlienVault and it has been used
by AlienVault since the first OSSIM release for several reasons:
Detailled attack information from passive listening
Security problems (Trojans, Virus, Worms...) Policy Violation (Pornography, p2p, messenger) Bad configurations
Snort is the main source of information for correlation directives
Near to Zero configuration
Snort events are used in logical, inventory and cross correlation
189
Thursday, May 3, 12
Snort and AlienVault Using snort, we can collect information that could also be
collected from many applications such as Web servers and Proxies logs.
The main advantage of Snort is that all the events will be generated in a passive mode (URL visited, program versions, Web server response...)
190
Thursday, May 3, 12
Snort and AlienVault Snort is integrated as a detector plugin in AlienVault
The AlienVault Sensor collects logs from the Snort Data Source agent and send them normalized to the SIEM or Logger
Snort should only be used in those interfaces collecting the network traffic
Snort loads all signatures in memory to analyze the network traffic in real time. If all rules are enabled and a lot of traffic has to be analyzed high performance hardware will be required.
AlienVault Appliances (Sensors) are built for this task
All the events generated by the snort rules will have the plugin_id 1001. The events generated by the Snort preprocessors will have a different plugin_id for each preprocessor within the range 1002-1500.
191
Thursday, May 3, 12
Snort Architecture The Snort engine can be divided into the following components:
Decoders Preprocessors Detection engine Output plugins
192
Thursday, May 3, 12
Architecture: Decoders Decoders: The decoders collect the network packets and
prepare them to be analyzed by the preprocessors and the detection engine.
193
Thursday, May 3, 12
Architecture: Preprocessors Snort's preprocessors are divided in two categories.
A number of attacks cannot be detected by signature matching via the detection engine. so some preprocessors are used to examine packets for suspicious activity.
The other preprocessors are responsible for normalizing traffic so that the detection engine can accurately match signatures.
194
Thursday, May 3, 12
Architecture: Preprocessors frag3: Its main objective is avoiding detection evasion techniques. It
reassembles network packets to be analyzed by the detection engine.
stream5: Reassembles TCP traffic and monitors TCP and UDP sessions
http_inspect: Normalizes and searches anomalies in http traffic
rpc_decode: Normalizes and searches anomalies in rpc traffic
bo: Detects back orifice traffic in the network
ftp_telnet: Normalizes and searches anomalies in ftp and telnet traffic
smtp: Normalizes and searches anomalies in smtp traffic
sfportscan: Detects network scans in the network
195
Thursday, May 3, 12
Architecture: Preprocessors arpspoof: Decodes ARP packets and detects ARP attacks,
anomalies and inconsistence's
ssh: Detects the use of different exploits against the ssh protocol
dcerp: Detects and decode SMB and DCE/RCP traffic
dns: Detects the use of different exploits against the DNS service
ssl: Detects the SSL/TSL traffic and determines whether it has to be analyzed or not
196
Thursday, May 3, 12
Architecture: Detection Engine Detection engine: The Detection engine analyzes every network
packet to determine if there is any malicious or forbidden activity on it.
The Detection engine uses rules that are loaded when Snort starts.
Rules are compared with every network packet. If one rule is matched, an event is generated.
197
Thursday, May 3, 12
Architecture: Detection Engine alert tcp $HOME_NET any -> 10.1.1.0/24 80 (flags: SF; msg: SYN-FIN Scan;)
Header:
alert: Action that has to be done if the rule is matched tcp: protocol $HOME_NET any: Source 10.1.1.0/24 80: Destination
Options:
flags: SF; msg: SYN-FIN Scan : Conditions that have to be met and name of the event if the rule is matched.
198
Thursday, May 3, 12
Architecture: Detection Engine Rules to detect access miscelaneous web activities:
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET POLICY Megaupload file download service access"; flow:to_server,established; content:"GET "; depth: 4; uricontent:"/?d="; content:"|0d 0a|Host\: "; content:"megaupload.com"; within:25; nocase; classtype:policy-violation; reference:url,doc.emergingthreats.net/2009301; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/POLICY/POLICY_Download_Services; sid:2009301; rev:2;)
alert tcp $HOME_NET any -> $EXTERNAL_NET 5050 (msg:"ET POLICY Yahoo Chat Signin Inside Webmail"; flow:established,to_server; content:"content-length\:"; nocase; depth:15; content:"
Architecture: Output Plugins Output plugins: The events generated by the detection engine or
by the preprocessors can be stored in different formats. The following output plugins can be enabled in Snort:
Syslog Database Tcpdump Unified v1 Unified v2
200
Thursday, May 3, 12
Architecture: Output Plugins More than one output plugin can be enabled at the same time
Syslog Database Tcpdump Unified Unified2 csv, prelude...
In production for optimal performance
Unified (Version 2)
201
Thursday, May 3, 12
Snort Operation
Network traffic
Packets capture
SNORT
Decoders
Output Plugins
Preprocessors
Decoders
Alerts
202
Thursday, May 3, 12
Snort Configuration Snort configuration files in AlienVault can be found in the following
directories:
/etc/snort/ The main configuration file is:
/etc/snort/snort.conf Each interface can have a different configuration file using the
following naming convention
/etc/snort/snort.eth0.conf /etc/snort/snort.eth1.conf
203
Thursday, May 3, 12
Startup Options All the startup options can be found in the Snort man file
# man snort To include any of those options when Snort is started as a service,
the parameter has to be included in the following file:
/etc/default/snort Monitoring networks and listening interfaces are configured in the
file
/etc/snort/snort.debian.conf This file is edited automatically by alienvault-reconfig and should not
be edited manually
204
Thursday, May 3, 12
Snort.conf includes The Snort.conf file can include other configuration files as well as
rule definition stored in different files:
E.g.:
include threshold.conf include classification.config
205
Thursday, May 3, 12
Snort.conf variables In the main configuration files, variables can be defined. This variables
can be used in the same configuration files or in the rules definition:
HOME_NET: Local network monitored. (Automatically edited by alienvault-reconfig based on the networks defined in the OSSIM inventory)
EXTERNAL_NET: External networks (Usually !$HOME_NET -> Networks not in $HOME_NET)
Defining other variables can help us reducing the number of false positives
DNS_SERVERS, SMTP_SERVERS, SSH_PORTS
emerging.rules:alert udp $HOME_NET any -> $DNS_SERVERS 53 (msg:"ET CURRENT_EVENTS Infected System Looking up chr.santa-inbox.com CnC Server"; content:"|03|chr|0b|santa-inbox|03|com"; nocase; classtype:trojan-activity;)
206
Thursday, May 3, 12
Snort.conf Preprocessors Preprocessors are configured in the main configuration file
(snort.conf) as follows:
preprocessor : Some rules use preprocessors so they should never be disabled
as it may affect to the detection engine functionality
Each preprocessor will have a different plugin_id (The range 1000-1500 in plugin_id is reserved for Snort events)
207
Thursday, May 3, 12
Snort.conf Outputs The output plugins can be configured in Snort.conf with the
following syntax:
output : The AlienVault Sensor needs Snort running with the 2the unified
output enabled as follows
output unified2: filename snort, limit 128
Unified Output stores events in Binary format
208
Thursday, May 3, 12
Snort Output - AlienVault Sensor The Snort plugin in the Agent must be configured according to the
configuration of the unified out