Effective(Proactive(Detection(of NetworkSecurity Incidents((€¦ · Effective(Proactive(Detection(of NetworkSecurity Incidents(((Piotr Kijewski ([email protected]) CERT Polska

Effective  Proactive  Detection  of  Network  Security  Incidents    

Piotr Kijewski ([email protected]) CERT Polska Congreso Seguridad en 2011 Mexico City, 25th November 2011

ABOUT  CERT  POLSKA  

History  of  CERT  Polska  

CERT  NASK  

1996   1997   1998   1999   2000   2001   2002   2003   2004   2005   2006   2007   2008  

   Transformation  

into  CERT  Polska  

Start    of  cooperation  with  ABW  

Accreditation  

SCP  (Microsoft  Security  

Cooperation  Program)  

2011  

ABUSE  FORUM  established  

Anti-­‐Phishing  Working  Group  

2009  

A part of NASK Research Institute plnationwide ISP)

2010  

Honeynet  Project  Member  

CERT  POLSKA  Activities  Qualified  incident  

response  

Education  and  awareness  activities      in  IT  security  area  

Research  and  reports  

IT  security  projects  

Helping  users  and  institutions  

Contact  with  media  

Cooperation  with    CERTs  and  security  

organisations  

Organisation  of  SECURE  Conference  

Registration  and  handling  of  incidents  affecting  

security  

Structure  

Core  IRT  Team     Security  Projects  Team  

CERT  Polska  13 employees

Sample  technical  projects    

Early warning system based on server honeypots

Global system for monitoring and analyzing

Internet threats A Framework for Information

Sharing and Alerting

Client honeypot system

Case  study:  HoneySpider  Network  

Case  study:  HoneySpider  Network  

Case  study:  HoneySpider  Network  

Initiatives  

Cooperation  between  Polish  

CERTs  

ABUSE-­‐FORUM  

Support  in  establishing  new  CERTs  in  Former  Soviet  Republics  

CLOSER  

Contact  point  for  illegal  content  

 now  outside  CERT  

National  and  international  trainings  for  

different  entities  

               Case  study:  CLOSER  Project  

2  -­‐  year  programme,  funded  by  NATO  2007-­‐2009    

Primary  goals:  help  in  setting  up  CERTs  in  former  Soviet  republics  

assistance  in  the  first  phases  of  their  existence  

New  CERTs  in  Armenia,  Azerbaijan,  Georgia,  Moldova,  Ukraine  

Assistance  in  joining  international  forums:  TERENA  TF-­‐CSIRT,  FIRST  

Operational  level      forwarding  relevant  incidents  

help  in  incident  handling  

monitoring  progress  of  incidents  being  handled  

Current  &  future  research  areas  

Looking  at  algorithms  for  

meaningful  analysis  of  large  security  

datasets  

Analysis  of  large  security  datasets  

Research  into  ways  of  automating  

analysis  of  botnets  

DNS  research  

Looking  at  ways  to  detect  malicious  domains  at  the  registry  level  

Botnet  research   Mobile  threats  

Research  into  automating  ways  of  

detection  and  analysis  of  malware  on  smartphones  

STUDY  ON  PROACTIVE  DETECTION  OF  NETWORK  INCIDENTS  

About  the  study  

ENISA  commisioned  study:  Proactive  Detection  of  Network  Security  Incidents  Focused  on  CERTs  (national/governmental)  but  also  other  security  teams  Major  goals:  

 do  a  stocktacking  of  measures  used  by  CERTs  to  proactively  detect  incidents    identify  shortcomings  and  provide  recommendations  on  how  to  mitigate  them  

What  is  meant  by  proactive  detection?  

Detection  of  malicious  activity  in  a  constituency  before  constituents  become  aware  of  the  problem  and  report  it  themselves  Effectively  early  warning  for  constituents  Can  be  achieved  by  using  external  data  feeds  that  report  incidents  or  by  deploying  internal  monitoring  tools  

Survey  overview  

Surveyed  105  different  CERTs,  primarily  from  Europe  but  also  a  few  outside  of  Europe.  Never  heard  from  11  CERTs,  5  refused  to  answer  Overall,  45  CERTs  responded  (despite  four  rounds  of  requests)  96  questions  in  all,  a  small  (but  key)  sample  presented  here    

Respondent  Organization  Profile  

33%  

32%  

14%  

12%  

7%   2%   Government/public  administration  Academic  

ISP  

Other(please  specify)  

Commercial  Company  

Financial  

How  do  you  obtain  incident  data  about  your  constituency?  

0

20

40

60

80

100

120

140

160

180

200

Internal  monitoring  

Monitoring  of  external    public  sources    

Monitoring    of  commercial  sources

Monitoring  of    closed  sources

Primary  source  

Auxiliary  source  

Not  used  

numbe

r  of  respo

nses  

Incoming  incident    reports  (reactive)    

Feelings  regarding  info  sources  

4%  

49%  47%  

We  are  fully  satisfied  with  information  sources  we  currently  have  

We  would  consider  to  try  other  sources  to  improve  

We  feel  information  deficit  in  general    we  think  there  are  significantly  more  incidents  we  do  not  know  about  We  feel  we  have  too  many  information  sources  

What  would  you  like  to  improve?  

30%  

26%  

22%  

12%  

10%  

Accuracy  

Coverage  

Timeliness  

Ease  of  use  

Resources  required  

Do  you  use  closed  sources  that  you  cannot  disclose?  

Yes  61%  

No  39%  

Most  often  used  sources  of  information  

22  

18   18  17  

15  14  

12   12  11   11   11  

7  6  

4   4  3   3   3  

2   2  1   1   1  

0   0   0   0  0  

5  

10  

15  

20  

25  

*  -­‐  including  non-­‐public   sources  which  respondents   could  not  disclose  

           40%  

Top  rated  sources  of  information  

0  10  20  30  40  50  60  70  80  90  

Num

ber  o

f  respo

nses   excellent   good   fair   poor  

Most  often  used  internal  tools  

0  

5  

10  

15  

20  

25  

30  

35  

40  

45  

50  

No  answer  I  never  used  it  and  will  not  use  it.  I  used  it  in  the  past,  but  dropped  it.  I  don't  use  it  but  plan  to  use  it  in  future.  I  use  it  

Do  you  collect  data  about  other  constituencies?  

45%  

43%  

7%  5%  

yes  

no  

cannot  tell  

not  sure  

Do  you  share  this  information?  

Yes  52%  

No  48%  

           23,4%    

In  what  form?  

38%  

32%  

24%  

6%  Preprocessed  data  (for  example,  to  lower  false  positives)  Interpreted  data  (for  example,  attribution  to  a  threat)  Raw  data  

Other  

Under  what  rules?  

56%  18%  

15%  

7%  4%  

Limited  access  

Other  

Anyone  (public)  

Commercial  

Public  subscription  based  

SO  WHAT  HAPPENS  IF  YOU  DO  COLLECT  ALL  THIS    

Scale  of  incidents    Poland  1h2011  (automated  submissions)  

Resources  available  

45%  

31%  

13%  

11%  

We  do  process  all  incoming  information,  but  only  higher  priority  incidents  are  further  handled,  more  input  information  would  leave  even  more  lower  priority  incidents  without  attention  

We  can  fully  handle  current  amount  of  incident  information.  We  could  handle  even  more  incident  information  

We  can  fully  handle  current  amount  of  incident  information,  but  would  not  be  able  to  handle  more  

We  cannot  properly  handle  even  the  amount  of  incident  related  information  currently  available  

Do  you  correlate?  

Yes  80%  

No  20%  

How  do  you  correlate?  

56%  26%  

18%  

Adhoc  

Automated  system  

Adhoc  and  automated  system  

           35,2%    

SOME  TOOLS  TO  HELP?  

Tools  for  correlation  &  sharing  

Abuse  Helper  (http://www.abusehelper.be/)  Megatron  (contact  SITIC/CERT.se)  Collective  Intelligence  Framework  (http://code.google.com/p/collective-­‐intelligence-­‐framework/  )  

 currently  in  beta)  

n6  PLATFORM    

n6  =  Network  Security  Incident  eXchange    Current  situation:  lots  of  different  sources  of  information,  need  to  be  handled  individually    Goal:  gather  as  much  as  possible  and  provide  an  unified  service    For  now,  focus  on  incidents,  not  other  types  of  data  

n6  PLATFORM    

n6  ENGINE  

files by SMTP files by HTTP

ISPs  

CSPs  

CERTs  

Banks  

Security  Data  Providers   H

TTPS

URLs

Domains

IPs

Malware

Credentials

n6   Assumptions  

We  provide  raw  data,  as  obtained  from  the  original  source    No  normalization  or  transformation    Sometimes  enhanced  by  additional  information:  e.g.  IP  and  ASN  data  for  domains    Sources  may  differ  in  quality    we  can  only  control  our  own  systems    

n6   Who  to  share  with  

Who  do  we  share  with  ISP  content  providers  public  administration  financial  institutions  

 Each  entity  gets  filtered  (relevant)  data  

ASN  /  IP  addresses  URLs  more!  

n6   What  to  share  

Aggregated  sources:  our  systems  (ARAKIS,  HSN,  ...)  external  organizations  -­‐  major  data  providers  and  less  known  ones  

infected  hosts  (bots)  

malicious  URLs   scanning  

Types  of  data  

malicious  artifacts    

DDoS  

fast  flux  

brute  force  

phishing  

C&C  servers  

n6   How  to  share  (requirements)  

Simple  format  =>  easily  processing    Easy  access  =>  quick  on-­‐boarding    Secure  access    Tailored  data      

n6   How  to  share  (solution)  

All  data  published  as  flat  files    Secure  transfer  over  HTTPS,  certificate-­‐based  authentication  of  clients  (own  CA)    Files  grouped  by  date    Every  client  gets  an  individual  view  

n6   Example  

Go  to  -­‐  https://N6.cert.pl/  [Valid  CRT  needed!]    Directory  listing       ./2011-­‐10-­‐09:       bots-­‐drone-­‐poland.csv       bots-­‐infected.txt       cnc-­‐ip-­‐poland.csv       ddos-­‐poland.csv       malurl-­‐1.txt       malurl-­‐2.txt       malurl-­‐hsn       scanning-­‐arakis-­‐pl-­‐1.txt       scanning-­‐arakis-­‐pl-­‐2.txt       ...     ./2011-­‐10-­‐10:       ...    

n6   Examples  of  Content  "IP  Address","Port","Channel","Country","Region","State","Domain",  "ASN","AS  

   "141.213.238.252  194.109.129.220  217.17.33.10  193.109.122.77  195.140.202.142  

 6667,  

"BAY  CITY  |  AMSTERDAM  |  WARSAW  |  AMSTERDAM  |  STOCKHOLM  |  -­‐  |  PARIS  |  -­‐  |    

"TEXAS  |  NOORD-­‐HOLLAND  |  MAZOWIECKIE  |  NOORD-­‐HOLLAND  |  STOCKHOLMS  LAN  |  -­‐  |  ILE-­‐DE-­‐FRANCE  |  -­‐    "UMICH.EDU  XS4ALL.NL  ATMAN.PL  UNDERNET.ORG  PEER-­‐IX.NET  ABOVE.NET  HARMEL.IN  MZIMA.NET  YANIR.CO.IL  -­‐  

 "UMICH-­‐AS-­‐5  XS4ALL  ATMAN  NL  PORT80  MFNX  BSOCOM  GTT  SMILE  COLOSOLUTIONS",  "University  of  Michigan  |  NL  XS4ALL  Internet  BV  |  ATMAN  Autonomous  System  |  BIT  BIT  BV  |  Phonera  Networks  AB  |  MFN  -­‐  Metromedia  Fiber  Network  |  BSO  Communication  Network  |  Global  Telecom  &  Technology  ASN  |  ASN  Euronet  

2011-­‐10-­‐09  08:09:02+00:00;64.37.60.15;;??;22;6;12  2011-­‐10-­‐09  08:09:40+00:00;70.79.1.126;AS6327;CA;445;6;12  2011-­‐10-­‐09  08:09:51+00:00;79.77.211.244;AS9105;GB;23;6;6  

n6   Future  Development  

Looking  for  new  sources    Comments  /  suggestions  are  welcome    Contact  us  if  you  want  to  participate!    e-­‐mail:  [email protected]  

SO  WHAT  ELSE  IS  MISSING?  

Are  some  incidents  underreported?  

Yes  75%  

No  25%  

Takeaways  

It  is  possible  to  proactively  detect  many  incidents  in  a  network  by  using  just  external  sources  Expanding  your  own  network  monitoring  capabilities  not  only  leads  to  better  detection  but  also  allows  you  to  reciprocate  to  data  providers  Automation  and  correlation  are  critical    there  are  tools  appearing  that  can  help  

And  finally    

ENISA  report  on  Proactive  Detection  of  Network  Security  Incidents  will  be  published  Q4  2011  30  different  data  providers  listed  and  assessed  16  shortcomings  identified  (both  technical  &  legal/organizational)  Tips  on  how  to  start  and  improve  (both  on  the  data  provider  and  consumer  side  

THANK  YOU!    

Web:  www.cert.pl    Facebook:  facebook.com/cert.polska  Twitter:  @CERT_Polska_en  Contact:  [email protected]