CSI ch07

Operating Systems Security

Chapter Seven

Prepared by: Raval, Fichadia

Raval FichadiaJohn Wiley & Sons, Inc. 2007

3The Big PictureElements of an operating

system environment.

Some risks that impact the operating system.

4Operating system primerOperating system: the system software on a machine upon

everything else depends.

Most critical software relied upon by almost every other application on a machine.

Key responsibilities of an OS include: CPU/process management

Memory management

Hardware management

Disk and file system management

Examples of popular operating systems include Windows, Unix, Linux, zOS, MacOS.

5Operating system primerOperating system: the system software on a machine

upon everything else depends.

Orange book describes the criteria for security ratings on an operating system.

Division D (Minimum Security) Class D Systems that arent rated higher

Division C (Discretionary Protection) Class C1 Discretionary security protection (C1 is no longer used to rate OSs) Class C2 Controlled access protection (rated versions include versions of OpenVMS,

Windows, AS/400, RS/6000) Division B (Mandatory Protection)

Class B1 Labeled security protection (rated versions include DEC, HP-UX, IRIX) Class B2 Structured protection (rated versions include XENIX 4.0) Class B3 Security domains (rated versions include XTS-200)

Division A (Verified Protection) Class A1 Verified design (rated versions include Boeing MLS)

6Management concernsConcerns about operating system security typically include

the following: Minimizing the companys risk exposure in the most

cost-effective manner. Defining the optimal security posture across a

heterogeneous environment. Keeping up with existing and upcoming security threats

against operating systems and implementing mitigating controls.

Having an effective backup, recovery, business resumption and a disaster recovery plan.

7Risks and controlsAuthentication: Process of validating user identity Most operating systems require a user ID and a

password combination for authentication. Some allow for third-party based authentication. User IDs are used by regular end users, administrators,

and applications to claim their identity. Knowledge of the password proves the users identity. Operating systems typically never store passwords,

rather they store the password hash - a unique fixed length output based on the password and/or a salt.

8Risks and controlsAuthentication: Process of validating user identity

Wrong password generate wrong hashes and user is denied access. Right password generates right hash and allows user access.

9Risks and controlsAuthentication risks:

Poor password selected by end users.

Dictionary and/or brute-force attacks against password hashes.

Controls:

Password management routines such as requiring periodic password changes, quality passwords, longer passwords, etc.

Enabling account lockouts after a given number of wrong password attempts.

Using two-factor authentication techniques: something you have and something you know.

10

Risks and controlsAuthorization: Process of determining user privileges

Most operating systems determine user IDs privileges after users authentication.

Protected resources have an Access Control List (ACL) that defines who has what level of access.

User IDs are assigned privileges to protected resources either directly, or by them being part of a group that has access.

Some operating system like Unix/Linux allow users to exceed their privileges via features like setuid/setgid.

11

Risks and controlsAuthorization risks:

Poorly defined ACLs.

Attacks that let users again unauthorized privileges (setuid/setgid exploits, exploiting services/applications with higher privileges).

Controls:

Periodic review of ACLs and privileges granted to users and groups.

Protecting setuid/setgid related files from exploited by implementing stronger file permissions.

Disabling unnecessary privileged services / uninstalling unnecessary privileged applications.

12

Risks and controlsTrust relationships: Relying on other OS for validating a

user identity.

Many operating systems allow access to their resources without verifying a users credentials if they trust other OS of doing so.

Trust simplifies and adds convenience for user and application by not having to reauthenticate.

Without trust relationships credentials may have to be stored in multiple places.

Trust can be unidirectional or bidirectional.

13

Risks and controlsTrust risks:

Spoofing attacks wherein users forge their identities.

Transitive trust that let users go from AB and from BC, however AC wasnt intended.

Controls:

Periodic review of trust relationships and their business need. Disable the ones that arent needed.

Implement patches and controls to prevent spoofing attacks.

14

Risks and controlsJob scheduling: Allows programs and scripts to run without

user interaction at scheduled times.

Most operating systems allow definition of jobs (programs, scripts, batch files) to be run periodically at scheduled hours with different level of privileges.

Schedulers can offload backups during off-peak hours.

Jobs can be scheduled at different hours, or upon different triggers such as system startup, shutdown, results of previous job etc.

15

Risks and controlsJob scheduling risks:

Piggybacking attacks wherein malicious code is inserted into unprotected scripts (or scripts referenced by the job) that are scheduled as privileged jobs.

Program substitution wherein a malicious script is executed instead of the original due to lax permissions on directories or because of lack of full path names.

Controls:

Periodic review of all jobs and their privileges. Disable the ones that arent needed and/or minimize privileges.

Implement strong file and directory permissions.

16

Risks and controlsFile systems: OS feature to manage files stored on the hard

drive(s).

File systems allow for tracking of the location of files and their directories, their classification, their attributes, their ACLs, and their security.

File systems can be for local files (files stored on locally on the computer) or for remote files (files shared with remote computers).

OSs can support multiple types of file systems such as NTFS, FAT, EFS (local), NFS, CIFS (remote) etc.

Local and remote file systems are protected via ACLs that define who has what type of access to the file.

17

Risks and controlsFile systems: OS feature to manage files stored on the hard

drive(s).

Most file systems classify files in an inverted tree structure.

18

Risks and controlsFile system risks:

Default ACLs or user-defined ACLs on local and shared files could be poor.

Critical files containing sensitive/confidential data could be poor secured.

Controls:

Periodic review ACLs for sensitive files and directories. Use encryption on sensitive files and directories. Baseline the file system to detect unauthorized

changes.

Periodic review ACLs for shared files.

19

Risks and controlsSoftware updates: Updating OS with vendor provided fixes.

Operating systems have defects that are identified either by end users or by vendors.

Vendors release fixes patches to fix/mitigate the defect.

Hotfixes are patches that have limited regression testing done and typically address a small number of issues.

Service packs are collection of patches that are regression tested and address several issues.

20

Risks and controlsSoftware update risks:

Inadequate testing of patches by vendor or end-user could break existing functionality.

Zero-day exploits could hit OS before patches are applied.

Controls:

Periodic review threats and their fixes.

Maintain inventory of OS and their patch level.

Have a test bed ready to test the impact of patches on existing functionality.

Use automated tools to deploy patches at the earliest.

21

Assurance considerationsAn audit to assess operating system security should include

the following:

Evaluate the need for each service on the OS. Unnecessary services should be turned off.

Assess the patch deployment process. All necessary patches should be implemented.

Review authentication methods, user accounts and their privileges. Ensure quality user ID and password management techniques are used.

Determine the legitimacy of trust relationships.

Ensure core operating system files are baselined and protected.

22

Assurance considerations Evaluate the need and security of all file shares.

Determine if additional controls like anti-virus, personal firewalls, and intrusion prevention systems are in place.

Review all scheduled jobs and their controls.

Ensure that functional plans for backup and recovery, business resumption, disaster recovery are in place.

23

Recap