Module 8 ********* Data Validation & Security Workforce Information Database Training Last update November 2006.

Module 8*********

DataValidation &

SecurityWorkforce Information Database TrainingLast update November 2006

Providing Accurate Data Consistently

Data validation allows us to provide data with confidence in its accuracy, and we can consistently provide this data by implementing thorough security.

Module 8Data Validation & Security

Data IntegrityValidity, consistency, and accuracy of the data in a database.

•Table-level•Field-level•Relationship-level•Business Rules

Module 8Data Validation & Security

Module 8Data Validation & Security

Data ValidationThe process of determining if an update to a value in a table’s data cell is within a preestablished range or is a member of a set of allowable values.

What are some common data quality problems that affect

data integrity and validation?

Module 8Data Validation & Security

Common Data Quality Problems

• Illegal values• Violated attribute dependencies• Uniqueness violation• Referential integrity violation• Missing values• Misspellings• Cryptic values• Embedded values• Misfielded values

Module 8Data Validation & Security

Common Data Quality Problemscontinued….

• Word transpositions• Duplicate records• Contradicting records• Wrong references• Overlapping data/matching records• Name conflicts• Structural conflicts• Inconsistencies

Module 8Data Validation & Security

What are some methods for checking for data validity?

Module 8Data Validation & Security

Methods for checking for data validity

• Visual/manual• Aggregation• Reviewers guide• Auto data checks• Record counts• Spell checks• Have data provider review

Module 8Data Validation & Security

Time to Exercise!!

Module 8Data Validation & Security

•Network permissions

Physical security(including backup & restoration plan)

Data Storage Access(Server or PC)

Web security

ApplicationsecurityUser level

ALMIS Database Security Considerations

Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

Security Concerns:ConfidentialityIntegrityAvailability

User levelUser level

Applicationsecurity

•RDBMS•ODBC

•Suppression flags•Data aggregation issues

Network (LAN/WAN)

security

The contact in my state is:______________

•Production vs. test •Production vs. test

Module 8Data Validation & Security

•Network permissions

Physical security(including backup & restoration plan)

Data Storage Access(Server or PC)

Web security

ApplicationsecurityUser level

ALMIS Database Security Considerations

Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

Security Concerns:ConfidentialityIntegrityAvailability

User levelUser level

Applicationsecurity

•RDBMS•ODBC

•Suppression flags•Data aggregation issues

Network (LAN/WAN)

security

The contact in my state is:______________

•Production vs. test •Production vs. test

Module 8Data Validation & Security

Physical Security

Questions to ask yourself:

• Where is your data actually stored?

• Are you responsible for physical security?

If you are...

Module 8Data Validation & Security

Physical Security

• What physical security measures do you have in place?

• Do you have the right hardware?

• Are you using obsolete hardware that is prone to crashing or hacking? Do you have a choice?

Module 8Data Validation & Security

Physical Security

• Do you have a database backup plan?

• Do you have remote backup so if a fire burns the building you don’t lose both your primary and backup data at the same time?

• Do you have a data restoration plan? Does the backup plan allow for feasible restoration?

Module 8Data Validation & Security

Physical Security

Restoration Plan

• Does your restoration plan allow for the restoration of individual tables or require the entire database?

• Do you maintain copies of the tables on your hard drive?

Module 8Data Validation & Security

Physical Security

• Who has access to the physical storage location?

• Is your database on a PC or server that someone could easily kick, spill coffee on, or accidentally reappropriate?

If you are not responsible for physical If you are not responsible for physical security, ask these questions of those security, ask these questions of those who are!who are!

Module 8Data Validation & Security

•Network permissions

Physical security(including backup & restoration plan)

Data Storage Access(Server or PC)

Web security

ApplicationsecurityUser level

ALMIS Database Security Considerations

Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

Security Concerns:ConfidentialityIntegrityAvailability

User levelUser level

Applicationsecurity

•RDBMS•ODBC

•Suppression flags•Data aggregation issues

Network (LAN/WAN)

security

The contact in my state is:______________

•Production vs. test •Production vs. test

Module 8Data Validation & Security

Data Storage Access

• What software are you using to store the database?

• SQL Server, Oracle, FoxPro, Access?

Survey says:

SQL Server 48%

Oracle 28%

Other 24%

Module 8Data Validation & Security

Data Storage Access

• What can be done at the server level to provide for security?

• Confidentiality, Integrity, Availability

DO apply advanced security to the most sensitive data, but DON’T apply advanced security to non-sensitive data

Module 8Data Validation & Security

Data Storage Access

Security at the server and/or database level…

• Permissions

• What kind of permissions exist?

• Read only, SA, etc.

• Who sets those permissions?

• Who has those permissions?

Module 8Data Validation & Security

ALMIS Database Security

Security at the data level…

• Suppression

• Suppression flags

• Suppression can be handled at the database level or the application level

Module 8Data Validation & Security

ALMIS Database Data Security

Two approaches:

1. ALMIS Database contains NO confidential data (all data available for use without restriction)

2. ALMIS Database contains confidential data (data access controlled by security and/or suppression)

Module 8Data Validation & Security

ALMIS Database Security

Tables that have suppression flags...

ces

indprj

industry

iomatrix

occprj

oeswage

stindprj

stoccprj

NOTE: tables stfirms and wage have no suppression flags but may contain confidential data

Module 8Data Validation & Security

ALMIS Database Data Security

Issues to consider regarding suppression:

• If your database doesn’t contain suppressed records, detailed data won’t aggregate to totals (validity checks?)

• Without proper suppression, confidential data can be back-calculated

Module 8Data Validation & Security

ALMIS Database Data Security

A note on confidential data:

If you are checking 202 data (or any other potentially confidential data) to see that it loaded right, make sure your EQUI data files and/or printouts are always secured and shred all photocopies when finished.

Check with your local BLS personnel or LMI administrator for specific confidentiality policies.

Module 8Data Validation & Security

•Network permissions

Physical security(including backup & restoration plan)

Data Storage Access(Server or PC)

Web security

ApplicationsecurityUser level

ALMIS Database Security Considerations

Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

Security Concerns:ConfidentialityIntegrityAvailability

User levelUser level

Applicationsecurity

•RDBMS•ODBC

•Suppression flags•Data aggregation issues

Network (LAN/WAN)

security

The contact in my state is:______________

•Production vs. test •Production vs. test

Module 8Data Validation & Security

User Access

Three major types of user access:

• PC - direct to database (ODBC or RDBMS)

• Network (LAN/WAN) through application

• Web (passive or active through application)

Module 8Data Validation & Security

User Access via the Web

• How do Web users access the data from the database?

• Passive (static tables automatically updated to web pages)

• Active (query through application)

Module 8Data Validation & Security

User Access via the Web

Questions to ask yourself:

• What kind of web server are you using? What are the security considerations with that choice?

• What kind of firewall do you have?

Module 8Data Validation & Security

User Access via the Web

• Does your web interface have security/confidentiality suppression?

• Do you display suppressed data to certain users? If so, how is access managed? Passwords? IP address?

Module 8Data Validation & Security

User Access via a Network (LAN/WAN)

Questions to ask yourself:

• What type of network system do you have?

• Who has rights/access to your data via the network? Through what applications?

• Are confidential data (suppressed records) available?

Module 8Data Validation & Security

User Access via a Network (LAN/WAN)

• How do you control access to confidential data? Through user Ids? Through server access permissions?

• DO recognize the important distinction between network security and data security.

Module 8Data Validation & Security

User Access via direct connection

Questions to ask yourself:

• Is local access machine password protected?

• Who has access to your machine?

• Is there a backup plan for your access machine?

• Is the source data for your database secure? Backup plan? Restorable?

Module 8Data Validation & Security

ALMIS Database Data Security

Documentation

It may prove beneficial to keep detailed records on…

• How security is done• Where security exists• Who is responsible for security• Who has access/permissions to what• Etc...

Module 8Data Validation & Security

Data SecurityDISCUSSION

Module 8Data Validation & Security