Statistical Software Packages: How do I get this into that? Gillian Byrne Memorial University of Newfoundland Atlantic DLI Training - April 23, 2004.

Statistical Software Statistical Software Packages:Packages:

How do I get How do I get thisthis into into thatthat??

Gillian ByrneGillian Byrne

Memorial University of Memorial University of NewfoundlandNewfoundland

Atlantic DLI Training - April 23, 2004Atlantic DLI Training - April 23, 2004

The BasicsThe Basics

Data is often available in flat ASCII text Data is often available in flat ASCII text filesfiles

Data Definition FilesData Definition Files

Statistical software programs need to Statistical software programs need to know what to do with the data.know what to do with the data.

Data Definition Files “explain” the text file Data Definition Files “explain” the text file to the software programto the software program

For example a data definition file can For example a data definition file can format the pile of numbers into cases and format the pile of numbers into cases and variables, provide variable labels, define variables, provide variable labels, define missing cases, and moremissing cases, and more

Data definition files differ between Data definition files differ between software packagessoftware packages

SPSS Syntax FileSPSS Syntax File

Location of the data

Variable labels (as seen in the SPSS Variable View)

Variables in the data file

Missing values for each variable

Value labels assign descriptions to the values of variables

Data Definition Files and Data Definition Files and the Codebookthe Codebook

Where do the data definition files derive Where do the data definition files derive from?from?

……the Codebook!the Codebook!

OtherOther Statistical Software Statistical Software PackagesPackages

SASSAS

Geared towards Geared towards power users: one power users: one of the most of the most powerful statistical powerful statistical packages, but also packages, but also has the steepest has the steepest learning curvelearning curve

Relies more on Relies more on programming programming rather than a point-rather than a point-and-click. interfaceand-click. interface

OtherOther Statistical Software Statistical Software PackagesPackages

StataStata Combination of Combination of

command language command language and point and click and point and click interfaceinterface

Used by economics Used by economics departments and departments and other social science other social science disciplinesdisciplines

Known for its strong Known for its strong graphing capabilities graphing capabilities

OtherOther Statistical Software Statistical Software PackagesPackages

ShazamShazam

Canadian productCanadian product used widely in used widely in

economics/econometrieconomics/econometricscs

Not as powerful as Not as powerful as other statistical other statistical programsprograms

Runs on DOS, Runs on DOS, Windows, Mac, Unix Windows, Mac, Unix platformsplatforms

OtherOther Statistical Software Statistical Software PackagesPackages

MS ExcelMS Excel

Not a dependable Not a dependable statistical package, statistical package, but…but…

Widely availableWidely available Easy to understand & Easy to understand &

useuse

Tips for Successful Tips for Successful InteroperabilityInteroperability

Data definition filesData definition files By far the easiest way to format raw dataBy far the easiest way to format raw data SPSS, SAS, and STATA data definition files SPSS, SAS, and STATA data definition files

(with commenting!) are available in IDLS(with commenting!) are available in IDLS Troubleshooting tips:Troubleshooting tips:

Ensure you correctly identify the file path to the Ensure you correctly identify the file path to the datadata

Make sure that commands don’t include breaks Make sure that commands don’t include breaks (carriage returns)(carriage returns)

Check to make sure the correct symbol is used Check to make sure the correct symbol is used to separate commands (in SPSS it’s a period, in to separate commands (in SPSS it’s a period, in SAS & STATA a semi-colon)SAS & STATA a semi-colon)

Tips for Successful Tips for Successful InteroperabilityInteroperability

Comma-Separated Values (csv) files:Comma-Separated Values (csv) files: Text files (with the extension .csv) with Text files (with the extension .csv) with

commas separating the datacommas separating the data Often csv files imported into statistical Often csv files imported into statistical

software will require tweaking (variable software will require tweaking (variable labels, layout, etc.) labels, layout, etc.)

csv files can be imported by most csv files can be imported by most programs:programs:

SPSS, SAS, Stata, ExcelSPSS, SAS, Stata, Excel csv files are available in ESTAT and csv files are available in ESTAT and

CANSIM II through CHASSCANSIM II through CHASS b2020 files can also be converted to csv b2020 files can also be converted to csv

for use in another programfor use in another program

ExampleExample

SOFTWARE SOFTWARE (Windows)(Windows)

FILE TYPESFILE TYPES

SPSSSPSS Fixed Field DataFixed Field Data

Blank-delimited Data Blank-delimited Data

Comma-delimited DataComma-delimited Data

SPSS Portable FileSPSS Portable File

SASSAS Fixed Field DataFixed Field Data

Blank-delimited DataBlank-delimited Data

Comma-delimited DataComma-delimited Data

SAS Xport File & SAS Cport FileSAS Xport File & SAS Cport File

StataStata Stata 4-5 & 7Stata 4-5 & 7

Fixed Field DataFixed Field Data

Comma-delimited DataComma-delimited Data

Blank-delimited DataBlank-delimited Data

ShazamShazam Fixed Field DataFixed Field Data

Blank-delimited DataBlank-delimited Data

ExcelExcel Tab-delimited DataTab-delimited Data

Comma-delimited DataComma-delimited Data

File Input ChartFile Input Chart

Adapted from: http://www.chass.utoronto.ca/datalib/caq/format.htm

Conversion SoftwareConversion Software Conversion software allows you to Conversion software allows you to

seamlessly transport data from one seamlessly transport data from one statistical program to anotherstatistical program to another

STAT/TransferSTAT/Transfer Supports over 30 software programs, Supports over 30 software programs,

including SAS, SPSS and Stataincluding SAS, SPSS and Stata Approx. $150 USD for single user licenseApprox. $150 USD for single user license

DBMS/Copy DBMS/Copy Supports over 80 software programs, Supports over 80 software programs,

including databases and spreadsheetsincluding databases and spreadsheets Approx. $500 USD for single userApprox. $500 USD for single user

RoundupRoundup There are a proliferation of statistical There are a proliferation of statistical

software packages, all of them with software packages, all of them with different strengths and weaknessesdifferent strengths and weaknesses

Concentrate on getting the data into the Concentrate on getting the data into the software – often users can take it from software – often users can take it from therethere

CANSIM II at CHASS, ESTAT, IDLS, and CANSIM II at CHASS, ESTAT, IDLS, and the DLI website all offer different file type the DLI website all offer different file type options – it can be worthwhile checking options – it can be worthwhile checking different sources to find the file type different sources to find the file type you’re looking for you’re looking for