Lesson 18 Using text files to share data with other programs

Python Mini-CourseUniversity of Oklahoma

Department of Psychology

Lesson 18Using text files to share data with

other programs

5/07/09Python Mini-Course: Lesson 181

Lesson objectives

1. Identify and describe the common file formats for data exchange, including text files, csv files, and tab-delimited files.

2. Read and write delimited text files in Python

3. Use a spreadsheet to read and create delimited text files


Data exchange

Key principle:Data must be stored in a common format

Industry standard formats for encoding textASCII, Unicode, etc.

Industry standard formats for exchanging dataDelimited text files


Delimiting

Method of marking the boundaries between data fields in databases, spreadsheets, and tabular data


Common delimited text files

White-space delimitedAny non-printable character code (space, tab, newline, etc)

Common in older systems and for numeric data (e.g. SAS data files)



Comma delimitedMost common formatFiles are designated as csv files (comma-separated values)

Example: USF normshttp://w3.usf.edu/FreeAssociation/



Tab delimitedExcellent format for tabular dataCan be read directly by ExcelSometimes called tsv filesExample: Substance Abuse and Mental Health Data Archivehttp://www.icpsr.com/SAMHDA/index.html


Example: sqrtable.py

# Create a table of squares and square roots

import math

start, stop, step = 1, 100, 1

delimiter = '\t'

filename = 'squares.txt'

num, sqr, sqr_rt = [], [], []

for val in range(start, stop+1, step):

num.append(val)

sqr.append(val**2)

sqr_rt.append(math.sqrt(val))


Example: sqrtable.py

# Save to a delimited text file

fout = open(filename, 'w')

hdr = 'Num%sSquare%sSqrRoot\n' % (delimiter, delimiter)

print hdr

fout.write(hdr)

for row in zip(num, sqr, sqr_rt):

line = '%d%s%d%s%2.4f\n' % \

(row[0], delimiter, row[1], delimiter, row[2])

print line

fout.write(line)

fout.close()


Exercises

1. Open the file square.txt with a text editor

2. Open the file square.txt with a spreadsheet application (e.g., Excel)

3. Create a similar spreadsheet and save as a text file


Example: readsqr.py

delimiter = '\t'

filename = 'squares.txt'

num, sqr, sqr_rt = [], [], []

fin = open(filename, 'r')

hdr = fin.readline().strip()

for row in fin:

line = row.strip()

entries = line.split(delimiter)

num.append(int(entries[0]))

sqr.append(int(entries[1]))

sqr_rt.append(float(entries[2]))

fin.close()


Example: readsqr.py

# Print a table of values

print hdr

for row in zip(num, sqr, sqr_rt):

line = '%d%s%d%s%2.4f' % \

(row[0], delimiter, row[1], delimiter, row[2])

print line


Lesson 18 Using text files to share data with other programs

Documents