7/23/2019 Lesson 6 - Advanced File IO Techniques
1/34
7/23/2019 Lesson 6 - Advanced File IO Techniques
2/34
Topics
A. File Format Comparison
B. Binary Files
C. TDMS Files
7/23/2019 Lesson 6 - Advanced File IO Techniques
3/34
A. File Format Comparison
3
At their lowest level, all files written to your
computers hard drive are a series of bits
Binary
TDMS
ASCII
7/23/2019 Lesson 6 - Advanced File IO Techniques
4/34
A. File Format Comparison
ASCII TDMS Direct Binary
Numeric
Precision
Good Best Best
Share
data
Best (Any program
easily)
Better (NI
Programs easily)
Good (only with
detailed meta
data)
Efficiency Good Best Best
Ideal Use Share data with other
programs when file
space and numeric
precision are not
important
Share data with
programs when
storing simple
array data and
metadata
Store numeric
data compactly
with ability to
random access
7/23/2019 Lesson 6 - Advanced File IO Techniques
5/34
B. Binary Files
5
Use Binary File functions to interact directly
with a binary file
7/23/2019 Lesson 6 - Advanced File IO Techniques
6/34
B. Binary FilesBits/Bytes?
6
A bit is a single binary value
Each bit is either on or off and is represented by a 1 or a
0
A byte is a series of 8 bits0 bit
00000000 byte
7/23/2019 Lesson 6 - Advanced File IO Techniques
7/34
B. Binary FilesStoringBoolean Values
7
LabVIEW represents Boolean values as 8-bit values
in a binary file
Eight zeroes represents False [00000000], and any
other value represents True [00000001],
[01000111], [11111111], and so on
Files are divided into byte-sized chunks, making
them much easier to read and process
7/23/2019 Lesson 6 - Advanced File IO Techniques
8/34
B. Binary FilesStoringBoolean Values
8
File Contents
Method A
00000001 00000001
00000000 00000001
00000000 00000001
Method B00101011
7/23/2019 Lesson 6 - Advanced File IO Techniques
9/34
B. Binary FilesStoring Integers
9
Binary Value U8 Value
00000000 0
00000001 100000010 2
11111111 255
7/23/2019 Lesson 6 - Advanced File IO Techniques
10/34
B. Binary FilesStoring Integers
10
Multi-byte integers are broken into separate bytes and
are stored in files in either little-endian or big-endian
byte order
Using the Write to Binary File VI, you can choosewhether you store your data in little-endian or big-
endian format
U32 Value Little-endian Value Big-endian Value
1 00000001
00000000
00000000
00000000
00000000
00000000
00000000
00000001
7/23/2019 Lesson 6 - Advanced File IO Techniques
11/34
B. Binary FilesStoringOther Data Types
11
Strings are stored as a series of unsigned 8-bit
integers, each of which has a value in the ASCII
Character Code Equivalents Table
This means that no difference exists between writing
strings with the Binary File Functions and writing them
with the Text File Functions
Clusters are best represented in binary files by using
Datalog Files
7/23/2019 Lesson 6 - Advanced File IO Techniques
12/34
B. Binary FilesStoring Arrays
12
Arrays are represented as a sequential list of the
elements
Element representation depends upon the element type
A header contains a 32-bit integer representing the size ofeach dimension
Example
A 2D array with a
header contains:row integer,
column integer,
then array data
7/23/2019 Lesson 6 - Advanced File IO Techniques
13/34
B. Binary FilesSequential/Random Access
13
Two methods of accessing data:
Sequential AccessRead each item in order, starting
at the beginning of a file
Random AccessAccess data at an arbitrary pointwithin the file
7/23/2019 Lesson 6 - Advanced File IO Techniques
14/34
B. Binary FilesSequential Access
14
To sequentially access all of the data in a file, you can
call the Get File Size function and use the result to
calculate the number of items in the file, based upon
the size of each item and the layout of the file
You can then wire the number of items to the count
terminal of the Read Binary VI
7/23/2019 Lesson 6 - Advanced File IO Techniques
15/34
B. Binary FilesRandom Access
15
Use the Set Position VI to set the read offset to the
point in the file you want to begin reading
The offset is in bytes; therefore, you must calculate
the offset based upon the layout of the file
7/23/2019 Lesson 6 - Advanced File IO Techniques
16/34
B. Binary FilesDatalog
16
Datalog is a specific type of binary file, designed for
storing a list of records to a file
Each record is represented by a cluster and can
contain multiple pieces of data with any data type
7/23/2019 Lesson 6 - Advanced File IO Techniques
17/34
B. Binary FilesDatalogRandom Access
17
7/23/2019 Lesson 6 - Advanced File IO Techniques
18/34
Exercise 6-1: Bitmap File Writer VI
18
Use Binary File I/O to write a file with a specified
format.
7/23/2019 Lesson 6 - Advanced File IO Techniques
19/34
C. TDMS Files
19
TDMS
Technical Data Management Streaming
Use TDMS files for the following purposes: To store test or measurement data
To create a structure for grouping your data
To store information about your data To read and write data at high speeds
7/23/2019 Lesson 6 - Advanced File IO Techniques
20/34
C. TDMS Files
20
TDMS file format:
TDMS file
Binary file (.tdms) that contains data and stores properties about
the data
TDMS_Index file
Binary index file (*.tdms_index) that provides consolidated
information on all the attributes and pointers in the TDMS file
Speeds up access to the data while reading
Automatically regenerated if lost
TDMS file format internal structure is publicly documented
7/23/2019 Lesson 6 - Advanced File IO Techniques
21/34
C. TDMS FilesData Hierarchy
21
Channel
Stores measurement signals or raw data in a TDMS file
Each channel can have properties describing the data
The data stored in the signal is stored as binary data ondisk to conserve disk space and efficiency
Channel Group
Segment of a TDMS file that contains properties and one
or more channelsUse channel groups to organize your data and to store
information that applies to multiple channels
7/23/2019 Lesson 6 - Advanced File IO Techniques
22/34
C. TDMS Files
22
Create TDMS Files in the following ways:
Use the Write to Measurement File and Read fromMeasurement File Express VIs
Allow you to quickly save and retrieve data from TDMSformat
Very little control over your data grouping and properties
Use the TDM Streaming API
Set of functions for opening, writing to, reading from, and
closing TDMS files Allows you to organize your data into channel groups and
channels
7/23/2019 Lesson 6 - Advanced File IO Techniques
23/34
C. TDMS FilesTDMStreaming API
23
7/23/2019 Lesson 6 - Advanced File IO Techniques
24/34
C. TDMS FilesWrite Data
24
Streams data to the specified TDMS file
Data subset to write is determined by group name and
channel name(s) input
7/23/2019 Lesson 6 - Advanced File IO Techniques
25/34
C. TDMS FilesRead Data
25
Reads the specified TDMS file and returns data
from the specified channel and/or channel group
7/23/2019 Lesson 6 - Advanced File IO Techniques
26/34
C. TDMS FilesSet Properties
26
Sets the properties of the TDMS file, channel
group, or channel
7/23/2019 Lesson 6 - Advanced File IO Techniques
27/34
C. TDMS FilesGet Properties
27
Returns the properties of the TDMS file, channel
group, or channel
7/23/2019 Lesson 6 - Advanced File IO Techniques
28/34
C. TDMS FilesFile Viewer
28
Opens TDMS file and presents the file data in
the TDMS File Viewer dialog box
7/23/2019 Lesson 6 - Advanced File IO Techniques
29/34
C. TDMS FilesGrouping Data
29
Carefully consider the best way to group your data,
because the data grouping can have a significant
impact on both the execution speed and
implementation complexity of writes and reads
Things you should consider when choosing a grouping
scheme include the original format of your data and
how you want to process or view the data
7/23/2019 Lesson 6 - Advanced File IO Techniques
30/34
Exercise 6-2: TDMS Reader VI
30
Learn how to read data from a TDMS file.
7/23/2019 Lesson 6 - Advanced File IO Techniques
31/34
SummaryQuiz
1. You need to store testresults and organize thedata into descriptivegroups. In the future, youneed to efficiently viewthe test results by group.Which file storage formatshould you use?
a) Tab-delimited ASCII
b) Custom binary formatc) TDMS
d) Datalog
2. You need to write aprogram which savesPortable NetworkGraphics (PNG) imagefiles. Which file storageVIs should you use?
a) Storage file VIs
b) Binary file VIs
c) ASCII file VIs
d) Datalog file VIs
31
7/23/2019 Lesson 6 - Advanced File IO Techniques
32/34
SummaryQuiz
32
3. You need to store data that other engineers will later analyze
with Microsoft Excel. Which file storage format should you
use?
a) Tab-delimited ASCII
b) Custom binary format
c) TDMS
d) Datalog
7/23/2019 Lesson 6 - Advanced File IO Techniques
33/34
SummaryQuiz
a) 00001010
00000000
00000000
00000000
b) 00000000
00000000
0000000000001010
c) 00001010
d)
0101000000000000
00000000
00000000
4. Which of the following is a little-endian representation ofan unsigned 32-bit integer (U32) with a value of 10?
7/23/2019 Lesson 6 - Advanced File IO Techniques
34/34
SummaryQuiz
True or False?
5. You can use the Binary File VIs to read ASCII files.
6. TDMS Files store properties only at the channel
or channel group level.