Telecooperation/RBG Technische Universität Darmstadt Copyrighted material; for TUD student use only Introduction to Computer Science I Topic 20: Streams.

Telecooperation/RBG

Technische Universität Darmstadt

Copyrighted material; for TUD student use only

Introduction to Computer Science ITopic 20: Streams and Input/OutputProf. Dr. Max MühlhäuserDr. Guido Rößling

Dr. G. RößlingProf. Dr. M. Mühlhäuser

RBG / Telekooperation©

Introduction to Computer Science I: T20

Outline

• Introduction to Input/Output Streams and Java Input/Output (I/O)

• Overview of Processing Streams

• Wrapping Streams and the Decorator Pattern

• User Defined Streams, StreamTokenizer and Random Access

2




Input / Output: Motivation

• So far: – Program input data is either coded in the program’s source

code, or passed as parameters from the console• define in Scheme• Variable declaration and initialization in Java• Actual arguments passed to functions in Scheme• args parameters to main

– Program output is shown on the display

• Problems: – Need to change the program in order to run it with different

data– “Closed world”:

• No interactivity with the outside world• No influence on the run is possible after the program

starts– Results cannot be stored

• This lecture discusses other ways of input/output 3




Input / Output (I/O) - Streams• Other sources / sinks for program data / results

– Keyboard entry (“standard I/O”), – Files on local machine ("file I/O")– Files/processes on the network ("network I/O")– Main memory ("memory I/O")

• To deal with all these sources/destinations uniformly, the concept of a data stream is introduced in Java– Input and output streams (I/O streams for short)

• Streams abstract from I/O “devices”– Hide details of the implementation and particularities of

operating various I/O devices • Keyboard, Files, Programs, Network, Memory, …

– Provide unified interfaces for read/write data access– Java program “talks to“ Java I/O stream objects

4




I/O-Streams

• In order to read data, an input stream is attached to a data source and the data is read element by element

5

• In order to write data, an output stream is attached to a data sink and the data is written element by element




Streams and delayed Lists

• The concept of streams is more general than just I/O

• We saw streams in Scheme– Delayed lists– We could create „virtual“ lists– We could link transparently data with first and rest

• These ideas fit well with I/O– Input: on access to the next list item, read the input

value– Output: on extension of the list, write the output value– Input stream delayed list without cons– Output stream delayed list without first/rest

6




Predefined streams in java.io

7

• Two orthogonal classifications:• According to the type of data• According to the structure of the stream




Classification of streams

• According to the type of the data– character streams read / write char (16-bit Unicode

character set)– byte streams read / write byte (8 bit):

• used to deal with binary data, e.g., images, sound, etc.

• According to stream structure– data streams: data is read/written directly from/to a

concrete source/sink device– processing streams: data is read from/written to

another stream • Data is filtered, buffered, manipulated, etc. after reading,

respectively before writing

8




Hierarchy of Stream Classes in java.io

9

• Character streams:– java.io.Reader/java.io.Writer provide the

interface and a partial implementation for character input/output streams

– Subclasses of Reader/Writer add/refine the implementation in Reader/Writer

• Byte streams:– java.io.InputStream/java.io.OutputStream

provide the interface and a partial implementation for reading/writing bytes

– All other byte streams are subclasses of InputStream/OutputStream




Character Stream Classes (Examples)

10

Reader

StringReader

InputStreamReaderInputStreamReader

FilterReaderFilterReader

FileReader

PushbackReaderPushbackReader

LineNumberReaderLineNumberReaderBufferedReaderBufferedReader

PrintWriterPrintWriter

StringWriter

FilterWriterFilterWriter

BufferedWriterBufferedWriter

Writer

OutputStreamWriterOutputStreamWriter FileWriter

shadows indicate processing streams




Byte Stream Classes (Examples)

11

InputStream

OutputStream

FileInputStream

PipedInputStream

FilterInputStreamFilterInputStream

ByteArrayInputStream

ObjectInputStreamObjectInputStream

SequenceInputStreamSequenceInputStream

StringBufferInputStream

LineNumberInputStreamLineNumberInputStream

BufferedInputStreamBufferedInputStream

PushbackInputStreamPushbackInputStream

DataInputStreamDataInputStream

FileOutputStream

PipedOutputStream

ByteArrayOutputStream

FilterOutputStreamFilterOutputStream

ObjectOutputStreamObjectOutputStream PrintStreamPrintStream

DataOutputStreamDataOutputStream

BufferedOutputStreamBufferedOutputStream






Data Streams: Overview

12

Type of Device Character Streams Byte Streams

Main Memory

Pipe

File

CharArrayReader,CharArrayWriter

StringReader,StringWriter

PipedReader,PipedWriter

FileReader,FileWriter

ByteArrayInputStream,ByteArrayOutputStream

StringBufferInputStream

PipedInputStream,PipedOutputStream

FileInputStream,FileOutputStream

• CharArrayReader/CharArrayWriter ByteArrayInputStream/ByteArrayOutputStream

– read from or write to an existing array in main memory • StringReader / StringWriter, StringBufferInputStream

– read from or write to Strings in main memory• FileReader/FileWriter FileInputStream/FileOutputStream

– read from or write to a file on disc




Common patterns of I/O-streams

• Working with streams follows a pattern, independent of the data type, source, or sink

• Reading• Open stream: implicit by creation• Read data as long as data is needed and available • Close stream

• After reading/writing, you should close() the stream!

• Writing• Open stream• Write data as long as required• Close stream

13




Common interfaces of I/O-streams

14

• Use I/O Streams only via common Interface• Information hiding via subtype polymorphism

InputStreampublic int read()public int read(byte[] bbuf)public int read(byte[] bbuf, int offset, int len)

Readerpublic int read()public int read(char[] cbuf)public int read(char[] cbuf, int offset, int len)

OutputStream public int write(int b) public int write(byte[] bbuf) public int write(byte[] bbuf, int offset, int len)

Writer public int write(int c) public int write(char[] cbuf) public int write(char[] cbuf, int offset, int len)




The java.io.Reader class

• public abstract int read(char[] c) throws IOException– reads the next Unicode characters into an array (c)– Returns the number of characters read, or -1 if the end of

the stream has been reached ("EOF“, End of File)– Any other problem causes an IOException,

• e.g., stream already closed, network connection lost, ...

15

For more details, see the API specificationhttp://java.sun.com/javase/6/docs/api/

public int read(char[] c) throws IOException public long skip(long n) throws IOException public void reset() throws IOExceptionpublic int close() throws IOException ...




Java Documentation

16




Java Documentation

17




File Streams

• File streams represent I/O streams from and to files in the file system:– FileReader for reading and FileWriter for writing

character-wise from/to a file– Similarly FileInputStream, FileOutputStream for

byte-wise reading/writing

• File streams are created by providing the source / destination file by means of:

– A file name (String)– A file object (java.io.File)– A file descriptor (java.io.FileDescriptor)

18




Class java.io.FileReader

19

Example: print file contents to screen (as int) and write 'a' to 'z' to a fileimport java.io.FileReader;import java.io.FileWriter;import java.io.IOException;

public class ReadWriteFile { // constructor etc.

public void readFrom(String fileName) throws IOException { FileReader in = new FileReader(fileName); int b;

while ((b = in.read()) != -1) System.out.print(b); in.close(); }

public void writeAToZ(String filename) throws IOException { FileWriter out = new FileWriter(filename); for (char c = 'a'; c <= 'z'; c++) out.write(c); out.close(); } // main() etc.}




Outline




• User Defined Streams, Stream Tokenizer and Random Access

20




Processing Streams• A processing stream contains another (data or processing)

stream – The latter is used as a data source or data sink, respectively

• Data might be transformed or functionality added – data buffering– counting lines– converting bytes and chars– compression– …

21

data drain

processing stream

data source

device data stream




Processing Streams: Overview

22

process character streams

byte streams

filtering

buffering

byte / charconversion

counting

datatypehandling

undoing

printing

FilterReader,FilterWriter

FilterInputStream,FilterOutputStream

BufferedReader,BufferedWriter

BufferedInputStream,BufferedInputStream

InputStreamReader,OutputStreamWriter

LineNumberReaderLineNumberInputStream

DataInputStream,DataOutputStream

PushbackReader PushbackInputStream

PrintWriter PrintStream

byte char

char byte




Roots of processing streams

• FilterReader/-Writer and FilterInputStream/FilterOutputStream are general super-classes for every processing stream• They encapsulate an internal stream in• Default implementations of an operation op:

pass on op() to the underlying stream• concrete filter (sub-)classes refine this functionality

23




Java Documentation

24




Buffered Streams

• A buffered stream temporarily stores data from another stream in an internal buffer

• Reading from a buffered stream fills the buffer when it is empty – Further "read"-operations will access the buffer

without reading from the attached stream– When the buffer is empty, additional data will be

read from the underlying stream• Writing to a buffered stream fills the buffer

before the attached data sink is written to– The buffer is also emptied if the stream is flushed

explicitly (flush()) or closed (close())

25




Buffered Streams

• BufferedReader and BufferedWriter– Subclasses of Reader/Writer, respectively

• BufferedInputStream and BufferedOutputStream– Subclasses of FilterInputStream / FilterOutputStream

26




Buffered Streams

2727

Data Sink ProgramBufferedWriter

bwrite

bbufwrite

write bbuf

write bbuf

flush




Buffered Streams: Performance

28

Reading a file of 2.5MB (x=buffer size, y=time [ms]) Buffer sizeTime[ms]

1 6433

2 3453

4 1935

8 988

16 471

32 365

64 363

128 87

256 327

512 65

1024 162

2048 214

4096 260

8192 61

16384 166

32768 159




Data Streams

29

• A data stream lets an application read/write primitive Java data types from/to an underlying I/O stream in a machine-independent way.

• An application uses a DataOutputStream to write data that can later be read by a DataInputStream.

FilterInputStream

DataInputStream

readBoolean(): booleanreadByte(): bytereadShort() : shortreadChar() : charreadInt() : intreadFloat() : float

FilterOutputStream

DataOutputStream

writeBoolean(boolean) : voidwriteByte(byte) : voidwriteShort(short) : voidwriteChar(char) : voidwriteInt(int) : voidwriteFloat(float) : void

<<interface>> DataInput

<<interface>> DataOutput




Data Streams: Example

30

import java.io.FileOutputStream; import java.io.DataOutputStream;import java.io.IOException;

public class DataStreamExample { String fileName = // some name; // some methods public void writeData() throws IOException { FileOutputStream fos = new FileOutputStream(filename); DataOutputStream out = new DataOutputStream(fos); out.writeInt(9); out.writeDouble(Math.PI); out.writeBoolean(true); out.close(); } // other methods public void readData() throws IOException { FileInputStream fis = new FileInputStream(fileName); DataInputStream in = new DataInputStream(fis); int i = in.readInt(); double d = in.readDouble(); boolean b = in.readBoolean(); in.close(); System.out.println("Read "+ i + ", " + d + ", and " + b+ "."); } }




Standard-I/O

• Package java.lang contains the class System with the following class variables: static in standard input

(keyboard)static out standard output (screen)static err standard error messages (screen)

• System.in yields an object of type InputStream

– enables to read() characters (bytes) from the keyboard

• System.out, System.err are of type PrintStream– offer print(...) and println(...) to output data to

the screen

31




PrintStream

32

FilterOutputStream

PrintStream

print(boolean) : voidprint(double) : voidprint(char) : voidprint(double) : voidprint(float) : void...println(Object): voidprintln(String): void...

• Adds the ability to conveniently print user-readable representations of various data values

• Never throws an IOException– Exceptional situations set an

internal flag that can be tested via the checkError method

• All characters printed by a PrintStream are converted into bytes using the platform's default character encoding – PrintWriter should be used

in situations that require writing characters rather than bytes.




Outline




• User Defined Streams, StreamTokenizer and Random Access

33




The Architecture of Java Streams• Streams can be wrapped around each other

– Abstraction layers, where underlying “primitive” streams are used by the enclosing (“higher”, more comfortable) streams

34

Data sink,e.g. file

FileOutputStream

BufferedOutputStream

ZipOutputStream

// create a buffered compressed output stream to a fileOutputStream out = new FileOutputStream(filename);out = new BufferedOutputStream(out);out = new ZipOutputStream(out); // ... more features can be dynamically added// the stream features are not visible for clients




The Architecture of Java Streams

• The technique that lets us combine streams at runtime is of more general interest. – A general way to extend objects with

new features dynamically

• In software technology, such kind of techniques are documented as design patterns.– Reusable, documented design ideas– More in the lecture „SE Design“

• The technique underlying the architecture of Java I/O streams is known as the Decorator Pattern.

35




Outline




• User Defined Streams, Stream Tokenizer and Random Access

36




User-Defined Processing Streams

• Sometimes it is useful to define streams that process (e.g., filtering, statistics, production of information chunks) data from other streams

• Best implemented by inheriting from FilterReader/FilterInputStream or FilterWriter/FilterOutputStream, respectively

• In the following example, a process stream filters out all lines which do not contain a certain substring Unix "grep" command

37




User-Defined Processing Streams

38

import java.io.BufferedReader;import java.io.FilterReader;import java.io.IOException;class GrepReader extends FilterReader { String substring; BufferedReader in; GrepReader(BufferedReader reader, String pattern) { super(reader); in = reader; substring = pattern; } // return the next line containing the search pattern String readLine() throws IOException { String line; while (((line = in.readLine()) != null) && (line.indexOf(substring) == -1)) ; return line; }}




User-Defined Processing Streamsimport java.io.*;public class Grep { public static void main(String[] args) { if ((args.length == 0) || (args.length > 2)) { System.out.println("Usage: java Grep <substring> [<filename>]"); System.exit(0); } try { Reader d; if (args.length == 2) d = new FileReader(args[1]); else d = new InputStreamReader(System.in); GrepReader g = new GrepReader(new BufferedReader(d), args[0]); String line; while ((line = g.readLine()) != null) System.out.println(line); g.close(); } catch (IOException e) { System.err.println(e); } }}

39

stream beingsearched

patternto look for




Random Access

• Until now, we have only considered sequential streams that could only be read or written sequentially

• Sequential streams are a good fit for sequential storage media, e.g. magnetic tapes

• Random Access Files allow for non-sequential access (random access) to the contents of a file

40




Motivating Example for Random Access

• How to extract one file?

41

• Let us regard the file format of a ZIP archive:– ZIP archives contain files and are usually

compressed to save disk space– Apart from the files, there is an additional entry at

the end of the file, the so-called dir-entry (for directory)

– The dir-entry is used to keep track of which files are contained and where – within the archive – they begin




Motivating Example for Random Access• Extracting a file using a sequential stream:

– Search the whole archive until the file is found– Extract the file

• On average: half of the entries are searched…

• Extracting a file using a random access stream:– Jump to the directory & read the entry of the file– Jump to the recorded position of the file– Extract the file

• Only two entities (directory & file) need to be read

• This approach is far more efficient

42




Random Access in Java

43

• Realized by the class RandomAccessFile• Can be used for reading and writing—implements

the interfaces DataInput and DataOutput

• Similar to FileInputStream and FileOutputStream, you open a RandomAccessFile on a file and pass a file name or a File object as a parameter

• Additionally, you have to pass a parameter to specify whether the file is opened for read-only or also for writing– You have to be able to read a file to be able to write to it




The Class RandomAccessFile

• Creation a RandomAccessFile for reading file “random.txt”:– new RandomAccessFile("random.txt", "r");

• Creating a RandomAccessFile for reading and writing on “random.txt”:– new RandomAccessFile("random.txt", "rw");

• Afterwards, read and write operations may be used to read and write data from/to the file

44




The Class RandomAccessFile

• RandomAccessFile supports a file pointer: points to the current position in a file

• Upon creation, the pointer is 0, i.e., beginning of the file• Calls to read and write operations automatically move

the file pointer by the number of read, respectively written bytes

45

• RandomAccessFile also supports positioning operations• int skipBytes(int n) throws IOException

moves read/write position by n bytes (relative positioning)• native void seek(long pos) throws IOException

positions read/write position right before pos (absolute positioning)• native long getFilePointer() throws IOException

returns the current read/write position




Stream Tokenizer

• java.io.StreamTokenizer supports the generation of tokens out of character sequences

• A token is obtained by calling nextToken()– Ignores Whitespace

• User can define what is to be regarded as whitespace– The type of the token read in is stored in the

attribute ttype

• The token types are explained on the next slide

46




StreamTokenizer: Token Types• Basic token types:

– StreamTokenizer.TT_NUMBER – number scanned; the number is stored in nval as a double

– StreamTokenizer.TT_WORD – a composed word is recognized which is stored in sval

– StreamTokenizer.TT_EOL – end of line, if the tokenizer is configured to recognize EOL as a token (eolIsSignificant(true))

– StreamTokenizer.TT_EOF – end of file

– Any other value is an encoding of the character read in

47




Customizing StreamTokenizer

• public void wordChars(int low, int hi)– Defines what is to be regarded as a word– All characters in the interval [low, hi] are "word parts“

• public void whitespaceChars(int low, int hi)– Defines whitespace– All characters in the interval [low, hi] are "space"

characters• public void eolIsSignificant(boolean

flag)– Defines whether the end of line character is of interest

• public void quoteChar(char c)– Defines quote character

• See the API documentation for more features…48




StreamTokenizer: Example

• Read a list of tea data with the following format:– Each line contains data for

one type of tea, i.e., the end of line token is of relevance

– Data items per tea type (separated by spaces)

• Tea name in quotes• Number of seconds

required• Number of recommended

tea spoons per liter

• What would be a complex program without further support, is easily written with the help of StreamTokenizer 49

"Ali Baba's 40 scents" 180 5

"Asatsuyu" 90 7"Generic Black Tea" 180 5"Caramel" 120 6"Ceylon Pekoe" 120 6"China Jasmin" 120 6"Chinese Love Dream" 150

5"One For All" 540 1"Cherry Cream" 120 6




StreamTokenizer Examplevoid parseTeas(InputStream is) throws IOException { StreamTokenizer stok = new StreamTokenizer(is); stok.eolIsSignificant(true); // EOL is important! stok.quoteChar('\"'); // quote char int token = 0; String teaName = null; int nrSecs, nrSpoons;

while ((token = stok.nextToken()) != StreamTokenizer.TT_EOF) { teaName = stok.sval; // 1. token (name) token = stok.nextToken(); nrSecs = (int)stok.nval; // 2. (secs) token = stok.nextToken(); nrSpoons = (int)stok.nval; // 3. (spoons) token = stok.nextToken(); // consume EOL System.out.println(teaName + "=>" + nrSpoons +

" spoons, seconds: " + nrSecs); }}

50Normally, you should check the type actually read in before assigning it!




Scanner

• Since Java 1.5, the Scanner class can be used for parsing

• A Scanner can be opened on a Stream, file, …– Typical use: Scanner sc = new Scanner(System.in);

• The class offers many helpful methods, such as…:– String next(); // returns the next token– x nextX(); // for x = byte, double, float, int, long, short,

Line• Throws InputMismatchException if the type is not found

– boolean hasNextX(); // same x as above• Checks if the next element would match type X

– You can redefine the delimiter, even to words– Additionally, regular expressions can be used for matching

• Check the API documentation for more information51




What else?

• Encrypted files, compressed files, files sent over internet connections, ...

• Exceptions! All I/O involves exceptions!– See part T16 for information on handling exceptions

52

try { statements involving I/O }catch (IOException e) { e.printStackTrace();}

Telecooperation/RBG Technische Universität Darmstadt Copyrighted material; for TUD student use only Introduction to Computer Science I Topic 20: Streams.

Documents

output streams io streams

t20 streams

data character streams

t20 iostreams

stream structure data

concept of streams

short streams

t20 input output io