L02 - Information 1 omp411 – Spring 2007 1/16/2007 What is “Computation”? Computation is about “processing information” - Transforming information from one form to another - Deriving new information from old - Finding information associated with a given input - “Computation” describes the motion of information through time - “Communication” describes the motion of information through space
42
Embed
L02 - Information 1 Comp411 – Spring 2007 1/16/2007 What is “Computation”? Computation is about “processing information” -Transforming information from.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
L02 - Information 1Comp411 – Spring 2007 1/16/2007
What is “Computation”?
Computation is about “processing information” - Transforming information from one form
to another - Deriving new information from old- Finding information associated with a
given input - “Computation” describes the motion of
information through time - “Communication” describes the motion of information through space
L02 - Information 2Comp411 – Spring 2007 1/16/2007
What is “Information”?
information, n. Knowledge communicated or received concerning a particular fact or circumstance.
Information resolves uncertainty. Information is simply that which cannot be predicted.
The less predictable a message is, the more information it conveys!
Carolina won again.
Tell me something
new…
“ 10 Problem sets, 2 quizzes, and a final!”
A Computer Scientist’s Definition:
L02 - Information 3Comp411 – Spring 2007 1/16/2007
Real-World Information
Why do unexpected messages get allocated the biggest headlines?
… because they carry the most information.
L02 - Information 4Comp411 – Spring 2007 1/16/2007
What Does A Computer Process?
• A Toaster processes bread and bagels
• A Blender processes smoothiesand margaritas
• What does a computer process?
• 2 allowable answers:– Information– Bits
• What is the mapping from information to bits?
L02 - Information 5Comp411 – Spring 2007 1/16/2007
Quantifying Information(Claude Shannon, 1948)
Suppose you’re faced with N equally probable choices, and I give you a fact that narrows it down to M choices. ThenI’ve given you
log2(N/M) bits of information
Examples: information in one coin flip: log2(2/1) = 1 bit roll of a single die: log2(6/1) = ~2.6 bits outcome of a Football game: 1 bit
(well, actually, “they won” may convey more
information than “they lost”…)
Information is measured in bits (binary digits) = number of 0/1’s
required to encode choice(s)
L02 - Information 6Comp411 – Spring 2007 1/16/2007
L02 - Information 10Comp411 – Spring 2007 1/16/2007
Huffman Coding
• A simple *greedy* algorithm for approximating an entropy efficient encoding1. Find the 2 items with the smallest probabilities2. Join them into a new *meta* item whose
probability is the sum3. Remove the two items and insert the new
meta item4. Repeat from step 1 until there is only one item
36/36
112/36
32/36
4/36 43/36
7/36
94/36
54/36
8/36
15/36
121/36
21/36
2/36
76/36
11/36
85/36
65/36
10/36
21/36
103/36
5/36
Huffman decodingtree
L02 - Information 11Comp411 – Spring 2007 1/16/2007
Converting Tree to Encoding
36/36
43/36
112/36
32/36
4/36
7/36
94/36
54/36
8/36
15/36
76/36
103/36
121/36
21/36
2/36
5/36
11/36
85/36
65/36
10/36
21/36
0
0
0
0
00
0
0
0
0
1
1
1
1
1
1
1
1
1
1
Huffman decodingtree
Once the *tree* is constructed, label its edges consistently and follow the paths from the largest *meta* item to each of the real item to find the encoding.2 - 10011 3 - 0101 4 - 011 5 - 0016 - 111 7 - 101 8 - 110 9 - 00010 - 1000 11 - 0100 12 - 10010
L02 - Information 12Comp411 – Spring 2007 1/16/2007
Encoding Efficiency
How does this encoding strategy compare to the information content of the roll?
Pretty close. Recall that the lower bound was 3.274 bits. However, an efficient encoding (as defined by having an average code size close to the information content) is not always what we want!
3063b
54433
333345b
ave
361
362
363
364
365
366
365
364
363
362
361
ave
.
L02 - Information 13Comp411 – Spring 2007 1/16/2007
Encoding Considerations
- Encoding schemes that attempt to match the information content of a data stream remove redundancy. They are data compression techniques.
- However, sometimes our goal in encoding information is increase redundancy, rather than remove it. Why?- Make the information easier to manipulate
(fixed-sized encodings)- Make the data stream resilient to noise
(error detecting and correcting codes)
-Data compression allows us to store our entire music collection in a pocketable device
-Data redundancy enables us to store that *same* information *reliably* on a hard drive
L02 - Information 14Comp411 – Spring 2007 1/16/2007
4
Error detection using paritySometimes we add extra redundancy so that we can detect errors. For instance, this encoding detects any single-bit error:
11 1111110 1111x1xThis difference is calledthe “Hamming distance”
“A Hamming distance of 1is needed to uniquelyidentify an encoding”
L02 - Information 17Comp411 – Spring 2007 1/16/2007
A Short DetourIt is illuminating to consider Hamming distances geometrically.
Given 2 bits the largest Hamming distancethat we can achieve between 2 encodings is 2.
This allows us to detect 1-bit errors if we
encode 1 bit of information using 2 bits.
With 3 bits we can find 4 encodings with a Hamming distance of 2, allowing the detection of 1-bit errors when3 bits are used to encode 2 bits of information.
We can also identify 2 encodingswith Hamming distance 3. Thisextra distance allows us to detect2-bit errors. However, we coulduse this extra separationdifferently.
00
10
01
11
000 001
010
100
011
111
101
110
Encodings separated bya Hamming distance of 2.
000 001
010
100
011
111
101
110
Encodings separated bya Hamming distance of 3.
L02 - Information 18Comp411 – Spring 2007 1/16/2007
Error correcting codes
We can actually correct 1-bit errors in encodings separated by a Hamming distance of 2. This is possible because the sets of bit patterns located a Hamming distance of 1 from our encodings are distinct.
However, attempting error correctionwith such a small separation is dangerous.Suppose, we have a 2-bit error. Our errorcorrection scheme will then misinterpretthe encoding. Misinterpretations alsooccurred when we had 2-bit errors in our1-bit-error-detection (parity) schemes.
A safe 1-bit error correction schemewould correct all 1-bit errors and detectall 2-bit errors. What Hamming distanceis needed between encodings toaccomplish this?
000 001
010
100
011
111
101
110
This is justa votingscheme
L02 - Information 19Comp411 – Spring 2007 1/16/2007
Information SummaryInformation resolves uncertainty
• Choices equally probable:• N choices down to M
log2(N/M) bits of information
•Choices not equally probable:• choicei with probability pi
log2(1/pi) bits of information• average number of bits = pilog2(1/pi) • use variable-length encodings
L02 - Information 20Comp411 – Spring 2007 1/16/2007
Computer Abstractionsand Technology
1. Layer Cakes2. Computers are translators3. Switches and Wires
(Read Chapter 1)
L02 - Information 21Comp411 – Spring 2007 1/16/2007
L02 - Information 26Comp411 – Spring 2007 1/16/2007
Why So Many Languages?
Application SpecificHistorically: COBOL vs. FortranToday: C# vs. Java
Visual Basic vs. Matlab Static vs. Dynamic
Code MaintainabilityHigh-level specifications are
easier to understand and modify
Code ReuseCode PortabilityVirtual Machines
L02 - Information 27Comp411 – Spring 2007 1/16/2007
Under the Covers
InputOutputStorageProcessing
DatapathControl
L02 - Information 28Comp411 – Spring 2007 1/16/2007
Under the Covers
InputOutputStorageProcessing
DatapathControl
L02 - Information 29Comp411 – Spring 2007 1/16/2007
Under the Covers
InputOutputStorageProcessing
DatapathControl
Cathode Ray Tube (CRT)The “last vacuum tube”Now nearing extinction
L02 - Information 30Comp411 – Spring 2007 1/16/2007
Under the Covers
InputOutputStorageProcessing
DatapathControl
Liquid Crystal Displays (LCDs)
L02 - Information 31Comp411 – Spring 2007 1/16/2007
Under the Covers
InputOutputStorageProcessing
DatapathControl
L02 - Information 32Comp411 – Spring 2007 1/16/2007
Under the Covers
InputOutputStorageProcessing
DatapathControl
L02 - Information 33Comp411 – Spring 2007 1/16/2007
Under the Covers
InputOutputStorageProcessing
DatapathControl
Intel Pentium III Xeon
L02 - Information 34Comp411 – Spring 2007 1/16/2007
Implementation Technology
RelaysVacuum TubesTransistorsIntegrated Circuits
Gate-level integrationMedium Scale Integration (PALs)Large Scale Integration (Processing unit on a
chip)Today (Multiple CPUs on a chip)
Nanotubes??Quantum-Effect Devices??
L02 - Information 35Comp411 – Spring 2007 1/16/2007
openclosed
Implementation Technology
Common Links?A controllable switchComputers are wires and switches
open
control
L02 - Information 36Comp411 – Spring 2007 1/16/2007
Chips
Silicon WafersChip manufactures build many
copies of the same circuit onto a single wafer. Only a certain percentage of the chips will work; those that work will run at different speeds. The yield decreases as the size of the chips increases and the feature size decreases.
Wafers are processed by automated fabrication lines. To minimize the chance of contaminants ruining a process step, great care is taken to maintain a meticulously clean environment.
L02 - Information 37Comp411 – Spring 2007 1/16/2007
Field Effect Transistors (FETs)
Modern silicon fabrication technology is optimized to build a particular type of transistor. The flow of electrons from the source to the drain is controlled by a gate voltage.
Source DrainGate
Bulk
n+ n+
p
IDS = 0
VDS
L02 - Information 38Comp411 – Spring 2007 1/16/2007
ChipsSilicon Wafers
Metal 2
M1/M2 via
Metal 1
Polysilicon
Diffusion
Mosfet (under polysilicon gate)
IBM photomicrograph (Si has been removed!)
L02 - Information 39Comp411 – Spring 2007 1/16/2007
How Hardware WAS Designed
20 years agoI/O Specification
Truth tablesState diagrams
Logic designCircuit designCircuit Layout
L02 - Information 40Comp411 – Spring 2007 1/16/2007
How Hardware IS Designed
Today (with software)High-level hardware specification languages
VerilogVHDL
L02 - Information 41Comp411 – Spring 2007 1/16/2007