Page 1
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.1
A first look at System TestingSelf Healing in Dependable Systems
Alfredo Benso & Paolo PRINETTOPolitecnico di Torino (Italy)
www.testgroup.polito.it
A quick look at Circuit Testing
Slide # 2.2 © Prinetto - 2003
Goals
– Presenting some of the major issues related to Digital Circuit Testing
Page 2
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.2
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.3 © Prinetto - 2003
Outline
– Introduction– Definitions– Testing a Circuit– Historical Evolution– Why testing– The basic approach to test– Testing diversifications– Conclusions.
Slide # 2.4 © Prinetto - 2003
How can I discover faults in my systems ?
Page 3
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.3
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.5 © Prinetto - 2003
Justtest them !!!
Slide # 2.6 © Prinetto - 2003
Test
Set of operations aiming at checking
whether a manufactured unit
works properly w.r.t. its specifications, or
not.
The result is:
• probabilistic
• valuable at the moment of the test, only
Page 4
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.4
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.7 © Prinetto - 2003
Test
Determine if a system is mission-ready.
If not, help establishing why not.
Diagnostic test
Slide # 2.8 © Prinetto - 2003
Capability of a product to be tested,
satisfying a set of given constraints in
terms of quality, cost, time, …
Testability
Page 5
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.5
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.9 © Prinetto - 2003
Capability of a design to guarantee that the final product will be testable, satisfying a
set of given constraints in terms of quality, cost, time,
…
Testability
Slide # 2.10 © Prinetto - 2003
Modify the logic in a way to make it easily testable
(e.g., Scan Design, BICS, Boundary
Scan, …)
Design for Testability
(DfT)
Page 6
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.6
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.11 © Prinetto - 2003
Outline
– Introduction– Definitions– Testing a Circuit– Historical Evolution– Why testing– The basic approach to test– Testing diversifications– Conclusions.
Slide # 2.12 © Prinetto - 2003
What is a fault?
• Impairments:– Faults– Errors– Failures
• Means• Attributes & Measures
Page 7
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.7
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.13 © Prinetto - 2003
Caveat
• A plenty of different definitions & view points exists !!!
Slide # 2.14 © Prinetto - 2003
A physical defect, imperfection, or flaw
that occurs within some hardware or
software components.
Fault
[Pradham 96]
Page 8
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.8
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.15 © Prinetto - 2003
A malfunction in a hardware, software or human component of
the system, which may introduce errors
and may lead to failures.
Fault
[Gibb_76]
Slide # 2.16 © Prinetto - 2003
Excited Faults
A fault is excited when a different behavior occurs, at the fault site, between the good and
the faulty machine.
Page 9
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.9
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.17 © Prinetto - 2003
Be careful !!!
Even if excited, a fault doesn’t necessarily show up, since the fault site can be not observable
from outside.
Slide # 2.18 © Prinetto - 2003
A different behaviour, between good and faulty machine, due to the existence of
one or more excited faults and observable
from the outside.
Error
Page 10
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.10
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.19 © Prinetto - 2003
A manifestation of a fault.
Error
Slide # 2.20 © Prinetto - 2003
A deviation from accuracy or correctness.
Error
[Pradham 96]
Page 11
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.11
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.21 © Prinetto - 2003
An invalid state of the system,
such as an incorrectly stored or transmitted items of
data.
Error
[Gibb_76]
Slide # 2.22 © Prinetto - 2003
It occurs when an error results in the system performing one of its functions
incorrectly.
Failure
[Pradham 96]
Page 12
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.12
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.23 © Prinetto - 2003
A fault condition in the system or a
functional unit that has influence on
service.
Failure
[Gibb_76]
Slide # 2.24 © Prinetto - 2003
No fault
A fault is present
but latent
The fault is excited
An error appears
A ≠ appears at the fault site
The ≠ reaches an observability point
The ≠ disappears but the fault is still present
The ≠ and the fault disappear
The fault disappears
A fault occurs
From Faults to Errors
Page 13
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.13
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.25 © Prinetto - 2003
3-universe model
• If no tolerance for specific faults exists, a fault may lead to a failure:
Fault(PhysicalUniverse)
Error(Informational
Universe)
Failure(External or User
Universe)
[Avizienis 82]
Slide # 2.26 © Prinetto - 2003
No fault
Error effects
System failure
A fault is present but latent
A fault is excited
An error appears
Page 14
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.14
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.27 © Prinetto - 2003
No fault
Error effects
System failure
A fault is present but latent
A fault is excited
An error appears
Partial failure
Slide # 2.28 © Prinetto - 2003
Degrades service, but does not interrupt it
completely.
Partial Failure
[Lan_86]
Page 15
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.15
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.29 © Prinetto - 2003
The ability of a system to
automatically decrease its level of
performance to compensate for hardware and/or software faults
Graceful degradation
Slide # 2.30 © Prinetto - 2003
No fault
Error effects
System failure
A fault is present but latent
A fault is excited
An error appears
Partial failure Total failure
Page 16
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.16
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.31 © Prinetto - 2003
Interrupts the service until the
system/functional unit is recovered or repaired or replaced
Total Failure
[Lan_86]
Slide # 2.32 © Prinetto - 2003
No fault
Error effects
System failure
A fault is present but latent
A fault is excited
An error appears
Partial failure Total failure Fail silent violation
Page 17
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.17
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.33 © Prinetto - 2003
The system/application produces incorrect
results, while it looks providing correct
ones
Fail silent violation
Slide # 2.34 © Prinetto - 2003
No fault
Error effects
System failure
The error is detected
A fault is present but latent
A fault is excited
An error appears
Page 18
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.18
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.35 © Prinetto - 2003
Error effects (cont’ed)
– it can be detected:. by one of the system Error Detection
Mechanism (e.g., hardware exceptions handling, software checks, …). The system reaches a safe state.
. during a test session.
Slide # 2.36 © Prinetto - 2003
No fault
Error effects
System failure
The error has no effect
The error is detected
A fault is present but latent
A fault is excited
An error appears
Page 19
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.19
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.37 © Prinetto - 2003
Error effects (cont’ed)
– It can have no effect, since:. it has been overwritten and thus
disappeared. it represents a potential Hazard: it is active
and could eventually have effects. it is not significant from the system
behavior point of view.
Slide # 2.38 © Prinetto - 2003
No fault
The ≠ is no longer on an observability
point
The ≠ and the defect disappear
The ≠ disappears but the defect is
still present
Error effects
System failure The error is latent
The error has no effect
The error is detected
A fault is present but latent
A fault is excited
An error appears
Page 20
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.20
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.39 © Prinetto - 2003
The length of time between the
occurrence of a fault and
the appearance of an error due to that fault.
Fault Latency
Slide # 2.40 © Prinetto - 2003
The length of time between the occurrence
of an error and
the appearance of the resulting failure.
Error Latency
Page 21
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.21
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.41 © Prinetto - 2003
Fault characteristics
• Cause• Nature• Duration• Extent• Value
Slide # 2.42 © Prinetto - 2003
Fault characteristics
• Cause (that which leads to the fault)– Specification Mistakes– Implementation Mistakes– External Disturbances– Physical Hardware Component Defects– Misuses
• Nature• Duration• Extent• Value
Page 22
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.22
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.43 © Prinetto - 2003
Fault characteristics
• Cause• Nature (relates to the intent of the cause of fault):
– Hardware– Software
• Duration • Extent• Value
Slide # 2.44 © Prinetto - 2003
Fault characteristics
• Cause• Nature• Duration (length of time for which the fault persists)
– Permanent– Intermittent– Transient
• Extent • Value
Page 23
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.23
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.45 © Prinetto - 2003
Fault characteristics
• Cause• Nature• Duration (length of time for which the fault persists)
– Permanent– Intermittent– Transient
• Extent • Value
It remains in existence indefinitely if no corrective
action is taken
Slide # 2.46 © Prinetto - 2003
Fault characteristics
• Cause• Nature• Duration (length of time for which the fault persists)
– Permanent– Intermittent– Transient
• Extent • Value It appears, disappears, and
then reappears repeatedly
Page 24
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.24
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.47 © Prinetto - 2003
Fault characteristics
• Cause• Nature• Duration (length of time for which the fault persists)
– Permanent– Intermittent– Transient
• Extent • Value
It can appear and disappear within a given period of time
Slide # 2.48 © Prinetto - 2003
Fault characteristics
• Cause• Nature• Duration• Extent (how far a fault propagates)
– Local– Global
• Value
Page 25
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.25
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.49 © Prinetto - 2003
Fault characteristics
• Cause• Nature• Duration• Extent • Value (consequence of the fault)
– Determinate– Indeterminate
Slide # 2.50 © Prinetto - 2003
Fault characteristics
• Cause• Nature• Duration• Extent • Value (consequence of the fault)
– Determinate– Indeterminate
e.g., the so called “Byzantine faults”
Page 26
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.26
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.51 © Prinetto - 2003
Byzantine faults
– It occurs when the system can fail & stop, or execute slowly, or execute at normal speed but produce erroneous values.
Slide # 2.52 © Prinetto - 2003
Genesis
Specificationmistakes
Implementationmistakes
ExternalDisturbances
ComponentDefects
SoftwareFaults
HardwareFaults
ErrorsSystemFailures
Page 27
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.27
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.53 © Prinetto - 2003
Intermittent faults
Design errors
Misuses
Transient faultsExternal
Disturbances
Permanent faultsComponentDefects
Design limit stressing
Errors
Slide # 2.54 © Prinetto - 2003
Misuses
• Operator error is the most common cause of failure
• Nevertheless many errors attributed to operators are actually caused by designs that require an operator to choose an appropriate recovery action without much guidance and without any automated help
Page 28
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.28
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.55 © Prinetto - 2003
ProductionIn-fieldoperation
Re-cycling
User’srequirements
DesignFaults can occur in any moment of
the product life cycle
Designerrors
Any problem
Randomdefects
Slide # 2.56 © Prinetto - 2003
Defect
The lack of something necessary or desirable
for completion or perfection; deficiency.
Page 29
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.29
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.57 © Prinetto - 2003
Physical Defects
Defects that can occur in the physical
structure of a system.
Slide # 2.58 © Prinetto - 2003
Physical Defects are obviously not numerable in
terms of type,
location, time of occurrence.
Page 30
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.30
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.59 © Prinetto - 2003
Hardware Fault model
A model of a defect to make it numerable, and thus tractable, making restricting
hypothesis on location and types.
Slide # 2.60 © Prinetto - 2003
Some Hardware Fault Models
– single (multiple) stuck-at– short (bridge)– stuck-on & stuck-open– slow-to-rise & slow-to-fall– single state-transition fault– coupling– pattern sensitive faults– functional– …
Page 31
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.31
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.61 © Prinetto - 2003
Stuck-at Faults
– Any given node in the network permanently stuck at 0 or 1.
FF1
FF2
PIPO
0
Slide # 2.62 © Prinetto - 2003
Selection conditioners
The fault model selection is heavily influenced by:– target system :
. technology
. type (IP core, chip, board, system; random logic, memory, microprocessor; …)
– available support tools (Automated Synthesis, Fault Simulation, ATPG, …)
– available test equipments (ATE, BIST) – ...
Page 32
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.32
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.63 © Prinetto - 2003
Status
The single stuck-at fault model, although introduced in 1958, is still a de-facto standard, since:
– it’s a metric– it’s technology independent– it’s easy to use– it’s managed by any CAD tool– …
Slide # 2.64 © Prinetto - 2003
Outline
– Introduction– Definitions– Testing a Circuit– Historical Evolution– Why testing– The basic approach to test– Testing diversifications– Conclusions.
Page 33
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.33
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.65 © Prinetto - 2003
How many Stuck at Faults?
• The total number of faults is 2N, where N is the number of gate terminals
= fault site
Slide # 2.66 © Prinetto - 2003
Equivalent Faults
• Let F1 and F2 be the functions performed by C in the presence of f1 and f2, respectively. Then faults f1 and f2 are equivalent if and only if F1 = F2
• Fault collapsing• Generate only one test for a group of equivalent
faults
s-a-0s-a-0
s-a-0
s-a-0 s-a-0 s-a-1 s-a-1
Page 34
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.34
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.67 © Prinetto - 2003
Testing of a Circuit
ab
c
d es-a-1
a = 1
b = 0
c = 0
G2G1
Slide # 2.68 © Prinetto - 2003
Controlling and Non-controlling Value
• Controlling value : when it present on at least one input of a gate, it forces the output to a known value– AND gate, NAND gate : 0– OR gate, NOR gate : 1
• Non-controlling value : the complement of (Sensitizing value) the controlling value– AND gate, NAND gate : 1– OR gate, NOR gate : 0
Page 35
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.35
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.69 © Prinetto - 2003
Automatic Test Generation
• Three steps– Set up (fault sensitizing)– Propagation (path sensitizing)– Justification (consistency check)
Slide # 2.70 © Prinetto - 2003
• The setup step is to produce a difference in the output signal at the gate where the fault is located between the two cases when the fault is present or it is absent
• D is called frontier
Test Generation (D-Algorithm)
D = 0 when fault occurs
1 no fault
H
H D01
stuck-at 1
s-a-1
Page 36
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.36
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.71 © Prinetto - 2003
Test Generation (D-Algorithm)
• The propagation step derives the D (or D) condition from the faulty gate to a output
JD0
1
stuck-at 1
D
1
H
Slide # 2.72 © Prinetto - 2003
0
F
G
H J
AB
CD
E
s-a-111
X
1
1
0
D
Test Generation (D-Algorithm)
• The last step is to force the logic values needed to sensitize the assumed fault from the primary inputs
Page 37
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.37
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.73 © Prinetto - 2003
– 1. excitation condition a = b = 1– 2. sensitization condition f = 0– 3. choose d = 1 ⇒ b = 0 (conflict)– try c = 1 (succeed)
• Backtracking : returning on one’s step and reversing a previous choice
Backtracking
s-a-1a
b
c
d
e
f
gG1G4
G3G2
Slide # 2.74 © Prinetto - 2003
1. excitation condition b = 02. sensitization condition c = 03. justification a =1 and b = 1 (conflict)
• There is no test for b s-a-1 fault• b is redundant• Replacing b by 1 ⇒ d = 0• The conflicting requirement derived from reconvergent
fanout (paths have a common source and a common sink)
Untestable Fault
G1 G2
s-a-1
ab
cd
Page 38
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.38
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.75 © Prinetto - 2003
Outline
– Introduction– Definitions– Testing a Circuit– Historical Evolution– Why testing– The basic approach to test– Testing diversifications– Conclusions.
Slide # 2.76 © Prinetto - 2003
First Design,then Test
Historical Evolution
Page 39
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.39
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.77 © Prinetto - 2003
Designers TestEngineers
Design
Slide # 2.78 © Prinetto - 2003
We cannot adopt Design for Testability techniques, since our system will be bigger and
slower. The resulting overhead is unacceptable for us !!
Page 40
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.40
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.79 © Prinetto - 2003
If you consider testability as part of the
specifications, its cost cannot be considered an
overhead
Slide # 2.81 © Prinetto - 2003
First Design,then Test
Design & Test
Historical Evolution
Page 41
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.41
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.82 © Prinetto - 2003
&Designers TestEngineers
Slide # 2.84 © Prinetto - 2003
First Design,then Test
Design & Test
TestableSynthesis
Built-In Self Testing&
Historical Evolution
Concurrent Engineering
Page 42
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.42
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.85 © Prinetto - 2003
Designers ARETest Engineers
Test responsibilities must be owned by
every member of the design team !!
Page 43
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.43
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.87 © Prinetto - 2003
Automated synthesis methodology that
considers testability as a design constraint.
Testable Synthesis
Slide # 2.88 © Prinetto - 2003
Automatic Testability enhancement
Test Synthesis
Page 44
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.44
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.89 © Prinetto - 2003
Outline
– Introduction– Definitions– Testing a Circuit– Historical Evolution– Why testing– The basic approach to test– Testing diversifications– Conclusions.
Slide # 2.90 © Prinetto - 2003
Why should I test ?
Page 45
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.45
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.91 © Prinetto - 2003
“If anything can go wrong …… it will !”
[Murphy]
Slide # 2.92 © Prinetto - 2003
“All customers are named Murphy !”
[Any test engineer]
Page 46
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.46
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.93 © Prinetto - 2003
Sooner or later nearly every computer
company suffers a glitch
Slide # 2.94 © Prinetto - 2003
Intel
– In 1994 Intel relented under pressure and replaced flawed Pentium processors.
– The chips contained an error that could have caused them to make mathematical mistakes.
– Cost:
480 M US $
≡ the overall EU Esprit budget for ’94 !!!
Page 47
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.47
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.95 © Prinetto - 2003
Apple
– In 1995, Apple Computer recalled Powerbook5300 laptops after some units burst into flames.
– A lithium ion battery was overheating. – Only 1,000 units had been shipped.
Slide # 2.96 © Prinetto - 2003
Toshiba
– In 1999 Toshiba settled a lawsuit alleging that the company sold 5 million defective laptops.
– The culprit was a semiconductor for controlling floppy drives.
– Cost: 2.1 G US $
Page 48
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.48
A first look at System TestingSelf Healing in Dependable Systems
Intel
– May 12, 2000 – Intel this week announced it has found defects in a chip
called the memory translator hub (MTH) used to route signals from Intel’s 820 chipset to the SDRAM in Pentium IIIs.
– Intel said noise in the MTH could cause PCs to reset, reboot, or freeze, and in some cases, cause data corruption. The company has not fixed the problem, but has offered to replace all defective motherboards with new ones fitted with Rambus memory, a faster memory than SDRAM.
– An Intel spokesman said about one million motherboardshave been shipped to end customers since November 1999 and could cost the company a few hundred million dollars. Analysts expect the recall to cost Intel $300 million and $400 million, placing it on a par with the infamous Pentium recall.
Slide # 2.98 © Prinetto - 2003
An Italian Car manufacturer
– 5,000 68HC11 processors, not burned-in, with bonding problems
– Mounted on cars– 2,000 cars sold before discovering the problem– 2,000 systems to replace:
. 1 year to find all of them
. global cost of 700 ML !!!
Page 49
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.49
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.99 © Prinetto - 2003
1
10
100
1.000
10.000
Device Board System In-field
Abstraction level
Co
st f
or
rep
laci
ng
a f
aulty
de
vice
The “Rule of 10”
Slide # 2.100 © Prinetto - 2003
According to the EU law, if a product damage you, you can
prosecute both the final manufacturer and the
manufacturers of all the sub-components
Page 50
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.50
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.101 © Prinetto - 2003
TEST to:• Improve quality• Improve yields • Reduce TTM, TTV, TT$• …
Time to Market
TTM
Time to Volume
TTV
Time to Money
TT$
RevenuesCosts
Profits
Page 51
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.51
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.103 © Prinetto - 2003
Outline
– Introduction– Definitions– Testing a Circuit– Historical Evolution– Why testing– The basic approach to test– Testing diversifications– Conclusions.
Slide # 2.104 © Prinetto - 2003
Which is the basic approach to
testing ?
Page 52
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.52
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.105 © Prinetto - 2003
Comparator
ReferenceSystem
Target SystemUnder Test
The basic approach to Testing
A propersequence of
values
Good / Faultyindication
Slide # 2.106 © Prinetto - 2003
The basic approach to Testing
FF1
FF2
PIPO
FF1
FF2
PIPO
0
=
Fault free
Faulty
TestSequences
/
Page 53
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.53
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.107 © Prinetto - 2003
Step #1
Off-line, generate the Test Sequences to be applied to
the UUT inputs.
Basic steps
Slide # 2.108 © Prinetto - 2003
Tools– ATPG (Automatic Test Pattern Generator)– Ad hoc Software – Ad hoc Hardware– Hand, brain, experience, – ...
Step #1
Off-line, generate the test sequences to be applied to
the UUT inputs.
Page 54
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.54
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.109 © Prinetto - 2003
Step #1
Off-line, generate the test sequences to be applied to
the UUT inputs.
Step #2
Off-line, determine the behavior of the reference (i.e., non faulty) unit when
Test Sequences are applied.
Basic steps
Slide # 2.110 © Prinetto - 2003
Practical approachStep #1
Off-line, determine the behavior of the reference (i.e., non faulty) unit when
Test Sequences are applied.
UUT descriptionTestsequences
Reference output
behavior
Page 55
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.55
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.111 © Prinetto - 2003
Basic steps
Off-line, generate the test sequences to be applied to
the UUT inputs.
Step #1
Step #2
Off-line, determine the behavior of the reference (i.e., non faulty) unit when
Test Sequences are applied.
Step #3
At test time:• apply Test Sequences to UUT
inputs
• compare UUT outputs with the reference output behavior
Slide # 2.112 © Prinetto - 2003
Basic steps
Off-line, generate the test sequences to be applied to
the UUT inputs.
Step #1
Step #2
Off-line, determine the behavior of the reference (i.e., non faulty) unit when
Test Sequences are applied.
Step #3
At test time:• apply Test Sequences to UUT
inputs
• compare UUT outputs with the reference output behavior.
Step #4
When a faulty unit is found:• UUT is repaired (if possible)
• Statistics are collected to fix the production and/or the test process.
Page 56
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.56
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.113 © Prinetto - 2003
How can we evaluate how good our tests
really are?
Slide # 2.114 © Prinetto - 2003
Carefully evaluate your Defect Levels
Page 57
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.57
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.115 © Prinetto - 2003
The % of faulty systems released as
good.
Defect level
Slide # 2.116 © Prinetto - 2003
Defect level
The Defect Level depends on :– the quality of the production process (yield)– the quality of the test (coverage).
Page 58
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.58
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.117 © Prinetto - 2003
The % of elements of a given set that are
considered to be defect free.
Yield
Slide # 2.118 © Prinetto - 2003
Be careful !!!
For TTM and TT$ reasons, in any new IC technology, the production starts when just 10% of the chips provided by the new process line works properly.
Page 59
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.59
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.119 © Prinetto - 2003
Coverage
Fault Coverage = # detected faults# possible faults
Defect Coverage = # detected defects# possible defects
Given:• a circuit• a set of possible faults• a test sequence
0% 100%
FC%
Slide # 2.120 © Prinetto - 2003
A test sequence detects (covers) a fault if the
values it generates on the UUT outputs when the fault is present are
different from the values generated when the
UUT is fault free.
Detected (or covered)
fault
Page 60
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.60
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.121 © Prinetto - 2003
Detected Faults
• A Test Pattern detects a single fault iff:– the fault is excited (opposite values on the fault
site)– the difference is propagated to at least one
output.
PIPO
01
Slide # 2.122 © Prinetto - 2003
Example of Defect Level evaluation
DL = 1 - Y(1-T)
where:– Y = process yield – T = coverage
[ T.Williams, N.Brown, IEEE Transactions on Computers, 1981 ]
Too approximated when the coverage is
estimated in terms of a single fault model, only.
Page 61
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.61
A first look at System TestingSelf Healing in Dependable Systems
Good part Y
Defective 1-Y
Defect coverage T
Defect level 1-P
Fault detected
Escapes
Slide # 2.124 © Prinetto - 2003
Outline
– Introduction– Definitions– Testing a Circuit– Historical Evolution– Why testing– The basic approach to test– Testing diversifications– Conclusions.
Page 62
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.62
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.125 © Prinetto - 2003
Is there just one kind of test or are there
alternatives to exploit ?
Slide # 2.126 © Prinetto - 2003
There is no“one-size fits-all”
solution !
Page 63
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.63
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.127 © Prinetto - 2003
Product Life Cycle
End-of-Production
Design
In-Field
Levels ofIntegration
Chip MCMBoa
rdSyst
em
Core
Technology
Analog
Memory
Processor
Random Logic
Testing Diversification (1)
Slide # 2.128 © Prinetto - 2003
When
Off-line Testing
On-line Testing
HowExterna
l Testin
g
Built-In
Self Tes
t
What
Structural Testing
Functional Testing
Parametric Testing
Testing Diversification (2)
Current-based Testing
Page 64
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.64
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.129 © Prinetto - 2003
• Let’s now focus on some of the above mentioned attributes
Slide # 2.130 © Prinetto - 2003
Product Life Cycle
End-of-Production
Design
In-Field
Levels ofIntegration
Chip MCMBoa
rdSyst
em
Core
Technology
Analog
Memory
Processor
Random Logic
Testing Diversification (1)
Page 65
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.65
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.131 © Prinetto - 2003
EOP test goals
– Minimize the Defect Level, by identifying faulty units
– Diagnose fault type and location– Improve:
. the production process
. the test process.
Slide # 2.132 © Prinetto - 2003
Fabricationprocess
Design
TestPASSPASS
Diagnosis
FAILFAIL
EOP test goals
Design improvements
Processimprovements
Product improvements
Page 66
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.66
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.133 © Prinetto - 2003
EOP test goals
Teststation
F, ~OK
Fix the production
process
F, ~OKFix theproduct
F, OKFix the test
process
FaultAnalysis& Repair
PASS
P, OKP, ~OK
FAIL
F, OKF, ~OK
Defect level
Slide # 2.134 © Prinetto - 2003
Product Life Cycle
End-of-Production
Design
In-Field
Levels ofIntegration
Chip MCMBoa
rdSyst
em
Core
Technology
Analog
Memory
Processor
Random Logic
Testing Diversification (1)
Page 67
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.67
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.135 © Prinetto - 2003
In-field test goals
– Detect Physical Defects, Fault and Errors as soon as possible
– Diagnose fault type and location– Identify faulty SRUs (Smallest Replaceable
Units).
Slide # 2.136 © Prinetto - 2003
When
Off-line Testing
On-line Testing
HowExterna
l Testin
g
Built-In
Self Tes
t
What
Structural Testing
Functional Testing
Parametric Testing
Testing Diversification (2)
Current-based Testing
Page 68
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.68
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.137 © Prinetto - 2003
When
Off-line Testing
On-line Testing
HowExterna
l Testin
g
Built-In
Self Tes
t
What
Structural Testing
Functional Testing
Parametric Testing
Testing Diversification (2)
Current-based Testing
Slide # 2.138 © Prinetto - 2003
Test performed when the system is idle.
Off-line Test
Page 69
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.69
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.139 © Prinetto - 2003
Test performed while the system is
normally working
On-line Test
• Concurrent, if it guarantees a zero error latency
• Not concurrent otherwise.
Slide # 2.140 © Prinetto - 2003
When
Off-line Testing
On-line Testing
HowExterna
l Testin
g
Built-In
Self Tes
t
What
Structural Testing
Functional Testing
Parametric Testing
Testing Diversification (2)
Current-based Testing
Page 70
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.70
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.141 © Prinetto - 2003
Structural test
It looks for faults that can occur in the
physical structure of the UUT.
Slide # 2.142 © Prinetto - 2003
Pro’s & Con’s
+ Tools available for automatic generation+ Precise Coverage evaluation+ Diagnostic capabilities+ Design independence− Applicable if the netlist is known, only.
Page 71
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.71
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.143 © Prinetto - 2003
Status
– One of standard EOP test performed by any IC manufacturer.
Slide # 2.144 © Prinetto - 2003
Functional test
It aims at checking the correct behavior of the target system, w.r.t. its specs, but regardless
its actual implementation
Page 72
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.72
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.145 © Prinetto - 2003
An uncompleted example
SHFTIN
RESET
CLOCK
– Reset– Shift four 1 and check that SHFTOUT be 0– Shift four 0 and check that SHFTOUT be 1
4 bit shift registerSHFTOUT
Slide # 2.146 © Prinetto - 2003
Pro’s & Con’s
+ Rather Easy to write+ No structural knowledge required+ Allow at-speed test− Very hard coverage evaluation− Specs dependence− Manual generation, only.
Page 73
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.73
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.147 © Prinetto - 2003
Status
– Usual approach to test memories and microprocessors, at the user level.
Slide # 2.148 © Prinetto - 2003
Functional Testverifies the correct functional behavior
at each level
Page 74
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.74
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.149 © Prinetto - 2003
Structural Testverifies the correct
implementationat each level
Slide # 2.150 © Prinetto - 2003
Parametric test
It aims at measuring the correctness of
some electrical entities of the target
system.
Page 75
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.75
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.151 © Prinetto - 2003
Status
– One of the standard approaches for both products and manufacturers qualification.
Slide # 2.152 © Prinetto - 2003
Test performed at the target nominal
frequency.
At-speed Test
Page 76
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.76
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.153 © Prinetto - 2003
Status
– 100% of microprocessors and memories are tested, at EOP, at-speed, too.
Slide # 2.154 © Prinetto - 2003
Experimental results
Data collected on 4,349 faulty devices of a sample of 26,415 dies.
[Maxwell et al, ITC’92]
3.1%61.1%
0.6%
0.8%
31.2%
2.8%0.4%
Current-based test
Functional test
Structural test
Page 77
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.77
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.155 © Prinetto - 2003
When
Off-line Testing
On-line Testing
HowExterna
l Testin
g
Built-In
Self Tes
t
What
Structural Testing
Functional Testing
Parametric Testing
Testing Diversification (2)
Current-based Testing
Slide # 2.156 © Prinetto - 2003
Test is performed resorting to an Automatic Test
Equipment (ATE)
External Testing
Page 78
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.78
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.157 © Prinetto - 2003
Controller
Comparator
UUT
Testsequences
Reference output
behavior
ATE architecture
Slide # 2.158 © Prinetto - 2003
The chip to be tested is inserted here !!
ATE for IC’s
Page 79
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.79
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.159 © Prinetto - 2003
ATE cost
From 3 to 6 M US $ !!
Test-per-pin architecture:5 K US $ / pin
Page 80
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.80
A first look at System TestingSelf Healing in Dependable Systems
ATE for boards
Bad-of-Nails
Some “Nails”
Page 81
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.81
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.163 © Prinetto - 2003
When
Off-line Testing
On-line Testing
HowExterna
l Testin
g
Built -I
n Self
Test
What
Structural Testing
Functional Testing
Parametric Testing
Testing Diversification (2)
Current-based Testing
Slide # 2.164 © Prinetto - 2003
Modify the logic in a way to make it
test itself.
Built-In Self Test
(BIST)
Page 82
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.82
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.165 © Prinetto - 2003
Migrate on board most of the ATE capabilities !!
Embedded ATE or
BIST(Built-In Self Test)
Slide # 2.166 © Prinetto - 2003
Embedded ATE
Is a total test solution embedded in silicon for test, diagnostic and measurement, at all the levels, from cores to systems, from EOP test to in-field maintenance.
Page 83
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.83
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.167 © Prinetto - 2003
LogicVision’s Solution
EmbeddedATE
(on chip)
Pattern GenerationResult Compression
Precision TimingDiagnostics
Power ManagementTest Control
Support forBoard-level Test
System-Level Test
(about 10k gates)
Logic
Processor, I/O, Audio, Video, Glue Logic, etc.
Mixed-Signal
PLL, ADC/DAC, Filter, Power Supplies, etc.
External ATE
Standard Digital TesterLimited Speed/Accuracy
Low Cost-per-Pin
Memory
SRAM, DRAM, ROM, Flash, FIFO, CAM, etc.
Reduced Pin-Count, Low Bandwidth External Interface
High-Bandwidth Internal Interfaces
I/Os & Interconnects
Drivers/Receivers, Boundary Scan, etc.
Very Deep Submicron Chip, SOC, Board or System
Slide # 2.168 © Prinetto - 2003
BIST costs
EmbeddedATE
(on chip)
Pattern GenerationResult Compression
Precision TimingDiagnostics
Power ManagementTest Control
Support forBoard-level Test
System-Level Test
(about 10k gates)
Logic
Processor, I/O, Audio, Video, Glue Logic, etc.
Mixed-Signal
PLL, ADC/DAC, Filter, Power Supplies, etc.
External ATE
Standard Digital TesterLimited Speed/Accuracy
Low Cost-per-Pin
Memory
SRAM, DRAM, ROM, Flash, FIFO, CAM, etc.
I/Os & Interconnects
Drivers/Receivers, Boundary Scan, etc.
Very Deep Submicron Chip, SOC, Board or System
10 K gate equivalent
Page 84
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.84
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.169 © Prinetto - 2003
BIST usage
BIST structures are today widely used: – in a big variety of “off-the-shelf” products:
. Microprocessors (Intel, Motorola, Toshiba, Sun, …)
. Personal computers
. Workstations
. Sawing machines
. Automotive applications
. …
Slide # 2.170 © Prinetto - 2003
BIST usage (cont’d)
– in almost all embedded memory IP cores– In a lot of SoC’s– …
Page 85
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.85
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.171 © Prinetto - 2003
Is it enough to test just at the
end of the overall production process?
Slide # 2.172 © Prinetto - 2003
NO !!!Remember
the “Rule of 10”.Test
sub-modulesasap.
Page 86
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.86
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.173 © Prinetto - 2003
1
10
100
1.000
10.000
Device Board System In-field
Abstraction level
Co
st f
or
rep
laci
ng
a f
aulty
de
vice
The “Rule of 10”
Slide # 2.174 © Prinetto - 2003
IC processing
Ship
WaferprobeTest
ICpackaging
PackagedIC Test
Burn-InPost
burn-inTest
An example of IC production test
Page 87
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.87
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.175 © Prinetto - 2003
Buy
Component Test
Bare-BoardTest
PC BoardAssembly
FinalSystem
AssemblyShip
ManufacturingProcess
Test
SystemTest
ProcessFunction
Test
Repair
Repair
An example of board production test
Slide # 2.176 © Prinetto - 2003
Outline
– Introduction– Definitions– Testing a Circuit– Historical Evolution– Why testing– The basic approach to test– Testing diversifications– Conclusions.
Page 88
© P.Prinetto 2003 - all rights reserved Vers. 1.0 2.88
A first look at System TestingSelf Healing in Dependable Systems
Slide # 2.177 © Prinetto - 2003
BE CAREFUL :If you don’t need testing
any longer, your microelectronic technology is not aggressive enough, and, quite soon, you’ll be
out of the market !!!
Slide # 2.178 © Prinetto - 2003
References
– M. Abramovici, M.A. Breuer, A.D. Friedman:Digital Systems Testing and Testable Design, IEEE Press, Piscataway, NJ (USA), 1995