Page 1
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
PCI Express™ Architecture
1.0a Specification Technical Update
March 31, 2003, San Jose, CA
PCI Express™ ArchitecturePCI Express™ Architecture
1.0a Specification Technical Update1.0a Specification Technical Update
March 31, 2003, San Jose, CAMarch 31, 2003, San Jose, CA
Page 2
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
2
AgendaAgenda8:00 Coffee, Registration
8:30 PCI Express Architecture Overview Bhatt
9:00 PCI Express Base 1.0a Electricals Dunning/Schoenborn
10:45 PCI Express Base 1.0a Protocol Ajanovic/Harriman
12:30 Lunch (provided)
1:00 PCI Express Card 1.0a Stancil
2:00 PCI Express 1.0a Compliance Requirements Hosler
3:00 PM Break
3:15 PCI Express Mini Card 1.0 Specification Overview Shaw/Saunders
4:00 PCI Express Bridge 1.0 Specification Overview Willke/Ajanovic
5:00 Technical Q&A Bhatt, et al
Page 3
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
3
PCI Express RequirementsPCI Express Requirements
Next Gen MultimediaNext Gen Next Gen
MultimediaMultimedia
High Performance
High High PerformancePerformance
Ease of UseEase of UseEase of Use
AdvancedArchitectureAdvancedAdvanced
ArchitectureArchitecture
Scalable Width, FrequencyHigher Bandwidth
Differentiated servicesIsochrony Support
Layered Architecture => longevityReliability, Availability, ServiceabilityAdvanced Power Management
New Form Factors/Innovative DesignsHot Plug / Hot Swap
I/O Simplification
I/O I/O SimplificationSimplification
Consolidate the I/OUnify proliferated segmentsWorks in existing PCI Environment
* Other names and brands may be claimed as the property of others
Features for Multiple Market Segments and ApplicationsFeatures for Multiple Market Segments and Applications
Page 4
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
4
PCI Express SummaryPCI Express SummaryPhysical Interface:
Point-to-point full-duplexDifferential low-voltage signalingEmbedded clockingScaleable frequency (2.5 Gb/sec initially)Scalable width (1,2,4,8,12,16,32) Supports connectors and cables
Protocol: Load Store architectureFully packetized split-transaction
PCI Compatibility:Configuration model and PCI Software Driver model
Advanced Capabilities:Enhanced Configuration Advanced Power ManagementRAS: CRC Data Integrity, Hot Plug, Advanced error logging/reportingQoS and Isochronous support
USB2.0USB2.0
GraphicsGraphicsGraphics MemoryBridge
MemoryBridge
PCI PCI ExpressExpress
HDDHDDHDDPCIPCI
MemoryMemoryMemory
CPUCPUCPU
SIO
Serial AT ASerial AT A
GbEthernet*
GbEthernet*
Add insAdd ins
PCI ExpressPCI Express
I/OBridge
I/OBridge
Add insAdd ins
Add insAdd ins
LPCLPC
Page 5
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
5
Comm.Backplanes
Cables
Comm.Comm.BackplanesBackplanes
CablesCables
Card & Connectors
Card & Card & ConnectorsConnectors
ModulesModulesModules
Covers today’s Desktop and Server applicationsRequire no change to the system form factors
Opportunity to extend to back-plane and cabled applications
Break away from 20 years of legacy form factorsSuitable for Desktops, Mobile and ServersHot Pluggable/Swappable
MiniPCI Express
MiniMiniPCI ExpressPCI Express
Covers Mobile Comm. applicationsBuild to Order/Configure to Order
PCI Express Mechanicals ScopePCI Express Mechanicals Scope
Future
Page 6
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
6
Interoperability
YesNoNoNox16
YesYesNoNox8
YesYesYesNox4
YesYesYesYesx1
x16x8x4x1Slot
Card
Card Slotsx1x4
x8x16
PCI Express Cards & Slots
Page 7
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
7
Size optimized for single function cardHalf size of today’s Mini-PCI Type III card
Defined area for I/O connectorsLAN, modem, & antenna signals for wireless applications
51mm
30 mm30 mm
51mm51mm
61 mm61 mm
MiniMini--PCI Type III CardPCI Type III Card
Mini PCI Mini PCI Express #1Express #1
30 mm30 mm
allowallow22ndnd
cardcard
Mini PCI Mini PCI Express #2Express #2
PCI Express Mini CardsPCI Express Mini Cards
PCI Express Mini Card
Removes need for dual function cardsRemoves need for dual function cards
Page 8
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
8
NEWCARD* ModuleNEWCARD* ModuleNew module form factor from PCMCIA
Replacement for CardBus and PC Card2 NEWCARDs in space of 1 CardBus CardCards can use PCI Express or USB 2.0 interfaces
Mobile and Desktop Lower cost drives broader adoption Replaces PC Card slots starting with first PCI Express platforms
Application FocusLow wattage, small footprint, consumer friendlyCommunications, Storage, I/O MigrationNot intended for graphics
Single Wide
Double Wide
ExtendedDouble WideSingle Wide
For Desktop and Mobile ApplicationsFor Desktop and Mobile Applications
Page 9
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
9
Server I/O Module Server I/O Module Server I/O Module are a new I/O adapter form factor
Utilizing PCI Express as the host interface
Server I/O Modules are being designed to meet the challenges of current and future systems
Designed for closed chassis installation/removalImprove system I/O densityProvide greater flexibility chassis designs
PCI Today
SIOM
Hot plug IO moves from top (PCI today) to rear
Page 10
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
10
Server I/O Module ConceptServer I/O Module ConceptRack serversCritical technologies... GbE, FC, 10GbE, IBA
Full Module
Base Module
Page 11
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
11
Spec DevelopmentCompliance/Promotion
MarketAdoptionProduct Enabling
Exploration/ Planning
Time
PCI Express Spec RoadmapPCI Express Spec Roadmap
Base 1.0aBase 1.0aCard 1.0aCard 1.0a
Mini Card 1.0 Mini Card 1.0 –– Release Candidate in SIG member reviewRelease Candidate in SIG member reviewBridge 1.0 Bridge 1.0 –– Release Candidate in SIG member reviewRelease Candidate in SIG member review
NEWCARD Module NEWCARD Module –– W G in PCMCIAW G in PCMCIA
Server I/O ModuleServer I/O ModuleATCA Blade Module ATCA Blade Module –– W G in PICMGW G in PICMG
Ultra Low Power RequirementsUltra Low Power RequirementsCable RequirementsCable RequirementsGen2 Signaling RequirementsGen2 Signaling Requirements
• Initiative progressing from Spec to Industry Enabling
• Implementations on track for 2H’03 Tools and Silicon
•• Initiative progressing from Spec to Industry EnablingInitiative progressing from Spec to Industry Enabling
•• Implementations on track for 2H’03 Tools and Silicon Implementations on track for 2H’03 Tools and Silicon
Page 12
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
12
PCI Express 1.0a Base & CardPCI Express 1.0a Base & Card
Errata characterizationNotable changes: LTSSM, LFSR, Reset, WAKE#/Beacon
v1.0a significantly reduces product validation
v1.0a significantly improves interoperability
Compliance is based on 1.0a specificationsCompliance is based on 1.0a specifications
February ‘03 March ‘03 April ‘03
Final updates out for review
Final updates out for review
Review end (3/27/03)
Review end (3/27/03)
1.0a specs available
(week of 4/14)www.pcisig.com
1.0a specs available
(week of 4/14)www.pcisig.com
Page 13
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
13
PCI Express Compliance RoadmapPCI Express Compliance RoadmapQ1 ‘03 Q2 ‘03 Q3 ‘03 Q4 ‘03
Compliance Checklists
Compliance Checklists
Electrical test specs &
collateral
Electrical test specs &
collateral
Configuration test specs
Configuration test specs
PCI ExpressCompliance Workshop
PCI ExpressCompliance Workshop
Performed in PCI-SIG Compliance WorkshopsAdd-in card vendors test with all MB vendorsDemonstrated operation includes:
Properly recognized and configuredPerform some typical operations across PCI Express
Page 14
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
14
Ultra Low Power MotivationUltra Low Power MotivationPCI Express robustness in the desktop and server platforms requires powerful buffers to drive lengthy interconnect
Implementation of these buffers in the mobile platform brings with it thermal challenges
The mobile paradigm is power consciousnessYou pay to get it in (battery life)You pay to get it out (heat pipes and fans)
Leave the receiver specification unchanged
PCI express bus lengths in a mobile system may be restricted in length for particular buses
x16 Graphics is the best candidate for power savings due to width, and an understanding of OEM implementation
Page 15
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
15
Call to ActionCall to ActionGet involved with the PCI-SIG to shape the future of PCI Express technology
Attend the PCI-SIG Developers Conference, June 2-3, San Jose Convention Center
Align your products to the PCI Express Compliance & Interoperability programs in 2H03
Visit www.pcisig.com for more information on PCI Express architecture
Visit www.pcisig.com for more information on PCI Express architecture
Page 16
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
16
AgendaAgenda8:00 Coffee, Registration
8:30 PCI Express Architecture Overview Bhatt
9:00 PCI Express Base 1.0a Electricals Dunning/Schoenborn
10:45 PCI Express Base 1.0a Protocol Ajanovic/Harriman
12:30 Lunch (provided)
1:00 PCI Express Card 1.0a Stancil
2:00 PCI Express 1.0a Compliance Requirements Hosler
3:00 PM Break
3:15 PCI Express Mini Card 1.0 Specification Overview Shaw/Saunders
4:00 PCI Express Bridge 1.0 Specification Overview Willke/Ajanovic
5:00 Technical Q&A Bhatt, et al
Page 17
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
17
Physical Layer Physical Layer Basics
InterconnectSignaling
Errata SummarySelected Errata Details
Data Scrambling PolynomialLoopbackErrata/ClarificationsReceiver Eye ClarificationsReceiver Detect ClarificationsElectrical Idle ClarificationsFaulty Lane Link Configuration
BackupEmbedded clockingLink ConfigurationAC couplingLow Power States
– L0, L0s, L1, L2, L3Error Behavior/Recovery“0 volts Common Mode at Rx”ESD, EMI, Noise SuppressionReceiver DetectTesting Electrical Compliance
Page 18
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
18
Physical Layer ErrataPhysical Layer ErrataKey messages
The majority of the errata are intended to clarify the intent of rev 1.0, 1.0a Re-read the specEnsure you interpret the spec in
ways consistent with the inserted clarifying statements
Page 19
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
19
Physical Layer BasicsPhysical Layer BasicsLogical Functions
Encoding/decoding/scramblingReset, initialization, De-skewBuilt in Test ModesConfiguration:
– Speed, Link width, Lane mapping, Polarity
Electrical FunctionsTransmitter/ReceiverClocks/PLLs, Clock/Data RecoveryLink Power management
PHY layer upgrades do not affect upper layersPHY layer upgrades do not affect upper layersPHY layer upgrades do not affect upper layers
SoftwareSoftware
MechanicalMechanical
Data LinkData Link
TransactionTransaction
PhysicalPhysical
LogicalLogical
ElectricalElectrical
Page 20
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
20
PCI Express InterconnectPCI Express Interconnect
Dual simplex point to point topologyDifferential low voltage signals Bit rate: 2.5Gb/sec/lane/direction and beyondSelectable lane width: x1, x2, x4, x8, x12, x16, x32
Ref. Clock Ref. Clock
Dev
ice
A Device B
Selectable Width
Page 21
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
21
Signaling Signaling
DifferentialSuperior voltage margins to single-endedVoltage independence; VCC can varyReduces EMI challenges
Vcc
Gnd
Logical
ElectricalTX- TX+RX+ RX-
TransactionData Link
Phy
sica
l
CAP CAP
Logical
ElectricalTX- TX+RX+ RX-
TransactionData Link
Phy
sica
l
CAP CAP
Vcc
Gnd
Serial LanesRemoves skew requirements between parallel lanesRelaxed routing rules within a linkTopology: 4 layer FR-4 boards; ~20 inches, 2 connectors
Page 22
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
22
Pictorial SummaryPictorial Summary
+-
50Ω 50Ω 50Ω 50Ω
V
Detect
NoSpec Logic
Transmission-Line
Transmission-Line
ClockSource
ClockSource
+-
50Ω50Ω50Ω50Ω
V
Detect
NoSpecLogic
Transmission-Line
Transmission-Line
ClockSource
ClockSource
AC Coupling
Hot Attach/Detach
Emphasis on Low Power
Error Behavior/Recovery
0 V Common Mode at Rx pins
ESD, EMI, Noise Suppression
Testing and DebugInitialization andConfiguration
Page 23
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
23
Link Training & Status State MachineLink Training & Status State Machine
Page 24
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
24
1.0a Physical Layer Errata1.0a Physical Layer Errata1.0a Physical Layer Errata addresses feedback from the entire PCI-SIG Community
The PCI EWG made judgment calls on which issues to address as well as the exact wording of the errataThe majority of the issues were clarifications, but some functional changes were also made
Page 25
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
25
Errata Summary (1 of 2)Errata Summary (1 of 2)The LFSR scrambling polynomial interacted poorly with the 16 bit CRC covering the DLLPs
The scrambling polynomial has been changedLoopback Slave must retransmit exact 10-bits as received, and processes SKPs on a per Lane basis
Also clarified Loopback Slave entry and exit behaviorAdded clarification that the eye diagram at the silicon receiver is implementation specific vs. the mandatory minimum RX eye diagram measured at the pins using the compliance/measurement loadSuggested language around Receiver Detect to prevent a false DetectionClarified terms for an entry and exit debounce time from Electrical Idle
Page 26
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
26
Errata Summary (2 of 2)Errata Summary (2 of 2)Clarified multi-Lane Link configuration behavior when encountering a broken Lane during Polling/ConfigurationDefined specific methodology for deriving TX UITX common mode now RMS valueClarified that receivers shall be tolerant to receive and process SKIP ordered sets that have a maximum separation dependent on the Max_payload_size a component supportsClarified exiting RX.L0s conditions due to N_FTS timeoutFixed several typos in figures, tables, and text
Page 27
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
27
Data Scrambling ErrataData Scrambling ErrataOld polynomial interacted poorly with the 16 bit CRC covering the DLLPs
Probability of undetected data corruption at the physical layer deemed too high
Polynomial for scrambling LFSR changed
Was: G(X)=X16 +X15 +X13 +X4 +1
Is: G(X)=X16 +X5 +X4 +X3 +1
Page 28
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
28
50Ω50Ω
V
Loopback UsageLoopback Usage
50Ω50Ω
V
Tx
+-
50Ω 50Ω
Rx
Loopback Slave:•Sends back the exact 10 bits that are received•10 bit data is looped around even if it doesn’t map to a valid 8b/10b character•SKP symbols must be insert/deleted as required for clock drift•Exit via electrical idle ordered set or electrical idle
Tx
50Ω 50Ω
Rx
+-
Loopback Master:•Useful as BIST type engine
Data Pattern,Control, and
Compare Logic
Page 29
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
29
Loopback ErrataLoopback ErrataLoopback Slave must loop data back in “10-bit domain”
A Loopback Slave must transmit the exact 10 bits as receivedA Loopback Slave adds/deletes SKP symbols on a per Lane basis if two components have different clock references
– # of SKPs symbols can vary by one across a multi-Lane Link when looped back to the Master due to per Lane processing
Clarified that a Loopback Slave must transition to Loopback on a symbol boundary and may interrupt any ordered-set being transmittedClarified Loopback Slave exit rules:
– A Loopback Slave may transmit undefined 10-bit data to the Loopback.Master during the time after the Electrical Idle ordered is received and when the receiver actually detects Electrical Idle
– Before a Loopback Slave can exit Loopback it must transmit everything it received before detecting Electrical Idle
Page 30
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
30
Receiver Eye ClarificationReceiver Eye ClarificationClarified that the TX/RX Eye Diagram specifications are at package pins and not the silicon Pads
Eye at silicon pads is implementation specific and will not be the same as the eye at the package pins
– Package type and construction of “traces” matter– Eye at TX pads will generally be larger due to frequency
dependent package + silicon impedance– Eye at RX pads will generally be smaller due to frequency
dependent package + silicon impedanceReturn loss specs do not guarantee that silicon will operate across all channels which meet the loss, jitter budgets
We are working to create collateral to assist designers
Page 31
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
31
Receiver Eye ClarificationReceiver Eye Clarification
50Ω50Ω
Die and Package PKGPins
SiliconBumps
Full Swing0.8-1.2 VDiff P-P
De-Emphasized0.57-0.51 VDiff P-P
0.7 UI (Min.)
Tx
Rx
Tx Eye must be met at Package Pins with 50 Ohm test load
Electrical specs are implementation specific at Silicon bumps
50Ω50Ω
Die and Package Where PackagePins would be
SiliconBumps
0.175 VDiff P-P
0.4 UI (Min.)
Rx Eye must be met at where PackagePins would be with channel and 50 ohm test load
Electrical specs are implementationspecific at Rx Silicon bumps (not drawn)
Page 32
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
32
Clarifications were added to avoid a false receiver DetectReceiver detect sequence example:
Far end receiver is powered downReceiver Detect Step 1: The TX initial common mode is at least ~600mV above ground
– If step2 of Detect goes higher in voltage then a false Detect will occur– If step2 of Detect goes lower in voltage everything works OK
• Keeps biasing of ESD diode from fooling detectESD ESD
DiodesDiodes
Initial fast rampInitial fast rampuntil ESD diodesuntil ESD diodes
forward biasforward bias
Diodes’ forward Diodes’ forward resistance couples resistance couples
thru cap, slows rampthru cap, slows ramp
Slowing ramp allows Slowing ramp allows diode to discharge a bitdiode to discharge a bit
Renewed ramp Renewed ramp forward biases forward biases diodes againdiodes again
Reversed Reversed ramp biases ramp biases
diodes offdiodes off
Receiver Detect ClarificationsReceiver Detect Clarifications
Page 33
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
33
Electrical Idle ClarificationsElectrical Idle ClarificationsTransmitter debounce clarification:
TTX-IDLE-SET-TO-IDLE is considered a debounce time for the TX to meet Electrical Idle after transitioning from L0
Transmitter debounce addition:TTX-IDLE-TO-TO-DIFF-DATA is considered a debounce time for the TX to meet all TX specifications after leaving Electrical Idle
Receiver debounce clarification:TTX-IDLE-MIN is utilized by the receiver to start looking for an Electrical Idle Exit after successfully receiving an Electrical Idle ordered-set
Page 34
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
34
Faulty Lane BehaviorFaulty Lane BehaviorMany minor clarifications in LTSSM
Intent to not change behavior, but make required behavior more concise
Several minor clarifications added to the Polling and Configuration states to crisply define behavior when one lane within a multi-lane link is faulty
Intent to not change behavior, but make required behavior more clear and concise
Page 35
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
35
AgendaAgenda8:00 Coffee, Registration
8:30 PCI Express Architecture Overview Bhatt
9:00 PCI Express Base 1.0a Electricals Dunning/Schoenborn
10:45 PCI Express Base 1.0a Protocol Ajanovic/Harriman
12:30 Lunch (provided)
1:00 PCI Express Card 1.0a Stancil
2:00 PCI Express 1.0a Compliance Requirements Hosler
3:00 PM Break
3:15 PCI Express Mini Card 1.0 Specification Overview Shaw/Saunders
4:00 PCI Express Bridge 1.0 Specification Overview Willke/Ajanovic
5:00 Technical Q&A Bhatt, et al
Page 36
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
36
Protocol TopicsProtocol Topics1.0a Highlights
Errata Feedback
Examples of Interesting Cases
Buffer Sizing
Active State Power Management
Page 37
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
37
Base Specification 1.0a HighlightsBase Specification 1.0a HighlightsIncludes published errata
Majority of errata are minor corrections, clarifications & typos
Some more significant items Clarify terminology & workings of reset (C46)Fix error in TD/EP definition (C3)Data Link Layer clarifications (C27, C31)
Look for 1.0a specifications at www.pcisig.com
Page 38
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
38
Errata FeedbackErrata FeedbackRe: Optional DLLP Timeout Mechanism – Add corresponding status and mask bits to Advanced Error
Not defined as “reported error” - consider for a future spec revision.
Re: Read Completion Boundary register – Better, but still not clear for “all other system elements”
RCB applies to all completers for Memory RequestsRe: Clarify Request Handling in D1, D2, D3hot
In the D1, D2 and D3hot states, non-configuration requests are handled as Unsupported Request
– Configuration Requests are handled as in D0– Note that non-sticky configuration state may become undefined in
D3hot
Page 39
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
39
REPLAY_TIMER:
REPLAY_TIMER:
REPLAY_TIMER:
Example: REPLAY_TIMERCase 1 Example: REPLAY_TIMERCase 1
... eventually, the remaining TLPs are Ack’d
REPLAY_TIMER resets and holds – no outstanding unacknowledged TLPs
An Ack is received that acknowledges three of the transmitted TLPs
REPLAY_TIMER reset and restarted, because there are still outstanding unacknowledged TLPs
Six TLPs are transmittedREPLAY_TIMER started
RetryBuffer TLP
RetryBuffer
Ack
1, 2, 3...
...N, 0, 1...
RetryBuffer
Ack...N, 0
RetryBuffer
RetryBuffer
Page 40
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
40
REPLAY_TIMER:
REPLAY_TIMER:
REPLAY_TIMER:
Example: REPLAY_TIMERCase 2Example: REPLAY_TIMERCase 2
... eventually, the remaining TLPs are Ack’d
REPLAY_TIMER resets and holds – no outstanding unacknowledged TLPs
Nak acknowledges three of the transmitted TLPs...
REPLAY_TIMER resets and holds
Six TLPs are transmittedREPLAY_TIMER started
RetryBuffer TLP
RetryBuffer
Nak
1, 2, 3...
...N, 0
RetryBuffer
Ack...N, 0
RetryBuffer
RetryBuffer
... causes retransmission of other three
REPLAY_TIMER restartedREPLAY_TIMER: 1, 2, 3...
RetryBuffer TLP
Page 41
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
41
Example: REPLAY_NUMExample: REPLAY_NUM0) Two TLPs are in in the Retry Buffer. 1) Nak received - acknowledges nothing - REPLAY_NUM will be incremented
to 1, and the 2 TLPs in the retry buffer will be replayed2) Nak received - acknowledges first TLP but not second - the following will
happen:2.1) Because Nak acknowledges a TLP, will cause REPLAY_NUM to be reset to 02.2) Because still unack'd TLPs in retry buffer, REPLAY_NUM incremented to 1,
second TLP will be replayed3) Nak received - acknowledges nothing - REPLAY_NUM will be incremented
to 2, TLP in the retry buffer will be replayed4) Nak received - acknowledges nothing - REPLAY_NUM will be incremented
to 3, TLP in the retry buffer will be replayed5) Nak received - acknowledges nothing - REPLAY_NUM will be incremented
to 0 - this will trigger Link retraining - following retrain, TLP in retry buffer will be replayed
6) Ack received for TLP sent - no more outstanding unacknowledged TLPs
Page 42
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
42
Example: L2/L3 Ladder DiagramExample: L2/L3 Ladder Diagram
Configuration Write Request -> D3hot
PME_Turn_Off TLP
Block scheduling of new TLPs
Wait on Ack for last TLP(s)
Completes L2/L3 Ready transition: Disables Data Link Layer, Physical Layer to
electrical idle
If PME message(s) would send it(them)
Prepares for loss of power (if needed)Blocks scheduling of new TLPsWaits on Ack for last TLP(s)
PM_Enter_L23 DLLP sent upstream continuously
Disables Data Link Layer, Physical Layer to electrical idle
Upstream Component Downstream Component
Sends PM_Request_Ack DLLP continuously Wait on electrical idle
PME_TO_Ack TLP
...
...Waits for PM_Request_Ack DLLP
Page 43
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
43
Optimal Buffer Sizing and Management - MotivationOptimal Buffer Sizing and Management - Motivation
PerformanceSub-optimal buffers or buffer management policies will reduce performance
PriceBuffers take up space, potentially eliminating other value adding elements
Flow Control and Retry mechanismsSignificantly different from conventional PCIUnderstand details to avoid surprises
Bottom line: you want to get these right!
Page 44
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
44
Retry Buffer SizingRetry Buffer SizingTransmitter out of Retry Buffer space → Stop transmitting
All transmitted TLPs must be kept in Retry Buffer until Ack’d by the other component on the Link
– Note that this includes Completions as well as Requests
Much outbound traffic → Retry Buffer optimization critical
Ack policy and L0s also affect optimal Retry Buffer sizing
A
B
RetryBuffer
Page 45
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
45
Retry Buffer Sizing vs. Ack PolicyRetry Buffer Sizing vs. Ack PolicyOther component’s Acknowledgement (Ack) policy affects your component’s performance
Works both ways – be a “good citizen”Useful Retry Buffer maximum size determined by Ack latency of the other component on the Link
Ack latency bound sets buffer size guideline– Buffer Size =
F(Ack Latency, Flight Time, Ack Processing Delay, Link Width, MAX_Payload_Size, TLP Overhead)
– See Sect. 3.5.3.1, Table 3-5**
Tradeoff: Sending Ack’s more often than necessary wastes outbound bandwidth
OK if no other transmissions ready to send
Page 46
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
46
Avoid Stalls due to Slow L0s ExitAvoid Stalls due to Slow L0s Exit
Initially, both A→B and B→A are in L0s
A’s transmitter exits L0s and starts a long burst of write Requests
B’s transmitter initiates L0s exit to send Ack...
...but the A’s receiver L0s exit time is large, and A stalls due to lack of Retry Buffer space
A is stalled by A’s own slow L0s exit time
B→A returns to L0, B transmits an Ack DLLP to A, and the stall is resolved
A
B
L0s L0s
...WrWrWrWr
sta
ll
Ack
...WrWrWrWr
RetryBuffer
Page 47
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
47
Retry Buffer Sizing RecommendationsRetry Buffer Sizing Recommendations
Assume worst case Ack frequency from other component and size buffer accordingly (Defensive Buffer Sizing)
Send Ack’s when no other transmissions are ready & Anticipate need to send Ack – initiate L0s exit accordingly (Smart Acknowledgement)
Minimize receiver L0s exit latency
Re: Ack latency - Retry Buffer should be sized according to normal expected behavior
In general, reasonable to assume very small L0s exit times OK to let performance suffer for cases L0s exit times not small
Page 48
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
48
Receive Buffer Sizing andFlow Control Credit Return PolicyReceive Buffer Sizing andFlow Control Credit Return Policy
Receive Buffer sizing and Flow Control (FC) credit return policy have a significant effect on performance
Much inbound traffic → Receive Buffer optimization critical
Tradeoff: Bandwidth for Update Flow Control Packets (UpdateFCPs) vs. Receive Buffer Size
UpdateFCPs affect outbound traffic
Width & Max/Typical Payload size must be considered
A
B
RcvBuffers Update
FCPs
FC Gate
Page 49
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
49
Flow Control Credit Return PolicyFlow Control Credit Return PolicyFlow Control Credit Return (Update) policy = F( Receive Buffer Size, Round-Trip Delay )
Consider:Delay to transmit, potentially including L0s exitTime for other component to process updateCredit consumed by incoming TLPs “in flight”
Additional considerations for Completions:Outbound Non-Posted Requests determine performance requirementsKey factors: Requested data size, Number of Requests simultaneously pending
Balance Performance against CostSome applications may not require optimal behavior
– Example: Incoming control writes may be rare
A
B
RcvBuffers Update
FCPs
FC Gate
TLP
FCP
FCP
TLP
Com
plet
ion Read
Page 50
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
50
Traffic Types & Receive Buffer/Flow Control PolicyTraffic Types & Receive Buffer/Flow Control Policy
Posted Requests (Memory Writes)Many incoming writes → Optimize Performance
– Example: frame bufferTypical control operations → Optimize Cost
Non-Posted Requests (Reads)Many incoming reads → Optimize Performance
– Example: Root Complex (main memory)Typical control operations → Optimize Cost
CompletionsEndpoint/Root Complex: Match to outbound ReadsSwitch: Support full Link Bandwidth
Page 51
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
51
Active State Power ManagementActive State Power ManagementSoftware controlled power management (PCI-PM) provides mechanisms for intelligent PM
Important in conventional PCI, important in PCI ExpressPCI Express is PCI-PM compatible
Active State Power Management (ASPM) provides additional benefitLow latency – minimum impact on performanceFiner granularity of control – more opportunity for power savings compared to software controlled PM
Serial signaling technology consumes power when not sending data“idle” = logical idle, not electrical idle PCI Express uses ASPM to reduce power in logical idle
ASPM is important to minimize power consumption with minimum performance impact
Page 52
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
52
Active State Power Management Fills the GapsActive State Power Management Fills the Gaps
ASPM Provides “Transparent” Power SavingsTransition aggressively to lowest power mode
Software Controlled
Power Management
Time (order of microseconds)
L1 L1 L1 L1
Time (order of nanoseconds)L0s L0s
Active State Power
Management
Frac
tion
of L
0 Po
wer
Con
sum
ed in
R
educ
ed-P
ower
Mod
e (E
xam
ple)
Time (order of milliseconds)
L2 L2 L2
Page 53
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
53
Use Pipelining and Grouping to Maximize Power SavingsUse Pipelining and Grouping to Maximize Power Savings
Minimize handshake overhead to maximize bandwidth utilization → Reduce Power
Ack multiple packets at onceEnlarge Receive Buffers → Fewer FC updates
Time (nanoseconds)L0s
Time (nanoseconds)
Pipeline &Enlarge Payloads
Time (microseconds)
L1 L1
Time (microseconds)
Buffer Locally forGrouping
Page 54
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
54
Enabling, Entering and ExitingEnabling, Entering and ExitingASPM = Power Management for idle/on devices
Ability to enable depends on Endpoint latency requirements
– Goal: ASPM is transparent
Best to support both L0s and L1 ASPMIf L0s exit time is poor, L1 may be fasterExample: Two components use different reference clocks – L1 may provide quicker exit
Page 55
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
55
Enabling, Entering and ExitingEnabling, Entering and ExitingL0s:
Other component’s receiver exit latency low → Enter quickly when nothing to sendPoor exit latency → Use L1 instead
– Applies to own latency or other component’sAnticipate need to send and initiate exit earlyApply “look back”
– preferably to depth = exit timeL1:
Periodic traffic patterns → Schedule entry/exit– Do not go through L0s in these cases
Random traffic → Use idle timer, fast L1 recovery
Page 56
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
56
SummarySummarySize Retry and Receive Buffers carefully
Practice Defensive Buffer Sizing
Schedule Ack’s to minimize performance impactDo Smart Acknowledgement
Use ASPM Aggressively to save powerAvoid performance impact
Page 57
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
57
Non-Trivial Errata by AreaNon-Trivial Errata by AreaTransaction Layer: C1, C2, C3, C6, C14, C16, C17, C28, C45, C46, C50, C52 Data Link Layer: C15, C27, C31, C33Physical Layer: See Chapt 4 revised textPower Management: C30, C46, C47System: C13, C34, C46, C47Software/Configuration: C1, C7, C9, C10, C11, C18, C22, C23, C25, C29, C30, C39, C43, C46, C47, C48
Underline = More Significant
Page 58
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
58
Important ClarificationsImportant ClarificationsC1 – Max_Payload_SizeC2 – 4K Request RestrictionC9 – Extended Tag Field ClarificationC14 – Byte Count/Lower Address for Non-Mem CompletionsC22 – Bus Master EnableC23 – Use of Link Retrain & Link Training BitsC25 – Modifying Common Clock Config BitC34 – Error Logging ClarificationC39 – Clarify Config Bit UseC44 – Use of Max Read Request SizeC48 – Device 0 DecodeC53 – Assorted (particularly PM)
These errata do not change the spec,but address often misunderstood areas
Page 59
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
59
AgendaAgenda8:00 Coffee, Registration
8:30 PCI Express Architecture Overview Bhatt
9:00 PCI Express Base 1.0a Electricals Dunning/Schoenborn
10:45 PCI Express Base 1.0a Protocol Ajanovic/Harriman
12:30 Lunch (provided)
1:00 PCI Express Card 1.0a Stancil
2:00 PCI Express 1.0a Compliance Requirements Hosler
3:00 PM Break
3:15 PCI Express Mini Card 1.0 Specification Overview Shaw/Saunders
4:00 PCI Express Bridge 1.0 Specification Overview Willke/Ajanovic
5:00 Technical Q&A Bhatt, et al
Page 60
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
60
CEM Spec 1.0a ChangesCEM Spec 1.0a ChangesLane reversal, polarity inversion clarificationJTAG clarificationAdd-in card retention Signal naming consistencyInterconnect BudgetWAKE# and Beacon clarificationSMBus functional requirementGraphics add-in card power
Page 61
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
61
Lane Reversal, Polarity InversionLane Reversal, Polarity InversionThe plus and minus connections from the system board’s transmit differential pair (PETp/PETn) may be reversed
Simplification for board routingReceiver is required to support Lane Polarity Inversion
If a component does NOT support lane reversal then the board (system or add-in card) must adhere to strict connection ordering (i.e. Lane 0 to Lane 0, Lane 1 to Lane 1, etc) to the add-in card connector
If a component DOES support lane reversal then the same lane ordering must be used for both the transmit and receive pair
Page 62
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
62
JTAGJTAGAdded text detailing the JTAG interface
JTAG support is STILL optional for the add-in card and the system
Alignment with the PCI Local Bus specification
Page 63
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
63
Add-in Card RetentionAdd-in Card RetentionSoftened wording regarding the ridge along the add-in card connector
Clarified the term “retention ready” to mean that “the add-in card manufacturer must have selected (or created) a retention mechanism and made provisions on the card to facilitate the retention mechanism”
It does NOT say that the add-in card must ship with a retention solution (e.g. a clip)
Page 64
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
64
Signal Naming ConsistencySignal Naming ConsistencyChanged instances of “HSI…” and “HSO…” to PETpx, PETnx, PERpx, PERnx for consistency with the base spec
PET = PCI Express TransmitPER = PCI Express Receive“p” stands for positive “n” stands for negative“x” is the lane number
Eliminated Power Good (PWRGD) and replaced it with PCI Express Reset (PERST#)
Page 65
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
65
Interconnect BudgetInterconnect BudgetInterconnect link budget allocations are different for transmit and receive
Accounts for any contributions from the AC coupling capacitors
Clarified loss budget reference measurements and values
AC coupling capacitor Transmitter & package
System Board
System board interconnect
PC
I Express C
onnector
R = 50 Ω R = 50 Ω
Measurement Reference Point Aconnector.
Page 66
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
66
WAKE# and BeaconWAKE# and BeaconClarified that the add-in card specification requires use of WAKE# (rather than requiring Beacon) for devices that are wakeup capable
Required of system AND add-in cardsA device may ALSO support Beacon but the system is not required to support it
Does NOT mean that non-wakeup capable devices need to support WAKE# !!!
Wakeup capable systems MUST provide 3.3Vaux power!
Page 67
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
67
SMBus Functional RequirementSMBus Functional RequirementSMBus is STILL optional for the system and add-in cards
If supported, then SMBus 2.0 specifications must be adhered to
SMBus 2.0 does NOT require support for the Address Resolution Protocol (ARP, a plug ‘n’ play mechanism) but support for the ARP IS required for PCI Express add-in cards and systems
Aligns with the PCI Local Bus spec version 2.3
Page 68
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
68
Graphics Add-in Card PowerGraphics Add-in Card PowerChanging power/thermal max limit on full-height, x16 graphics add-in cards from 60W to 75W
ECR in progress
Re-defining pin B3 from RSVD (reserved) to +12VNew total of 5 +12V power pins for a 66W limitTotal current capacity of +12V and 3.3V connector pins is 76.89W BUT the max current draw is 75W (10W allowed to be drawn from the 3.3V rail)
Page 69
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
69
AgendaAgenda8:00 Coffee, Registration
8:30 PCI Express Architecture Overview Bhatt
9:00 PCI Express Base 1.0a Electricals Dunning/Schoenborn
10:45 PCI Express Base 1.0a Protocol Ajanovic/Harriman
12:30 Lunch (provided)
1:00 PCI Express Card 1.0a Stancil
2:00 PCI Express 1.0a Compliance Requirements Hosler
3:00 PM Break
3:15 PCI Express Mini Card 1.0 Specification Overview Shaw/Saunders
4:00 PCI Express Bridge 1.0 Specification Overview Willke/Ajanovic
5:00 Technical Q&A Bhatt, et al
Page 70
PCI-SIG Confidential70
PCI Express Compliance
Clear Test OutputClear Test OutputMapsMapsDirectly to Test Directly to Test SpecSpec
ChecklistsChecklistsDescribesDescribesDesign CriteriaDesign Criteria
C&I Test SpecsC&I Test SpecsInterpretInterpret
••ChecklistsChecklists••AssertionsAssertions
DefineDefineTest CriteriaTest Criteria
••Test DefinitionsTest Definitions
Test H/W & S/WTest H/W & S/WValidatesValidatesTest CriteriaTest Criteria
••ComplianceCompliance••InteroperabilityInteroperability
PASS
FAILTest ToolsTest ToolsAnd ProceduresAnd Procedures
ComplianceChecklists
C&I Test Spec
Workshops
Compliance defined by PCI-SIG WGsCompliance defined by PCI-SIG WGs
Page 71
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
71
ChallengesChallengesVarious categories of systems
Desktops, servers, mobile, embedded
Many flavors of peripheralsCard x1, x4, x8, x16, mobile form factors, modules
Multiple processor and OS categories
Different pre-boot environmentsBIOS, EFI
Focus on volume platforms initiallyFocus on volume platforms initially
Page 72
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
72
Provide design-time ‘rules’ that implementations should follow
Checklists for Root Complex, Endpoint, Switch, addincard, and motherboard
Simple set of ‘yes/no’ questions
PHY.2.6#1
Training sequence ordered-sets are never scrambled but always8b/10b encoded.
yes ___ no___
The receiver terminations must remain enabled in Electrical Idle. yes ___ no___
The Beacon signal must contain minimum width pulses >= 2 ns. yes ___ no___
ChecklistsChecklists
Page 73
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
73
Clarify description of what is being tested
Contains:Assertions
– Prioritized set pulled from checklistsTest Descriptions
– What a test does, and what assertions it checks
Test SpecificationsTest Specifications
Page 74
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
74
Describes how to run testsRequired equipmentStep-by-step instructions
Detailed procedures help ensure repeatabilityAt Compliance WorkshopsIn development labs
Test ProceduresTest Procedures
Page 75
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
75
Electrical Compliance testing is crucialHelps ensure interoperability
– Prevents subtle issues– Prevent problems with untested silicon/product combinations
Key to successful ramp of PCI Express ready platforms and devices
Electrical Testing includesSignal quality
– Upstream and downstream signaling– Eye pattern, jitter and bit rate– with and without Spread Spectrum Clocking– De-emphasis and transition bit eye diagrams
Electrical Layer TestsElectrical Layer Tests
Page 76
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
76
Compliance load board (Platform Signal Quality)
Compliance baseboard(Add in Card Signal Quality)
Electrical Test FixturesElectrical Test Fixtures
Page 77
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
77
• On device under test, place 50 ohm termination on TX data lines
• Device enters polling.compliance sub-state
• Capture waveform on oscilloscope• Run eye analysis software
OscilloscopeOscilloscope
CLB Motherboard
Sample System Hook-up
Page 78
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
78
Standalone Eye Diagram .EXETransition/Non Transition Masks
Transition bitsDe-emphasis bits
Supports Spread Spectrum ModulationSupports Common data file formats
.tsv
.csvAllows text file mask/parameter configuration
FlexibleSupports multiple technologies
Signal Quality SoftwareSignal Quality Software
Page 79
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
79
FPGA RA
MPGM
Mid-bus Probe
Mid-bus Probe
USBTest
Control
PCI Express
USB
Transaction Layer
Link Layer
Physical Layer
DUTTest Control Initiates Test Traffic
Traffic is Monitored and Injected Through the USB Port
Mid Bus Probe Points Allow Any Protocol Analyzer to be Connected for Broader Test Analysis
Protocol ComplianceProtocol Compliance
Page 80
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
80
Flow Control Credit Operation
Initialization
Management
Legacy InterruptsProper transaction format and protocol
Data payload variationsTransaction Timeouts
Transaction Layer Test AreasTransaction Layer Test Areas
Page 81
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
81
NAK and timeoutsEnsure that the Device Under Test (DUT) can respond to NAKs withretries, expected timeouts on ACK/NAKsLink retraining without losing contents of link buffers and link state
Error ConditionsGenerate CRC and bad data bit errors - verify the responsesGenerate undefined encoding and corrupt reserved fields
Propagation of ResetAcross secondary ports of a Switch from primary side
Data Link Layer Test AreasData Link Layer Test Areas
Page 82
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
82
Check config space register characteristics and default valuesOne test for each new capability structure
PCI Express CapabilityDevice Capabilities, Control, and StatusLink Capabilities, Control, and StatusMSIAdvanced Error ReportingVirtual Channel Device Serial NumberPower Budgeting
Configuration Register ChecksConfiguration Register Checks
Page 83
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
83
System UnderTest USB
Compliance Test Card
ControlSystem
It Is Possible To Run Most Tests With Control System Only Software
Platform BIOS Test SetupPlatform BIOS Test Setup
Page 84
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
84
Platform and protocol tests
released
Electrical and Config tests
released
Feb ’03 Mar ‘03Jan ‘03 Apr ‘03 May ‘03 Jun ‘03 July ‘03 Nov ‘03Aug ‘03 Sept ‘03 Oct ‘03 Dec ‘03
Q1 ‘03 Q2 ‘03 Q3 ‘03 Q4 ‘03Feb ‘03Jan ‘03 Mar ‘03
Q1 ‘04
Compliance Checklists published
First PCI ExpressCompliance Workshop
Compliance TimelineCompliance Timeline
Page 85
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
85
AgendaAgenda8:00 Coffee, Registration
8:30 PCI Express Architecture Overview Bhatt
9:00 PCI Express Base 1.0a Electricals Dunning/Schoenborn
10:45 PCI Express Base 1.0a Protocol Ajanovic/Harriman
12:30 Lunch (provided)
1:00 PCI Express Card 1.0a Stancil
2:00 PCI Express 1.0a Compliance Requirements Hosler
3:00 PM Break
3:15 PCI Express Mini Card 1.0 Specification Overview Shaw/Saunders
4:00 PCI Express Bridge 1.0 Specification Overview Willke/Ajanovic
5:00 Technical Q&A Bhatt, et al
Page 86
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
86
What is Mini PCI Express?What is Mini PCI Express?Replacement for Mini PCI
Targeted for BTO/CTO solutions
PCI Express and USB 2.0 enabledOptimized for communication add-ins
Card envelope: 30mm x 56mm x 5mmEqual to ½ width of Mini PCI Type IIIa card
Page 87
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
87
When and WhoWhen and Who
Draft 60-DayReview
Candidate1.0
Final1.0
11/5/02 3/21/03 5/27/03 6/1/03
WG voting members:Agere Systems, AMD, Dell Computer, FCI, Foxconn, HP, IBM, Intel, Microsoft, Molex, Qualcomm
WG observer members:LSI Logic, NEC
Page 88
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
88
Communications CentricCommunications Centric
Page 89
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
89
Targeted ApplicationsTargeted ApplicationsWireless-Personal Area Network (W-PAN)
Bluetooth / Ultra wideband
Local Area Network (LAN)10/100/1G/10G Ethernet
Wireless-LAN (W-LAN)802.11b/g/a, etc.
Wide Area Network (WAN)V.90/V.92 modem / xDSL / cable modem
Wireless-WAN (W-WAN)GSM/GPRS / UMTS / CDMA
Page 90
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
90
Half the Size of Mini PCIHalf the Size of Mini PCI
Mini PCI(Type IIIa)
Mini PCIExpress
3030
5151
Page 91
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
91
Upgradeability / ServiceabilityUpgradeability / ServiceabilityAngled insertion and removal
OEM optimized retentionInternal clips / screws / door attached clip
BTO / CTOSingle connectorMultiple technologies
Field replacement by service technicians
Reduce TCO / services costs
Page 92
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
92
Mechanical Summary (1)Mechanical Summary (1)
Card outline dimensioning
Cross-section
Page 93
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
93
Mechanical Summary (2)Mechanical Summary (2)
Keep out zones
I/O connector zone
Page 94
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
94
Mechanical Summary (3)Mechanical Summary (3)
Two socketarrangement
definedSystemboardfootprint
Page 95
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
95
Signal SummarySignal Summary
Page 96
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
96
Pin ArrangementPin Arrangement
PCIExpress
REFClock
USB
52 pin solutionArranged to
assure isolation
Page 97
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
97
Signal IntegritySignal Integrity
Generation 1 PCI Express requirements
Sample connectors validated to these specs
Page 98
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
98
Power and ThermalsPower and Thermals
Power Density Uniform Loading@80% coverage
00.010.020.030.040.050.060.070.080.090.1
0.095 0.115 0.135 0.155 0.175 0.195 0.215
Top Side (W/Sq. cm)
Bot
tom
Sid
e (W
/Sq.
cm
)
Power Density Uniform Loading@80% coverage
00.010.020.030.040.050.060.070.080.090.1
0.095 0.115 0.135 0.155 0.175 0.195 0.215
Top Side (W/Sq. cm)
Bot
tom
Sid
e (W
/Sq.
cm
)
2.3W MAXthermal
dissipation
3W MAX powerconsumption
Page 99
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
99
Designed for Power EfficiencyDesigned for Power EfficiencyRobust power management features
ACPI and PCI PM supportedIn-band wake mechanisms supportedWake# – enables lowest system power solutionenables lowest system power solutionSMBus – available for advanced featuresavailable for advanced features
Dual power planes3.3V – nominal voltage required for I/O drive requirementsnominal voltage required for I/O drive requirements1.5V – reduces need for onreduces need for on--card regulationcard regulation
Two power statesPRIMARY – 3.3V and 1.5V fully ONAUXILIARY – 3.3VAUX available on +3.3V pins in D3HOT
Page 100
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
100
Status IndicatorsStatus IndicatorsThree LEDs
W-PANW-LANW-WAN
Single-ended,9 mA sinkcapable
LED support via I/O connector
Still an option
Page 101
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
101
Ease of DesignEase of DesignDigital / Analog physical separation
RF is not as near to digital– High speed digital on host connector– Analog on I/O connectors
Spread Spectrum Reference clock supports
Reduced EMI emissions
Software compatibilityPer native bus definitions
– PCI Express– USB 2.0– SMBus 2.0
ANALOG
DIGITAL
Page 102
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
102
AgendaAgenda8:00 Coffee, Registration
8:30 PCI Express Architecture Overview Bhatt
9:00 PCI Express Base 1.0a Electricals Dunning/Schoenborn
10:45 PCI Express Base 1.0a Protocol Ajanovic/Harriman
12:30 Lunch (provided)
1:00 PCI Express Card 1.0a Stancil
2:00 PCI Express 1.0a Compliance Requirements Hosler
3:00 PM Break
3:15 PCI Express Mini Card 1.0 Specification Overview Shaw/Saunders
4:00 PCI Express Bridge 1.0 Specification Overview Willke/Ajanovic
5:00 Technical Q&A Bhatt, et al
Page 103
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
103
PCI Express Bridge TopicsPCI Express Bridge TopicsBackground
System View Benefits
Bridge Architecture Overview
Bridge Operation & Performance
Summary
Page 104
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
104
BackgroundBackgroundServer IO fan-out through bridging
Current solutions use proprietary technologiesCurrent solutions use proprietary technologiesCurrent solutions use proprietary technologies
LAN
100
100
System Bus
CPUCPU
133
133
IOBridge
SCSI
133
Base IO
100
133
133
ProprietaryProprietaryI/O solutionsI/O solutions
CPUCPU. . .. . .
DRAM
StandardStandardPCI/PCIPCI/PCI--X X ExpansionExpansion
IOBridge
IOBridge
IOBridge
Host Bridge/Hub
LAN
System BusCPUCPU
SCSI
CPUCPU
DRAM
Host Bridge
StandardPCI
Expansion
PCI-PCIBridge
33Mhz/64b(32b)
33Mhz/64b(32b)
“Vertical” vs. “Horizontal” expansion
Page 105
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
105
Introducing PCI ExpressIntroducing PCI Express
PCI family technologies top to bottomPCI family technologies top to bottomPCI family technologies top to bottom
LAN
System Bus
CPUCPU
PCI ExpressBridge
StorageI/F
Base IO
100
100
PCI ExpressPCI Express
CPUCPU. . .
DRAM
PCI/PCIPCI/PCI--X X ExpansionExpansion
PCI ExpressBridge
PCI ExpressBridge
RootComplex
. . .133/266 133/266
Page 106
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
106
PCI Express Bridge SpecificationPCI Express Bridge Specification
PCI-SIG Workgroup formed May 2002PCI Express Bridge Specification Rev 1.0 Draft - 2/19/03Workgroup members:
Q2’02 Q4’02 Q1’03Q3’02
Draft OutlineInternal Draft
RevisionsPCI-SIG
60 Day Review
Timeline
Page 107
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
107
PCI Express Bridge TopicsPCI Express Bridge TopicsBackground
System View Benefits
Bridge Architecture Overview
Bridge Operation & Performance
Summary
Page 108
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
108
Standardized Fan-out & ScalabilityStandardized Fan-out & Scalability
Top-to-bottom industry-standard I/O solution
Switches and multi-port bridges for fan-out
Enables large, complex system designs
PCI Express Bridge standardizes PCI/PCI-X fan-outPCI Express Bridge standardizes PCI/PCIPCI Express Bridge standardizes PCI/PCI--X fanX fan--outout
System Bus
CPUCPU
PCI ExpressBridge
100
100
PCI ExpressPCI Express
CPUCPU. . .
DRAM
PCI/PCIPCI/PCI--X X ExpansionExpansion
PCI ExpressBridge
PCI ExpressBridge
RootComplex
. . .133/266 133/266
SwitchPCI
ExpressEndpoint
PCI ExpressEndpoint
PCI ExpressBridge
133/266
PCI ExpressEndpoint
PCI ExpressEndpoint
PCI ExpressEndpoint
PCI ExpressEndpoint
Page 109
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
109
Improved System LayoutImproved System LayoutLower pin count required of RC to support IO fan-out requirements
Avoids layout “hot-spots” around platform’s CPU/Memory/RC complex
Fast/wide PCI-X moved to periphery; ~20” routing length from RC to the Bridge
Root Complex
PCI Express to PCI-XBridge
64-Bit PCI-X Slots 8-Bit PCI Express Slots
I/O Subsystem Layout Example
Simpler Layout = Lower Cost & Higher RobustnessSimpler Layout = Lower Cost & Higher RobustnessSimpler Layout = Lower Cost & Higher Robustness
Page 110
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
110
Platform Compatibility -Software and HardwarePlatform Compatibility -Software and Hardware
PCI Express backwards compatible with:Legacy OS-es and PCI.SYS Bus Driver PCI device driver binariesPCI Software Stacks (PCI PM, SHPC,...)
PCI Express Bridging:Hardware compatibility with PCI/PCI-X at the platform level
PCI Express provides system level compatibility with existing PCI technologies
PCI Express provides system level PCI Express provides system level compatibility with existing PCI technologiescompatibility with existing PCI technologies
Page 111
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
111
Example – System OEMExample – System OEMPCI Express used for fan-out and point-of-attachBridge provides backwards compatibility at the PCI/PCI-X component and slot level
PCI-X Add-In Card
CPUCPUSystem Board
Memory
RootComplex
RootComplex
PCI ExpressBridge
PCI Express Add-In Card
Page 112
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
112
Example – Silicon/Card VendorExample – Silicon/Card VendorEnable PCI Express add-in card solutions using PCI/PCI-X silicon
Bridges expand market for add-in siliconBridges expand market for addBridges expand market for add--in siliconin silicon
PCI ExpressBridge PCI(-X)PCI(-X)
PCI Express Add-In Card
PCI-X Add-In Card
PCI ExpressEndpoint
PCI ExpressEndpoint
“Rev erse”Bridge
Enable PCI/PCI-X add-in card solutions using PCI Express silicon
Page 113
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
113
PCI Express Bridge TopicsPCI Express Bridge TopicsBackground
System View Benefits
Bridge Architecture Overview
Bridge Operation & Performance
Summary
Page 114
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
114
Key Architecture RequirementsKey Architecture Requirements
PCI-compatible programming model No device driver required for bridge
Hardware/Protocol compatibility:Primary side: PCI Express Base 1.0Secondary side: PCI-X 2.0 (Mode 1 & 2) and/or PCI 3.0 Full PCI and PCI-X feature set and applicable PCI Express capabilities
Support for multi-port (PCI/PCI-X) configurationsProvisions for transitional legacy features Cost effective and scalable solution
Minimize translation complexity
Deliver Optimal PCI/PCI-X Bridging SolutionDeliver Optimal PCI/PCIDeliver Optimal PCI/PCI--X Bridging SolutionX Bridging Solution
Page 115
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
115
Programming ModelProgramming ModelBackwards Compatibility
Based on PCI-to-PCI Bridge ArchitectureReports Base Class/Sub-Class of PCI-PCI bridge
PCI Express CAP StructureDev/Type is PCI Express to PCI/PCI-X BridgeBridge specific bits added
Other Capabilities (optional): PCI-X, SHPC, PM, AER, etc.
Enhanced ConfigurationExtended config. space (4K) using mem-mapped accessConfiguration retry
PCI ExpressExtended
ConfigurationSpace
(Not accessiblewith legacy OS)
PCI Configuration
Space(Accessible with
legacy OS)
0
0x100
0x1000
ExtendedConfiguration
Space
PCI Express Capability
Type 1 PCI-PCI BridgeConfig. Header
0x40
Extended Capability
Extended Capability
Page 116
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
116
Transaction ForwardingTransaction ForwardingPCI Express to PCI/PCI-X
Memory, Configuration, and I/O transactions forwardedPCI Express completions translate to either Delayed (PCI) or Split (PCI-X) CompletionsLocks propagatedVendor-Defined Message translation is optional
PCI/PCI-X to PCI ExpressMemory and I/O transactions forwardedDevice ID Message translation is optional
PCI ExpressInterface
PCI ExpressBridge
PCI/PCI-XInterface
ControlAddr/Data
ArbitrationMisc.
PCI_CLK
1 2 3 4 5 6 7 8 9 10
AD[31:0]C/BE[3:0]#FRAME#
IRDY#TRDY#
DEVSEL#
Frame Seq # Hdr Data CRC Frame
Page 117
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
117
RAS SupportRAS SupportData Integrity through the Bridge
PCI Express side – TLPs: LCRC-32b, optional ECRC-32b– DLLPs: CRC-16b
PCI/PCI-X side: parity, ECCForwarding of Data Errors
– Poisoned data (EP=1)– Bad data ECC or data parity
Hot Plug - supported usage models:Bridge on main planar – PCI side HP using SHPCBridge on add-in card – PCI Express Native HP
Page 118
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
118
Error Signaling/HandlingError Signaling/HandlingError Signaling Messages
Signal interface-level and device-level errorsFixed severities for PCI errors unless AER is supportedPCI masks applicable
Advanced Error Reporting (optional)
Bridge-specific versionIncludes PCI/PCI-X errors and header log
...
...
REQFATALSERR Enable or
FERE
Address Parity Error
OPTIONALNONFATALSERR Enable or
NFERE
Data Parity Error
(Target)
REQNONFATALMAM and (SERR
Enable or NFERE)
Received Master-Abort
Required/ Optional
ExpressMessage
MaskPCI/PCI-X Error
Customized Error Handling for the ApplicationCustomized Error Handling for the ApplicationCustomized Error Handling for the Application
Page 119
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
119
Power Management EventsPower Management EventsPME# may be routed around or connected to bridge
PME# converted to Power Management Message
PM Message Requester ID uses secondary interface bus numberLevel to edge semantic conversionInternal PME generation supported
Bridge Polls Line
Device BPME#
Device APME#
1st PM Msg 2nd PM Msg 3rd PM Msg
Wired-OR
PCI ExpressBridge
Wired-ORPME# Bus 0
Wired-ORPME# Bus 1
PM Message(Bus No.)
Page 120
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
120
Interrupt SupportInterrupt SupportMSI/MSI-X preferred method
MSI forwarding is given
PCI InterruptsConverts INTA# - INTD# into Interrupt Signaling MessageLevel semantics preserved“Virtual wire” assignment based source Device ID & INTx#
Internal interrupts may be either MSI/MSI-X or “virtual wire” INTx
Page 121
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
121
PCI Express Bridge TopicsPCI Express Bridge TopicsBackground
System View Benefits
Bridge Architecture Overview
Bridge Operation & Performance
Summary
Page 122
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
122
Bridging to PCIBridging to PCIPosted request (write or message)
Bridge creates one or more new transactions/TLPs using buffered data
Non-Posted request (read or write)PCI: Bridge handles as Delayed Transaction
– Must reserve buffer space for resulting completionsPCI Express: Bridge issues PCI requests as needed to build completion TLP
Similar to PCI-X to PCI BridgingSimilar to PCISimilar to PCI--X to PCI BridgingX to PCI Bridging
Page 123
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
123
Key Protocol DifferencesKey Protocol Differences
Memory Read Block and Memory Read DWORD use different completion formats
Requests may not cross 4 KB boundaries
Max Memory Read Byte Count range:512 – 4096 bytes
Max_Read_Request_Size range:128 – 4096 bytes
Target may disconnect transactions at ADBs (128 Byte boundaries)
TLPs may not exceed Max_Payload_Size
PCIPCI--XXPCI ExpressPCI Express
Bridging to PCI-XBridging to PCIBridging to PCI--XX
Page 124
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
124
Key Protocol Differences (cont.)Key Protocol Differences (cont.)
Tag size: 5 bitsAll Sequence IDs must be unique
Tag size: 5 (default) or 8 bitsTag is reserved for Posted Writes
Memory Write command permits discontiguous BEs up to 4KB
No discontiguous BEs on TLPslonger than 2 DWORDs
Read completions follow ADB rules
Read Completion Boundary:Both 64B and 128B supported
No support for TC/VCTC/VC support
PCIPCI--XXPCI ExpressPCI Express
Bridges must manage these differencesBridges must manage these differencesBridges must manage these differences
Bridging to PCI-XBridging to PCIBridging to PCI--XX
Page 125
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
125
Posted RequestPosted RequestPCI-X: Bridge generally forwards sequence unmodified. Fragmentation required if request:
Exceeds Max_Payload_SizeCrosses 4KB boundaryContains discontiguous Byte Enables
PCI Express: Resulting PCI-X Posted Write must have unique Sequence ID. Bridge may:
Reassign Sequence IDTrack outstanding non-posted requests from same requester
Bridging to PCI-XBridging to PCIBridging to PCI--XX
Page 126
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
126
Non-Posted RequestNon-Posted RequestPCI-X: Bridge generally forwards request unmodified. Bridge must take ownership if request:
Exceeds Max_Read_Req_SizeCrosses 4KB boundaryIs a Memory Read DWORD command
Ownership requires bridge to save transaction state information to create resulting completion
Buffer space rules similar to PCI-X bridgeImpacted by RC Read Completion Boundary
Bridging to PCI-XBridging to PCIBridging to PCI--XX
Page 127
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
127
Non-Posted Request (cont.)Non-Posted Request (cont.)PCI Express: Bridge generally forwards unmodified. Bridge must take ownership if request:
Is a two DWORD non-prefetchable read with discontiguous BEsIs a configuration space read or writeContains an 8-bit Tag or TC ≠ 0
Bridging to PCI-XBridging to PCIBridging to PCI--XX
Page 128
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
128
Performance ConsiderationsPerformance ConsiderationsPCI-X devices:
Avoid initiating requests thatcross 4 KB boundaryMinimize use of Memory Read DWORD commandMinimize use of discontiguous byte enables
PCI Express devices:Avoid 2 DWORD reads with discontiguous BEsImplement 128 Byte RCB in RCs
State perTransaction
Maximum #OutstandingTransactions
Page 129
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
129
Performance Considerations (cont.)Performance Considerations (cont.)
Software:Use same setting for:
– Max Memory Read Byte Count (PCI-X)– Max_Read_Request_Size (PCI Express)
Avoid 8-bit Tag or TC ≠ 0 for devices that communicate through bridge to PCI-XTune PCI-X Split Transaction Commitment Limit
– Balance bridge buffer utilization against potential backpressure
– Default value creates no backpressure but limits request forwarding
128 B
512 B
4kB
PCI-X Max Memory Rd Byte Count
1kB 2kBPCI Express Max_Read_Request_Size
Page 130
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
130
PCI Express Bridge TopicsPCI Express Bridge TopicsBackground
System View Benefits
Bridge Architecture Overview
Bridge Operation & Performance
Summary
Page 131
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
131
SummarySummaryThe PCI Express Bridge Architecture Offers:
PCI 2.3(3.0)/PCI-X 2.0 support while enabling systems to incorporate native PCI Express
Compatibility with existing PCI software/firmware
Scalable set of capabilities
High degree of implementation flexibility
Straightforward performance optimizations
Page 132
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
132
AgendaAgenda8:00 Coffee, Registration
8:30 PCI Express Architecture Overview Bhatt
9:00 PCI Express Base 1.0a Electricals Dunning/Schoenborn
10:45 PCI Express Base 1.0a Protocol Ajanovic/Harriman
12:30 Lunch (provided)
1:00 PCI Express Card 1.0a Stancil
2:00 PCI Express 1.0a Compliance Requirements Hosler
3:00 PM Break
3:15 PCI Express Mini Card 1.0 Specification Overview Shaw/Saunders
4:00 PCI Express Bridge 1.0 Specification Overview Willke/Ajanovic
5:00 Technical Q&A Bhatt, et al
Page 133
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
Page 134
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
134
Layered ArchitectureLayered ArchitectureSoftwareSoftware
Data LinkData Link
MechanicalMechanical
TransactionTransaction
PhysicalPhysical
PCI Express Layering Enables Modularity, Reuse and Scalability PCI Express Layering Enables
Modularity, Reuse and Scalability
Software PCI compatible and extended configuration mechanism
Transaction PCI-like 32bit/64bit address load-store transaction semantics
Data Link Provides reliable data transport services
PhysicalElectrical interface and signaling mechanisms
MechanicalMarket segment specific form factors
Page 135
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
135
Physical LayerPhysical LayerInterface initialization, maintenance control, and status tracking
Reset/Hot Plug control/status Interconnect power management Width and Lane mapping negotiationLane and Polarity reversal
Symbol and special ordered-set generation8-bit/10-bit encoding/decodingSymbol scramblingEmbedded clock tuning and alignment
Symbol transmission and alignmentTransmission circuitsReception circuitsElastic buffer at receiving sideMulti-Lane de-skew (for widths > x1) at receiving side
System DFT support featuresScope patternLoop back
SoftwareSoftware
Data LinkData Link
MechanicalMechanical
TransactionTransaction
PhysicalPhysical
Page 136
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
136
Data Link Layer Data Link Layer Initialization and power management
Convey power state requestsConvey link state
– active/reset/disconnected/power managed
Data protection, error checking, and retry CRC generation and error checkingTransmitted TLP storage for Data Link level retryTLP acknowledgment and retry messagesError indication for error reporting and logging
Every detectable error is correctable Every detectable error is correctable Every detectable error is correctable
SoftwareSoftware
Data LinkData Link
MechanicalMechanical
TransactionTransaction
PhysicalPhysical
Page 137
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
137
Transaction LayerTransaction LayerBased on Load-Store architecture
Common flat 32/64 bit address spacePCI addressing
Split-transaction, packet-based protocol
Credit based flow control
“PCI-X like” producer-consumer ordering rules
Relaxed ordering, no snoop support
End-to-end CRCPreserving PCI architecture while addressing basic limitationsPreserving PCI architecture while addressing basic limitations
SoftwareSoftware
Data LinkData Link
MechanicalMechanical
TransactionTransaction
PhysicalPhysical
Page 138
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
138
Software LayerSoftware LayerPCI-compatible configuration mechanism
Defined to boot PCI 2.2 compliant OSs unmodifiedCompatibility with PCI device driver model
– PCI Power Management, SHPC Hot-plug, MSI support
PCI-compatible in-band signalingPME supported using in-band messagingMSI is the native interrupt mechanism
Enhanced configuration mechanismFlat memory-mapped access
– Removes bottlenecksExtends device configuration space Improves configuration transaction performance
PCI Express does not require new OS and BIOSPCI Express does PCI Express does notnot require new OS and BIOSrequire new OS and BIOS
SoftwareSoftware
Data LinkData Link
MechanicalMechanical
Transaction
PhysicalPhysical
Page 139
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
139
PCI Express x1 connector
PCI Express x16connector
PCI Express layout and connectors in 4 layers
Graphics – x16 PCI Express
Generic I/O – x1 PCI Express
Board Layout AdvantagesBoard Layout Advantages
• Board Area Reduced by 53%
• Board Layer Count Reduction
• Component Count Decreases
Fewer pins and simpler routing -> Lower cost and complexityFewer pins and simpler routing -> Lower cost and complexity
Page 140
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
140
Embedded ClockingEmbedded Clocking8b/10b Encoding
Limits the run length of 1’s and 0’s to fiveData transitions enable bit level clock recovery at the receiverLimits the range for frequency content in signalMaintains a DC balance on each differential pair
Higher performanceHigh frequencies -> lossy transmission lines
System advantagesEliminates need for routing clocks
Byte Value 00HByte Value 00H
1 1 10
0
8b/10b 8b/10b Encoded ValueEncoded Value 0 00 1 1 0
Page 141
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
141
Link ConfigurationLink Configuration
Link WidthUpstream wider than downstreamDownstream wider than upstream
Lane Ordering/Reversal
Lane ordering can be swapped within a link
Avoid the “bow tie” dilemma
Polarity Inversion (D+, D-)
Device B
RX+RX-
TX+TX-
RX+RX-
CAPCAP
CAP
Device ATX+TX-RX+RX-
CAP
TX+TX-
Port 0 Port 0
Port 1
Device B
RX+RX-
TX+TX-
RX+RX-
CAPCAP
CAPCAP
TX+TX-
Link 0
Link 1TX+TX- CAP
CAP
CAPCAP
Device ATX+TX-RX+RX-
RX+RX-
TX+TX-
Link 1
Link 0
Device B
RX+RX-
TX+TX-
RX+RX-
CAPCAP
CAPCAP
TX+TX-
Port 0
Port 1TX+TX- CAP
CAP
CAPCAP
Device ATX+TX-RX+RX-
RX+RX-
TX-TX+
Port 1
Port 0
Simplifies PCB Routing Simplifies PCB Routing Simplifies PCB Routing
Page 142
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
142
AC CouplingAC CouplingAllows for Different DC common mode voltages at Tx and Rx
Allows for different processes for Tx and RxAllows for different Tx and Rx architectures
Allows for any supply voltageRequires DC balanced signalsAllows for different grounds on communicating componentsCan lead to a reduction of power across channel
Tx = Transmitter, Rx = Receiver
Page 143
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
143
Power – L0 state – ExamplePower – L0 state – Example2.5 Gb/s data exchange
Maximum allowable power not specified– Recommended power target = <80 mW (per lane)
+-
50Ω50Ω50Ω50Ω
V
T-Line
T-Line
ClockSource
ClockSource
Clock Source ON
Low ImpedanceTermination
Tx Rx
Data being transmitted(May be Logical Idle)
Clock Source ON
Driver ON Receiver ON
Note: Both directions of link are in L0 at same timewith one exception being L0s (next slide)
Note: Only one direction shown here.
Page 144
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
144
Power – L0s state – ExamplePower – L0s state – ExampleTx holds line at Electrical Idle
Maximum allowable power not specified – Recommended power target = <20 mW (per lane)– Recommended latency L0s -> L0 <100 nsec
+-
50Ω50ΩXΩXΩ
V
T-Line
T-Line
ClockSource
ClockSource
Must be stable to meetresume latency target:
Expectation = ON
Recommend HighImpedance Termination
Tx Rx Held at Operating DC commonMode voltage
Notes: For a Tx/Rx pair on one component (not shown above), the pair are not required to be in L0s at the same time.
L0s entered via reception of the Electrical Idle ordered-set.
L0s transitions to L0 via reception of FTSs, which allows the Rx re-establish bit, symbol lock and lane-lane deskew.Expected implementations store settings regarding bit lock, symbol lock and lane-lane deskew in order to reduce theL0s to L0 latency.
Must be stable to meetresume latency target:
Expectation = ONNote: Only one direction of link shown here.
Page 145
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
145
Power – L1 state – ExamplePower – L1 state – ExampleTx holds line at Electrical Idle
Maximum allowable power not specified – Recommended power target = <5 mW (per lane)– Recommended latency L1 -> L0 <10 usec
+-
50Ω50ΩXΩXΩ
V
T-Line
T-Line
ClockSource
ClockSource
Must be stable to meetresume latency target:
Expectation = ON if sharedOFF if local to Tx/Rx
Recommend HighImpedance Termination
Tx Rx Held at Operating DC commonMode voltage
Notes: For a Tx/Rx pair on one component (not shown above), the pair are required to be in L1 at the same time.
L1 entered via direction from link layer followed by the successful reception and sending of the Electrical Idle ordered-set
L1 transitions to Recovery via reception of TS1s or if directed , which allows bit lock, symbol lock and lane-lane deskew.
Must be stable to meetresume latency target:
Expectation = ON if sharedOFF if local to Tx/RxNote: Only one direction shown here.
Page 146
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
146
Power – L2 state – ExamplePower – L2 state – ExampleTx holds line at Electrical Idle
Maximum allowable power not specified – Recommended power target = <1 mW– Recommended latency L2 -> L0 12 msec – 50 msec
+-
50Ω50Ω
V
T-Line
T-Line
ClockSource
ClockSource
High Freq. Clock sourceexpected to be OFF
Low Freq source for Beaconmay be needed
Low or High ImpedanceTermination
Tx Rx Not required to hold
Operating Common Mode voltage
Notes: For a Tx/Rx pair on one component (not shown above), the pair are required to be in L2 at the same time.
L2 entered via direction from link layer followed by the successful reception and sending of the Electrical Idle ordered-set.
L2 exits eventually back to Detect to configure a new Link.
High Freq. Clock sourceexpected to be OFF
Low Freq source for Beacondetection may be needed
Driver expected to be OFFReceiver expected to be OFF
XΩXΩ
Note: Only one direction shown here.
Page 147
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
147
Power – L3 state – ExamplePower – L3 state – ExampleNo power
A description in the PHY layer chapter does not applyPower target = 0W (power is off)Latency L3 -> L0 12 msec – 50 msec after power is stable at both Tx and Rx
+-
BigBigBigBig
V
T-Line
T-Line
ClockSource
ClockSource
Clocks OFF
High ImpedanceTermination
Tx Rx
Notes: For a Tx/Rx pair on one component (not shown above), the pair are by definition in L3 at the same time.
L3 entered when power is removed.
L3 transitions to L0 via reception of full training sequence.
Clocks Off
OFF OFF
High ImpedanceTermination at Rx
Note: Only one direction shown here.
Not required to holdOperating Common Mode voltage
Page 148
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
148
Error Behavior/RecoveryError Behavior/RecoveryPrimary data integrity function = CRCs
CRCs are not checked at the PHY LayerThere is minimal required error checking and recovery in PHY layer
Data that is valid per 8b/10b encoding rules symbols with correct disparity are passed to data link layerData that is not valid per 8b/10b encoding rules symbols and/or correct disparity is invalid and sets the receiver error bit
Packet framing optionally checked at PHY LayerFraming errors detected at PHY layer sets the receiver error bit
Higher Layers are required direct the Physical Layer from L0 to Recovery
Page 149
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
149
“0 Volts Common Mode at Rx”“0 Volts Common Mode at Rx”Terminations grounded at Receiver
This is required for DetectThis does not require the DC common mode voltage = 0 volts at the inputs to the receiver differential amplifier (comparator)
A simple bias network allows the receiver to operate at the optimal common mode
+-
50Ω50Ω
NoSpec
Big
Big
Small
Big
Big
Small
One Possible Option
Ratio of resistors sets DCcommon mode Voltage at receiver
Page 150
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
150
ESD, EMI, Noise SuppressionESD, EMI, Noise SuppressionESD specified per JEDEC JESE22-A114-A
2000 V human body model500 V charged device model
Parasitic Capacitance and/or Inductance can cause impedance discontinuities at receivers
To run in the GHz range, we all need to be careful hereA spec can not solve this
EMI: Data is scrambled
Noise Suppression: Differential
Best Known Methods to operate I/O in GHz rangeBest Known Methods to operate I/O in GHz range
Page 151
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
151
Receiver DetectReceiver DetectComponent A
X+Ω X+Ω
Component B
50Ω50Ω
V_BA
Attach/Detach Point
Time Constant to charge ~= (X)(Cpad +Cinterconnect)= Small compared to below
CAC Coupler
CAC Coupler
Cpad
Component A
X+Ω X+Ω
Component B
50Ω50Ω
D+D-
V_BA
Attach/Detach Point
Time Constant to charge ~= (X)(CAC Coupler)= Large compared to above
CAC Coupler
CAC Coupler
Cpad
Cpad +Cinterconnect << CAC Coupler
D+ D-
Page 152
Copyright © 2003, PCI-SIG, All Rights ReservedPCI-SIG Confidential
152
Testing Electrical ComplianceTesting Electrical Compliance
50Ω50Ω
V
T-Line
T-Line
Attaching 50 ohm probes to one or both pairs of conductors of a Lane causes the compliance pattern to be output.
Tx
50Ω
50 ohm probe (test equipment)
Compliance Pattern
PHYLogic
PHYLogic
50Ω
50Ω
50 ohm probe
50Ω50Ω50Ω
V
Tx
Testing of the TX Eye Diagram
Testing of the RX Eye Diagram
Used to test compliance of the TX and RX Eye DiagramUsed to determine what component (TX, interconnect, RX) is out of specification in an inter-operability dispute.