Frank Lemke Experiences and results using the CBMnet Experiences and results using the CBMnet protocol including precise time protocol including precise time synchronization and clock distribution synchronization and clock distribution DPG – Frühjahrstagung Münster 2011 HK 20.7 University of Heidelberg Computer Architecture Group Frank Lemke, Sven Schenk, Ulrich Brüning 22.03.2011
16
Embed
DPG – Frühjahrstagung Münster 2011 HK 20.7 University of Heidelberg Computer Architecture Group
DPG – Frühjahrstagung Münster 2011 HK 20.7 University of Heidelberg Computer Architecture Group Frank Lemke, Sven Schenk, Ulrich Brüning 22.03.2011. Experiences and results using the CBMnet protocol including precise time synchronization and clock distribution. Outline. Motivation - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Frank Lemke
Experiences and results using the CBMnet protocol including Experiences and results using the CBMnet protocol including precise time synchronization and clock distribution precise time synchronization and clock distribution
DPG – Frühjahrstagung Münster 2011
HK 20.7
University of Heidelberg
Computer Architecture Group
Frank Lemke, Sven Schenk, Ulrich Brüning
22.03.2011
Frank Lemke2
OutlineOutline
Motivation CBM Protocol Structure CBM Network Synchronization Beamtime results Dec. 2011 Adaptions heading towards CBMnet V2.0 Conclusion & Outlook
Frank Lemke3
MotivationMotivation
Standard Protocols abilities are only partly reasonable for the special demands within CBM DAQ System
An optimized design of a CBM protocol will result in
clearly better performance
CBM unique requirements considered under the topics of Space and cost limitations High bandwidth Specific system synchronization Reusability of protocol modules for different
System requirements Clock recovery and distribution for all system levels Guaranteed deterministic timing during runtime at all times
Special features provided by DLM usage Heterogenous synchronized system initialization Enables periodic time synchronization for epoch markers Various special user defined event signalling
with deterministic latency
Implementation specifics Priority request insertion Deterministic and structural separated HDL coding Specific HW requirements - FPGA configuration scheme and settings
Cosy Beamtime December 2010Cosy Beamtime December 2010
9
DCB
Jitter cleaner device50 meter cable toread-out controller
3 meter cables toABB and ECS
Frank Lemke
CBMnet Beamtime ResultsCBMnet Beamtime Results
Detector Control Messages Worked without problems during complete beamtime tests
Data Transfer Messages Merging of 4 data-streams within DCB into a single stream No fatal data transmission error No data flow problems
Deterministic Latency Messages All counters values were reset synchronous Synchronization using DLMs showed no problems
10
Frank Lemke11
Packet Format CBMnet V2.0Packet Format CBMnet V2.0
SOP Payload
16 Bit 2 - 64 Byte
R outing LME CRC EOP
16 Bit 16 Bit 16 Bit 16 Bit
Long messages extension (LME) requires a small message definition Insertion of extension character Reduction of buffer space Balanced network message flow LME packet well defined No additional overhead for small messages Type field within LME code required Additional overhead circa 2% Total overhead 11%
Frank Lemke12
Long Message Extension (LME)Long Message Extension (LME)
TYPE MTAG DSN
4 Bit 4 Bit 8 Bit
Type: 4’b Identifier for LME, 1111 reserved
Message TAG (MTAG): 32 different long messages can be handled
Decrementing Sequence Number (DSN): Shows the amount of packets following
Maximal length = 256*64Byte = 16KByte
Frank Lemke13
LME - Data Flow InfluencesLME - Data Flow Influences
DLMs still easily insertable with priority request insertion into message streams without increasing complexity
DCMs and administration chars can be arbitrated inbetween LME packets
Numerous message streams can be merged into one aggregate data stream
SOP Payload
16 Bit 2 - 64 Byte
R outing LME CRC EOP
16 Bit 16 Bit 16 Bit 16 Bit
SOP Payload
16 Bit 2 - 64 Byte
R outing LME CRC EOP
16 Bit 16 Bit 16 Bit 16 Bit…
DLM
16 Bit
SOSC Payload
16 Bit 4 - 64 Byte
R outing CRC EOP_C
16 Bit 16 Bit 16 Bit
R outing
16 Bit
…
…SOP Payload
16 Bit 2 - 64 Byte
R outing LME CRC EOP
16 Bit 16 Bit 16 Bit 16 Bit
=> LME has no major influence on the network flow characteristics
Frank Lemke14
Features towards CBMnet 2.0Features towards CBMnet 2.0
CBMnet V1.0 Features Planned additional CBMnet V2.0 extensions
• Communication over one optical linksupporting DTM, DCM and DLM
• Optimized data utilization about 91 % ( about 73 % considering 8b/10b)
• Optimized easy to use Interface• Highly modular CBM LP code structure• Special adapted routing scheme• Fast and efficient administration packets• Retransmission for Control Packets• System wide clock recovery with low jitter• Deterministic link latency feature for well
defined Deterministic Latency Messages
• Lane handling for unbalanced communication
• Support for large messages within hardware
• Planned data loss strategy in cases of overloads at the end of epochs
• Communication also reliable for data stream
• Meta data detection adaption to find Meta data within message streams in early system stages
Frank Lemke15
Conclusion Conclusion && Outlook Outlook
Beamtime results show reliability and usability of CBMnet V1.0 Planned extensions fit into CBMnet scheme for V2.0
Interface stays almost the same Implementation complexity should be manageable
The next step will be to include CBMnet logic into frontend ASICs Also next generation FPGA ROCs and a possible read-out ASIC are