CMS Endcap Muon DDU:  Output Data Format

Early First Prototype Version

used in May 2003 - June 2004 beamtests, still used at UCLA

---

DDU Transmission Format

Note:  the 1st Header word from every DMB (always Code word   "8" or "9"; see the DMB-DDU Format page) are processed and discarded by the DDU zero suppression algorithm.  The useful information contained in those words is summarized in the DDU event header. The DDU operates using a 64-bit word architecture.
 

The DDU output format is equivalent in both the Main (S-Link64) and Private EMU (Gigabit Ethernet) DAQ paths:

The main CMS DAQ path follows the S-Link64 protocol and transmits 64-bit wide data, plus a "65th" bit (bit64) for use as the control character (aka "K-bit"); this bit is only true (i.e. "high") for the first header and last trailer of every event.  There are other "CMS Directives" for the S-Link DAQ protocol that are implemented in the DDU Format; these details are specified below.

DDU data is transmitted byte-wise via Gigabit Ethernet lowest-byte first; the LSB (bit0) is always on the right.

Overview of the DDU event record

T = Event Type (4 bits)
N = Event Number (24 bits)
X = BX number (12bits)
I = Source ID (10+2 bits)
V = Format Version (4 bits)
K = S-Link Status (lowest 4 bits are reserved in a S-Link64 K-word)
S = DDU Status/Error (32 bits)
Z = DMB-DAV (16 bits)
Y = Active DMB Count (4 bits)
W = DDU 64-bit Word Count (24 bits)

DDU_WC = (6 + 25*N_ts*nCFEB + 3*nDMB)

U = Event Status (8 bits) {repeat some elements of DDU Status here}
R = CRC Check Word (16 bits) {not used yet}
? = not used yet, but reserved for future use (4 bits each)
More details below

bit[63:60] = 0x5       //hard-coded in DDU firmware by CMS directive
bit[59:55] = 4-bit "event type"       //CMS directive, no details yet
bit[55:32] = 24-bit Level 1 event number       //from DDU L1A counter
bit[31:20] = 12-bit Bunch Crossing Number       //by CMS directive

*may come from DDU, DMB, ALCT or TMB
*appropriate matching req'd at DDU

bit[19:8] = 12-bit Source ID          //CMS directive, no details yet
bit[7:4] = 4-bit "format version"   //CMS directive, no details yet
bit[3:0] = 4-bits reserved               //CMS directive, no details yet

*special "S-Link64 Protocol" status bits

Header2

bit[63:0] = a 64-bit constant, hard-coded in DDU firmware:     0x8000/0001/8000/8000

*this is a "DDU unique word" near the start of the event packet

Header3

bit[63:32] = 32-bit DDU status/error code for EMU use, defined below:

bit63: DDU Output Limited Buffer Overflow (status only)
bit62: DDU Giga-Bit Ethernet FIFO Full Warning (status only)
bit61: DDU Giga-Bit Ethernet FIFO Near Full Warning (status only)
bit60: DDU Giga-Bit Ethernet Fiber Error (status only!)
bit59: DDU Bad First Data Word From DMB Error (bad event)
bit58: DDU L1A-FIFO Full Error (RESET req'd)

*the DDU L1A-event info FIFO went full; some triggers/events may be lost or garbled

bit57: DDU Data Stuck in FIFO Error (RESET req'd)
bit56: DDU NoLiveFibers Error (status only)

*no DDU fiber inputs are connected, something is wrong; will cause other errors...

bit55: DDU Special Word Incosistency Warning (possible bad event?)

*a bit-vote failure occured on an input fiber channel

bit54: DDU Ethernet FPGA clock-DLL Error (status only!)

*the Ethernet system on the DDU lost it's clock for an unknown period of time
*only the private Ethernet readout path is affected by this

bit53: DDU S-Link Full Bit set (status only)
bit52: DDU S-Link Not Ready (status only)
bit51: DDU TMB Error detected(bad event)

*TMB trail word not found or TMB L1A, CRC or wordcount inconsistent

bit50: DDU ALCT Error detected(bad event)

*ALCT trail word not found or ALCT L1A, CRC or wordcount inconsistent

bit49: DDU TMB or ALCT Word Count Error detected(bad event)

*TMB/ALCT wordcount inconsistent

bit48: DDU TMB or ALCT L1A Number Error detected(bad event)

*TMB/ALCT L1A Number mismatch with DDU

bit47: DDU Critical Error, irrecoverable (RESET req'd)

*OR of all possible "RESET required" cases

bit46: DDU Single Error (bad event)

*OR of all possible "bad event" cases

bit45: DDU Single Warning (possible bad event?)

*OR of bit55, bit42

bit44: DDU FIFO Near Full Warning (status only)

*OR of all possible "Near Full" cases

bit43: DDU RX Error on 1 or more input fibers (bad event, RESET req'd)

*future DDU designs will handle this differently...

bit42: DDU Control FPGA clock-DLL Error (lost phase lock, RESET req'd)

*the DDU lost it's clock for an unknown period of time; some triggers/events/data may be lost

bit41: DDU DMB Error (B-code received via DMB, bad event)
bit40: DDU Lost In Event Error (bad event, but end was found)

*the DDU failed to find an expected control word within the event

bit39: DDU Lost In Data Error (bad event, RESET req'd)

*looks like different events were possibly run together!
*irrevocably lost in the data stream
*found at least one of the following in the event data stream, all of which are very bad:

-Extra DMB_Header1 before DMB_Last_Word
-Extra DMB_Header2 before DMB_Last_Word
-Lone Word before DMB_Last_Word
-Extra TMB/ALCT_Trailer before DMB_Last_Word
-Extra DMB_Trailer1 before DMB_Last_Word
-DMB_Trailer2 before DMB_Trailer1

Note:  DMB_Last_Word == DMB_Trailer2

bit38: DDU Timeout Error (bad event, RESET req'd)

*data from a fiber input either never started or never finished
*an unknowable amount of data has been irrevocably lost

bit37: DDU Critical Data Error (bad event, RESET req'd)

*more than one bit-vote failure (or Rx Error) has occured in one 64-bit word on an input fiber channel

bit36: DDU Multiple Transmit Errors (bad event, RESET req'd)

*one bit-vote failure (or Rx Error) has occured on multiple occassions for the same input fiber channel

bit35: DDU FIFO Full Error (bad event, RESET req'd)

*an unknowable amount of data has been irrevocably lost

bit34: DDU Fiber Error (hardware configuration change, RESET req'd)

*change of connection status on 1 or more DDU fiber inputs; a hardware problem probably exists

bit33: DDU L1A Match Error (bad event, RESET?)

*the DDU L1A event number match failed for 1 or more CSC boards; possible one-time bit error
*if error continues for consecutive events then RESET req'd

bit32: DDU CRC Error (bad event, RESET?)

*CRC check failed for ADC data on 1 or more CFEBs; possible one-time bit error
*if error continues for consecutive events then RESET req'd

Note:  error bits requiring RESET persist until the RESET occurs.

bit[31] = 0, hard-coded in DDU firmware
bit[30:16] = 15-bits indicating which DMBs have data for this event, where 1 DMB == 1 CSC; one bit allocated per DDU fiber input
bit[15:4] = HEX 000, hard-coded in DDU firmware
bit[3:0] = number of DMBs which have data for this event

Trail-2

bit[63:0] = a 64-bit constant, hard-coded in DDU firmware:     0x8000/FFFF/8000/8000

*this is a "DDU unique word" near the end of the event packet

Trail-1

decoded the same way as Header3 above

Trail (S-Link64 DAQ Control word)

bit[63:60] = 0xA       //hard-coded in DDU firmware by CMS directive
bit[59:55] = 4-bits unspecified       //CMS directive, no details yet
bit[55:32] = 24-bit DDU Word Count for the Event       //from DDU
bit[31:24] = 8-bit Event Status       //from DDU, by CMS directive

*currently equivalent to Trail-1 bits [47:40]

bit[23:20] = 4-bit unspecified       //CMS directive, no details yet
bit[19:4] = 16-bit Event CRC       //CMS directive, no details yet
bit[3:0] = 4-bits reserved       //CMS directive, no details yet

*special "S-Link64 Protocol" status bits

Link to Production DDU (small red DDU) Output Data Format.

• Useful for Slice Test from October 2005 and beyond.

• Useful for beamtest data from October 2004.
• Also good for big red DDU at CERN Slice test (old DDU with updated firmware).

Revision Summary:

• 23 October 2002: Original version (J. Gilmore, gilmore@mps.ohio-state.edu)
• 5 December 2002: Updated status decode in Header3 (bit57, 40, 39) and added comments in table (J. Gilmore)
• 1 May 2003: Modify trigger board info now that TMB and ALCT have separate data flows (J. Gilmore)
• 19 April 2004: Fix error in Header1 "fov" description (J. Gilmore)
• 4 May 2004: Fix typo in bit50 (J. Gilmore)
• 24 Aug 2005: Update for production history (J. Gilmore)