DOCSIS 3

DOCSIS 3.0 Tutorial – Introduction

October 12, 2009 Brady 18 Comments

DOCSIS 3.0 Gets Fast

This is the first of a new series of Tutorials focused on the Data Over Cable Service Interface

Specifications (DOCSIS) version 3.0. I will make the assumption that you are familiar with the

DOCSIS 1.x / 2.0 standards or have already reviewed my DOCSIS Basics Tutorial as I will be

using many terms without explanation since they were previously covered.

The DOCSIS 3.0 specification is an extension of the DOCSIS 1.x and 2.0 specification which

dramatically increases the data throughput by adding a technology known as channel bonding to

the DOCSIS downstream and upstream, adding increased security, adding support for IPv6, and

substantially improving the back-office management support (MIBs, SNMP, IPDR, etc.) for

DOCSIS. Each of these topics will covered in much greater detail in this DOCSIS 3.0 tutorial in

multiple posts yet to come.

First and foremost, DOCSIS 3.o is most recognized for its dramatic downstream and upstream IP

data throughput capabilities. Typically these are four times those that DOCSIS 2.0 can support

in the downstream / upstream and 12 times what DOCSIS 1.x can support in the

upstream. Without getting into the details yet, this is accomplished by using a rather simple

concept called “channel bonding”. In DOCSIS 1.x and 2.0, the CMTS transmits data to cable

modems using one downstream QAM RF upconverted channel (“DS channels” for short). The

DOCSIS 3.0 specification has developed some unique methods to allow new CMTS

architectures to communicate with DOCSIS 3.o modems using four DS channels. So if a 256-

QAM channel can transport 38 Mbps, then four 256-QAM bonded channels can transport 152

Mbps. Similarly, DOCSIS 1.x/2.0 cable modems transmit data to the CMTS using one upstream

RF channel using a number of different digital modulation schemes in a TDMA (or S-CDMA)

format. In DOCSIS 3.0, the specification allows for up to four upstream channels to be bonded

together.

The following table shows the speed increases that can be expected for the downstream and

upstream in both DOCSIS and Euro-DOCSIS systems when channel bonding is used in a

DOCSIS 3.0 system.

DOCSIS 3.0 Speed Comparison

Note that this chart also shows a DOCSIS 3.o downstream with “(8 channels)” bonded, yielding

a data throughput of 343.04 Mbps. When I stated channel bonding allowed four DS channels to

be bonded, that indicated this was just a minimum. Broadcom, one of the chipset vendors that

makes cable modem chipsets, currently has a chipset solution which supports eight bonded

downstreams. Many cable operators are looking at using more than four bonded downstream

channels. Either to have the ability to provide more data bandwidth to the end user or to provide

both data and MPEG/IPTV set top box capabilities in the same chipset solution. It is important

to understand this capability of the DOCSIS standard, the CMTS, the home (premise) equipment

and potentially the test equipment used for installation, all of which will be covered later.

Summary

DOCSIS 3.0 offers tremendous capabilities over previous revisions to the specification. The

drivers for the new specification have been part do to new data intensive applications, but much

more as a response to competitive threats such as Verizon’s FIoS and AT&T’s U-

Verse. DOCSIS 3.0 does offer much more than just speed, such as a distributable architecture,

enhanced security, support for IPv6, enhanced back office management support and much

more. DOCSIS 3.0 does require a fork-lift upgrade if existing headend equipment is very old,

however some vendors have migration paths for their carrier class platforms such as Cisco’s

ubr10k, Motorola’s BSR 64k and Arris’ C4 CMTSs. In either case, the future (at least for the

next five years) is DOCSIS 3.0 and there are significant business cases to justify the investment

in migrating to the new technology.

DOCSIS 3.0 Tutorial – CMTS Architecture

June 21, 2010 Brady 4 Comments

CMTS Architecture :: I-CMTS vs. M-CMTS

Before we dive into bits, bytes and protocol, first we will discus some hardware. During the

evolution of DOCSIS 3.0 there were a number of interesting interim steps along the way, shall

we say building blocks, to arrive at a full blown D3.0 CMTSs. This left us with two

fundamentally different system architectures in production by CMTS vendors, Integrated and

Modular CMTSs. It is important to understand these “architectures” from a purchasing,

operational and deployment standpoint as they have different requirements in some cases, some

are better than others depending on the system layout.

Integrated CMTS (I-CMTS)

A DOCSIS Integrated CMTS architecture, or I-CMTS, is one that we are more accustomed to

based upon DOCSIS 1.x and 2.0 architectures. That is a pizza box or chassis-based CMTS

whereby all components necessary for DOCSIS 3.0 operation are integrated. The first CMTS

vendor to gain full DOCSIS 3.0 qualification was CASA Systems, who provides a complete

DOCSIS 3.0 CMTS and edge QAM (eQAM) [eQAMs will be described in a later blog] in a 1RU

platform. The benefits of the I-CMTS architecture are its ease of deployment and lowered cost

since all components are integrated. Seemingly there should be fewer points of failure since

cabling and interfaces between external components are eliminated.

The I-CMTS can be found in Arris CMTSs. A strength of this architecture is the all-one

solution, minimizing the amount of RF and IP combining in the headend, potentially reducing

the number of points of failure. Though some of the weaknesses may arguably density and the

inability to remotely locate DOCSIS DTI time sources [to be defined in more detail later] in hub

sites, however these are fine selling points which I will leave to the Sales Engineers much more

qualified to defend their products than I.

Modular CMTS (M-CMTS)

A DOCSIS Modular CMTS, or M-CMTS, is one in which DOCSIS 3.0 channel bonding is

achieved by using a conventional CMTS and extending it with an eQAM, DOCSIS DTI time

source, and Gigabit Ethernet (GigE) interfaces. The conventional CMTS line card provides a

Local or Primary downstream, which contains all of the DOCSIS MAC layer protocol

information necessary for DOCSIS-to-cable modem communication. The basics of this

information contain timing Sync messages, Upstream Channel Descriptors (UCDs), Upstream

Bandwidth Allocation Map (MAP) messages, Ranging messages, etc. The CMTS line card also

provides the inputs for the upstream channels returning from the cable modems, which are then

bound by the CMTS. The CMTS will usually have a specialized interface on it with a GigE

connector that communicates directly with an eQAM. The eQAM provides the additional

downstream bonded channels transmitted to the cable modems. Extremely accurate timing is

maintained between the CMTS and the eQAM via a DOCSIS DTI timing reference. The eQAM

and the CMTS can be remotely located and multiple DTI timers used and synchronized via GPS

to ensure stable clock references. This enables the eQAMs to be located in hub sites where local

video distribution is occurring while the CMTS remains at the headend or visa-versa. The

modularity of the M-CMTS architecture is extremely advantageous in large systems. eQAMs can

be re-purposed for DOCSIS or video as demand calls. The system scales well from four bonded

channels, to eight and more as necessary.

Challenges that come with the M-CMTS architecture are on both the RF and IP fronts. In the RF

realm, there is a vast amount of RF combining that must occur between the CMTS and the

eQAM. At a minimum, headends / hubsites will require significant re-cabling. During this time,

one must consider the potential for isolation issues that may occur between different

downstreams and upstreams as combining is created for different node combinations. Isolation

issues which occur at the source will not be resolvable at the subscriber premise.

In the IP realm, multiple potential failures are introduced with M-CMTS architecture. GigE

interfaces are introduced between the CMTS to the eQAM, the CMTS to the back office, and the

DTI timers to both the CMTS and the eQAMs. One should ensure that each interface has a

redundancy and that the redundant path is fully tested under load prior to any live network

failure. Often has been the case where a redundant path has been in place, but has never been

tested. When the redundant path was actually exercised it failed to operate or failed to operate at

full line rates. Again, the details of M-CMTS architecture advantages are best left to Sales

Engineers and I recommend that you see your vendor for specific recommendations. For all

Salesmen and Sales Engineers, please take my comments lightly, your welcome to send me your

recommendations, but I have both I-CMTS and M-CMTS architectures and like them equally.

DOCSIS 3.0 Tutorial – M-CMTS Architecture

June 26, 2010 Brady 2 Comments

In my post on DOCSIS 3.0 CMTS Architecture, I covered at a high level, the differences

between Integrated and Modular CMTSs (I-CMTS and M-CMTS). In this article I am going to

further explore the M-CMTS in order to describe two import elements of DOCSIS 3.0 network,

the edge-Quadrature Amplitude Modulator or EQAM and the DOCSIS Timing Interface

Specification Server or DOCSIS Timing Server. Before I cover these components I will show

how they are integrated with the M-CMTS architecture. As a recap, however, I came across the

following video which reviews once again the differences between the two architectures. This

six minute video is at a rather high level, but is a perfect primer for those of you who may be

more visually oriented.

If you watched the video above or you are familiar with general M-CMTS architectures, you will

be familiar with the following diagram:

M-CMTS Architecture Diagram

In today’s DOCSIS 3.0 CMTSs, the M-CMTS core and Upstream Receiver reside within the

CMTS mainframe. In the case of a Cisco 10k or Motorola BSR64k, these can be configured

as modular CMTSs provided they have the correct CPU, IOS, line cards, and special wideband

interface cards for communicating with the DOCSIS Timing Server and Edge QAM. From the

line card perspective, the upstream ports are utilized as they have been previously, although

special consideration will need to be taken if and when upstream channel bonding is

implemented. The downstream ports on the CMTS line card are used as the Primary

Channels. Primary channels in a DOCSIS 3.0 network are those channels which carry

information needed by cable modems for registration and network operation. Later on I will

discuss in more thorough detail the different types of downstream channel types, such as

primary, local, and secondary downstream channels.

A DOCSIS 3.0 CMTS vendor will provide some additional specialized DOCSIS 3.0 cards. One

of these cards will typically be a gigabit Ethernet card that enables connectivity to the DOCSIS

Timing Server using the DOCSIS Timing Interface specification (DTI). The connection to the

DOCSIS Timing Server is usually implemented via a CAT5e cable. It is also strongly

recommended that each CMTS be connected to a second DOCSIS Timing Server for redundancy

and so you will find two RJ-45 ports for the DTI Server connection.

The second DOCSIS 3.0 specialized interface card is the one for communicating between the M-

CMTS core and EQAM. This card will typically have a GBIC interface so that it can be

configured with either copper or (CAT5e) or optical (multi-mode fiber) between the CMTS

chassis and the EQAM. There are multiple ports because CMTS will likely be transferring more

traffic in the downstream than one gigabit Ethernet port can handle. From a configuration

standpoint, once the wideband interface card is added, all of the DOCSIS 3.0 downstream

channel bonding commands are enabled in the DOCSIS 3.0 IOS that is installed in the system. It

is at this point that configuration of the downstream bonding groups to their respective MAC

address in the EQAM occur. While configuration of a CMTS for downstream channel bonding

is described in all vendor documentation, it is rather elaborate and beyond the scope of this blog

to fully describe the configuration process. Once properly configured, the CMTS core will be

bound to the EQAM over the physical connection between the wideband card and

EQAM. Downstream data is transmitted over the primary (CMTS) downstream channel and

over the bonded EQAM channels as defined in the CMTS running configuration. The link

between the CMTS wideband card and the EQAM uses a DOCSIS 3.0 protocol called

Downstream External-Phy Interface (DEPI). DEPI is based upon the L2TPv3 protocol, which in

layman’s terms is a very good protocol for transmitting multiple streams of IP data over the same

physical media. The reason this is important in the DOCSIS 3.0 network is because the DEPI

must be able to manage not just data, but also video while managing QoS. I will cover

downstream DEPI protocol in another article.

In summary, a DOCSIS 3.0 M-CMTS is made by first taking a typical CMTS chassis and

upgrading it – rather significantly actually because the costs are not trivial. This upgrade enables

connectivity to two very important pieces of equipment; 1) a DOCSIS time server which ensure

that the M-CMTS core, Edge QAM and upstream are synchronized to nanosecond levels to

support the existing DOCSIS requirements for frequency and timestamps that existed in the

traditional CMTS; 2) an EQAM, which enables multiple downstream RF channels and channel

bonding. In my next couple of articles I will cover the DOCSIS time server and the EQAM in

more detail as these are critical elements of a DOCSIS 3.0 network.

DOCSIS 3.0 Tutorial – The EQAM

July 01, 2010 Brady 1 Comment

In my article on DOCSIS 3.0 M-CMTS architecture, I talked about

the distributed nature of the CMTS with an M-CMTS core (the CPU of the system), a DOCSIS

Timing Server, and an edge Quadrature Amplitude Modulator (EQAM). I am going to cover the

EQAM in detail in this article because in the past couple of years, EQAM (also spelled eQAM)

has rapidly become part of our vocabulary but its operation and value often go

unappreciated. Further, in order to fully understand DOCSIS 3.0 operation, downstream channel

bonding, and possible issue which may arise, a thorough understanding of the eQAM is critical.

An eQAM is also used for switched digital video (SDV), video-on-demand (VOD) in addition to

DOCSIS 3.0 transmission. This makes it a universal piece of equipment to have in any headend,

hubsite or telco VHO/VSO operating broadband video communications (i.e. FiOS). So let’s first

take a look at what the inputs to a typical eQAM consist of, then how IP data is mapped in the

device and ultimately to an a radio frequency (aka RF).

Inputs

Unlike conventional QAM modulators that have a DVB ASI input, eQAM modulators have

multiple Gigabit Ethernet (GigE) data inputs. At a minimum, they should have four GigE inputs.

(Carrier-class systems may have multiple 10 GigE inputs).

Why four minimum? Typically eQAM modulators will be able to generate a minimum of 48

QAM channels. We know that at 256 QAM, video channels can transport about 37 Mbps of

video data. So 48 * 37 Mbps = 1.776 Gbps, which is more data than one GigE port can handle.

So right away, we need two ports to support all of the data we may be feeding the device.

Now we must take into account the requirement for redundancy, which will add one port each to

the two we already require, bringing the total to four.

In addition to GigE ports, eQAM modulators will have provisions for DOCSIS Timing Interface

(DTI) clock inputs, with redundancy. A DTI server is an extremely precise reference clock

defined in the DOCSIS 3.0 DTI specification, which aims to ensure that the eQAM modulator

and the DOCSIS 3.0 CMTS are concisely synchronized.

As previously discussed, in DOCSIS 3.0 M-CMTS configurations, the CMTS core and the

eQAM modulator are physically located in two separate chassis. The DTI server clock is

connected to each chassis in order to maintain accurate timing, synchronization and jitter

between the CMTS and eQAM downstreams. This is a necessity in order to provide high quality

of experience (QoE) for voice over IP (VoIP) and IPTV applications in a DOCSIS 3.0 bonded

channel environment to our subscribers.

DTI servers enable the ability to be remotely synchronized via GPS satellites, so an eQAM

modulator in a hub site can be time-aligned with the CMTS in a headend. In the unlikely event of

DTI server failure, the backup DTI server should be online and connected to the modulator’s

secondary port. Never forgo your backup regimen!

The actual content that you would send to an eQAM over the GigE interface would generally be

framed in an MPEG-2 transport stream (TS). A function of the eQAM is to support both Single

Program Transport Stream (SPTS) and Multiple Program Transport Streams (MPTS).

Additionally, the eQAM can support unicast video (VoD), multicast video (Switched Digital

Video), and M-CMTS (DOCSIS framing in the MPEG-2 TS). The eQAM can also re-stamp

PCR timestamps for de-jitter processing, which helps reduce network impairments. It can also

support various levels of video compression, such as MPEG-4 and new standards that are

emerging. From an IT infrastructure perspective, it can very quickly be seen the power an eQAM

can have on network efficiencies.

Mapping

Let’s say you have an MPEG-TS stream (aka video) or a DOCSIS channel that you want to

associate with a physical QAM modulator port. How does it occur in the eQAM device?

Through simple IP address and UDP (User Datagram Protocol) mapping to each QAM channel.

Here’s how.

The GigE port of the eQAM modulator is manually assigned an IP address, the same way that

you assign your PC an IP address if you took it out of DHCP mode and filled in the information

manually. Most eQAM modulators have an internal Web page that let you do this for each GigE

port.

Next, the device gives you a list of all of the QAM channels that it has available and lets you set

the RF frequency and RF power based on standard HRC/IRC channels for Annex B. (You can

also configure non-standard channels). If you use an international channel plan such as Annex A

or C, then you would select the appropriate channel plan accordingly.

Finally, there is a UDP port setting for each channel. For example, I might set 501 MHz to port

49120, 507 MHz to port 49140, 513 MHz to port 49160, and so on. You get the picture. I am

basically incrementing my frequency in 6 MHz steps and incrementing my UDP ports in integers

of 20.

Now, when you want to send a video stream to a particular QAM frequency, you must only enter

the IP address of the GigE port and UDP port of the QAM frequency that the video needs to be

sent out on. You will also need the appropriate backend video content delivery system—a

separate topic for my blog. (See bradyvolpe.com.)

Further, each QAM channel will support more than one video stream. In my example above,

mapping is usually done on a more granular level.

Let’s suppose we want to send two high definition (HD) and one standard definition (SD) video

channels on 501 MHz. We would then map 501 MHz with three UDP ports as follows: 501.1 =

49120; 501.2 = 49122; 501.3 = 49124. The first HD would be streamed to UDP port 49120, the

second to UDP port 49122 and the SD to UDP port 49124.

Outputs

The outputs of the eQAM modulator are standard F connectors. (MCX RF connectors are

available in high-density platforms.) Most manufacturers will bond four ITU Annex B QAM

channels to one (1) physical RF connector. So a 48 QAM channel eQAM modulator will

typically have 12 RF connectors.

From here conventional RF combining schemes apply, though one exception must be taken into

consideration.

A typical QAM modulator/RF upconverter has a maximum RF output of 60-61 dBmV. An

eQAM modulator with all four outputs enabled on an RF connector has a typical maximum

output of 54 dBmV. This means you may have to make adjustments to your current combining

schemas in your headend when migrating from legacy QAM to eQAM equipment.

Channel Bonding

Finally we come to the crux of DOCSIS 3.0. In the running-config of a DOCSIS 3.0 enabled

CMTS, you will map a number of desired DOCSIS channels to the UPD ports in the eQAM. It

is also necessary to MAP the MAC address of the next hop, which may be the eQAM itself or an

IP switch which may be in between the CMTS and the eQAM. This mapping configuration tells

both the CMTS and eQAM how many Downstream Channels are possible to be bonded. Can it

be more than four? Absolutely. Today it is common to bond eight downstream channels

because the current production chipset support eight channels. There are chipsets in

development that will support 12 or more downstream bonded channels. That is a lot of data –

or a lot of different possibilities, such as high bandwidth data, along with multiple IP TV

channels, video conferencing, dedicated gaming, etc. Follows is an example of a partial

configuration for an eight channel mapping in Cisco CMTS running-config:

controller Modular-Cable 1/0/0

ip-address 10.0.0.25

modular-host subslot 5/1

rf-channel 0 cable downstream channel-id 24

rf-channel 0 frequency 603000000 annex B modulation 256qam interleave 32

rf-channel 0 ip-address 10.0.0.13 mac-address 0024.1494.cc6e udp-port 53248

rf-channel 1 description Used for WB channel 0

rf-channel 1 cable downstream channel-id 25

rf-channel 1 frequency 609000000 annex B modulation 256qam interleave 32

rf-channel 1 ip-address 10.0.0.13 mac-address 0024.1494.cc6e udp-port 53249

rf-channel 2 cable downstream channel-id 26

rf-channel 2 frequency 615000000 annex B modulation 256qam interleave 32

rf-channel 2 ip-address 10.0.0.13 mac-address 0024.1494.cc6e udp-port 53250

rf-channel 3 cable downstream channel-id 27

rf-channel 3 frequency 621000000 annex B modulation 256qam interleave 32

rf-channel 3 ip-address 10.0.0.13 mac-address 0024.1494.cc6e udp-port 53251

rf-channel 4 cable downstream channel-id 28

rf-channel 4 frequency 627000000 annex B modulation 256qam interleave 32

rf-channel 4 ip-address 10.0.0.13 mac-address 0024.1494.cc6e udp-port 53252

rf-channel 5 cable downstream channel-id 29

rf-channel 5 frequency 633000000 annex B modulation 256qam interleave 32

rf-channel 5 ip-address 10.0.0.13 mac-address 0024.1494.cc6e udp-port 53253

rf-channel 6 cable downstream channel-id 30

rf-channel 6 frequency 639000000 annex B modulation 256qam interleave 32

rf-channel 6 ip-address 10.0.0.13 mac-address 0024.1494.cc6e udp-port 53254

rf-channel 7 cable downstream channel-id 31

rf-channel 7 frequency 645000000 annex B modulation 256qam interleave 32

rf-channel 7 ip-address 10.0.0.13 mac-address 0024.1494.cc6e udp-port 53255

eQAM Benefits

Why has the industry migrated to the eQAM moduators? Let’s name a few reasons:

They offer a lot of QAM channels in a small form factor. They enable new services such as SDV and VOD by seamlessly enabling an operator to change

video content from one QAM channel to another just by changing a UDP port number. They help make complex cabling a thing of the past. They enable diverse video inputs with variable video compression levels. They enable DOCSIS 3.0 downstream channel bonding. Finally, they enable convergence. The same eQAM modulator used for video distribution can be

used simultaneously for DOCSIS 3.0 data transport. As data demands grow, video channels can be re-purposed for more DOCSIS channels.

Well, apologies for the very long article, but I am quite excited about eQAMs. I should probably

write a little more on the technical side, but I try to stick to the nuts and bolts and not get into the

ball bearings. Hang in there, soon we will be getting deeper into the details of DOCSIS 3.0

protocol.

DOCSIS 3.0 Tutorial – DOCSIS Timing Interface

Specification

July 05, 2010 Brady No comments

Before DOCSIS 3.0 and before modular CMTS architectures, a CMTS

existed in one chassis. Life was much simpler for everyone. Inside the chassis existed a 10.24

MHz clock or oscillator. This was a master time keeper that kept every event in synchronization

with every other event. Timing is very important in communications networks, especially when

dealing with the microsecond timing calculations necessary for DOCSIS transport – remember

the “tick” (6.25 usec). This article is going to address the DOCSIS Timing Interface

Specification (DTI) and DTI time servers that have arisen due to the distributed architectures in

M-CMTSs and DOCSIS 3.0 CMTSs. In these architectures, it is possible to have the CMTS

core in say the headend, with the eQAM and upstream receivers in remote hubsites. Suddenly

the single 10.24 MHz clock keeping the system in synchronization is no longer an option. Three

separate, free running 10.24 MHz clocks would also not work because they would not be in

phase and would not be running at the same frequency, causing the entire system to out of

synchronization – there would packet collisions and lost data and VoIP packets all over the

place. It would be chaos! So the smart folks at Cablelabs put together the DTI specification to

resolve these issues. Here are some of the details.

The following is taken from Figure 1-1 in the DOCSIS Timing Interface Specification located

both on the Cablelabs website and in the Library.

DOCSIS Timing Interface Specification - DTI

This diagram illustrates several important details. First, there is a primary DTI server, called the

Root Server. The DTI root server provides a 10.24 MHz clock source. Second, there is (actually

it is strongly recommended) a DTI Slave Server. The DTI slave server is nothing more than a

redundant 10.24 MHz clock that monitors the root server. In the event that the root server should

fail, the slave server will automatically provide a 10.24 MHz reference clock to all sources. The

current vendor of DTI servers has the provisions in their firmware to monitor root and slave,

providing for minor and major alarms when there are problems with the servers. The DOCSIS

DTI Specification also also defines the method by which the root and slave communicate with

each other over the IP network, so be certain to provision all timing servers. Next, the DTI

servers provide 10.24 MHz clock sources to the eQAM, the CMTS and the upstream

receivers. As indicated at the beginning of this article and in this diagram, all three of these

devices can be physically located in different buildings. Since it is not practical to run very long

CAT5 cables between Headends and Hub sites, DTI time servers are installed in each

location. Synchronization of time servers between the locations is recommended (though not

required). The way that the time servers are synchronized is via the GPS (Global Positioning

Satellite) antenna port on the time servers or network streams. The time servers synchronize

themselves to the global time server signal originating from the GPS satellites, ensuring that

every time server connected to a GPS is synchronized to within 100 nsec of every other time

server in the network. If the GPS option is not used, it is recommended to use a network-based

source such as PDH, SONET, or equivalent.

In addition to being a highly accurate time source, the DTI server can also function as an NTP

(Network Time Protocol) server for the cable modems and other devices in your network. This

will provide IP devices with the time of day during the DHCP process. This is a function of the

Symmetricom time servers, which at the time of this writing, is the only game in town for getting

time servers. But their equipment works quite well and I have found both their technical support

and RMA departments quite competent.

To dive a little deeper into the implementation end of things, here are some considerations that

one should consider when setting up DTI servers:

Traditional CAT5e cables works fine for interconnects Lengths of cables may not exceed 200 meters When using GPS mode, it can take up to three (3) days, worse case, to synchronize the time

server clock with the GPS clock. This is called Time of Coincidence (ToC) and is explained in more detail in the DTI Specification. Symmetricom’s Time Creator 1000 makes this process a little smoother, but don’t get concerned if it takes a while

Shutdown all unused DTI ports on the time servers as if they are not connected they will give you errors

Modes of Operation

The DTI Server can support three modes of operation:

1. Free running Master: no external source is provided to control frequency or timestamp accuracy 2. GPS Traceable: source of timestamp and frequency traceability is GPS 3. Network Traceable: a standard network PDH, SONET, or SDH interface is used to provide

traceability to a Stratum 1 frequency reference

In M-CMTS, deployment where all the elements are located in the same area GPS traceability is

not required. The DTI server should be connected to all devices.

For distributed applications, the DTI Specification states (in items #2 and #3 above) that a GPS

or network traceable source must be used if the network shows degradation as measured over a

35 second period of time. What types of degradation is the DOCSIS specification looking

for? At the very minimum, the specification states a BER of <1e-8, but it further details

requirements for synchronization jitter, timing, skew, etc. So it is important to understand that

just good BER is not an indication that the system is meeting adequate timing. Other timing

impairments could be present which will impact voice and video that will not impact data only

applications.

In summary, DOCSIS 3.0 with DTI servers has upgraded the timing in DOCSIS networks from a

Tier 3, low end timing service to a much higher end, telco quality service. Using a DTI server in

collocated M-CMTS architecture provides for a Tier 2 timing service with 5 nsec of timing

resolution between the devices. Using multiple DTI servers between remotely distributed

devices in an M-CMTS architecture traceable to network or GPS timing elements provides for a

Tier 1 timing service with 100 nsec accuracy between each device. This is unprecedented timing

accuracy in our continuously improving DOCSIS networks enabling evolving service offerings

with enhanced quality of service and quality of experience.

DOCSIS 3.0 Tutorial – Basic Protocol 1

July 07, 2010 Brady 1 Comment

Now that we have established the two primary architectures available in

DOCSIS 3.0, I-CMTS and M-CMTS (though hybrids do exist), and the hardware components of

these architectures, it is time to delve into the protocol of the DOCSIS specifications that make

up DOCSIS 3.0. There are five primary specifications that I will be drawing upon from here on

out listed below and located in the Library and also on the CableLabs website. Here are

documents to focus on if you really want some heavy reading:

DOCSIS 3.0 Series of Specifications (found in Library)

CM-SP-PHYv3.0 Physical Layer Specification CM-SP-MULPIv3.0 MAC and Upper Layer Protocols Interface Specification CM-SP-OSSIv3.0 Operations Support System Interface Specification CM-SP-SECv3.0 Security Specification CM-SP-CMCIv3.0 Cable Modem CPE Interface Specification

I will be taking key excerpts from these documents and boiling them down into a more legible

format in my blogs from here on out to focus on the key enhancements in the DOCSIS 3.0

specification that separate it from previous versions of the DOCSIS standard. Here are some

areas to start with that will be explored in much more detail later:

Downstream Channel Bonding with Multiple Receive Channels: DOCSIS 3.0 introduces the concept of a cable modem (CM) that receives simultaneously on multiple downstream channels. Downstream Channel Bonding refers to the ability (at the MAC layer) to schedule packets for a

single service flow across those multiple downstream channels. It is Downstream Channel Bonding that gives DOCSIS 3.0 the ability to provide throughput speeds from 150 Mbps and up, depending on the number of channels bonded together.

Upstream Channel Bonding with Multiple Transmit Channels: DOCSIS 3.0 introduces the concept of a CM that transmits simultaneously on multiple transmitting upstream channels. Upstream Channel Bonding, refers to the ability to schedule the traffic for a single upstream service flow across those multiple upstream channels. Upstream Channel Bonding offers significant increases in the peak upstream data rate that can be provided to a single CM. It is Upstream Channel Bonding that enables upstream data rates of over 100 Mbps.

IPv6: DOCSIS 3.0 introduces built-in support for the Internet Protocol version 6. DOCSIS 3.0 CMs can be provisioned with an IPv4 IP address, an IPv6 IP address, or both. Further, DOCSIS 3.0 CMs can provide transparent IPv6 connectivity to devices behind the cable modem (CPEs), with full support for Quality of Service and filtering. IPv6 is not just new technology, but a requirement for many MSO who are running out of IPv4 IP addresses for the numerous subscribers and IP devices in the network.

Source-Specific Multicast: DOCSIS 3.0 supports delivery of Source-Specific IP Multicast streams to CPEs. Rather than extend the IP multicast protocol awareness of cable modems to support enhanced multicast control protocols, DOCSIS 3.0 takes a different approach. All awareness of IP multicast is moved to the CMTS, and a new DOCSIS-specific layer 2 multicast control protocol between the CM and CMTS is defined which works in harmony with downstream channel bonding and allows efficient and extensible support for future multicast applications. This becomes important for network efficiency as well as multicast storm suppression when devices have problems. In effect it puts a VLAN in the DOCSIS network for DOCSIS Multicast traffic.

Multicast QoS: DOCSIS 3.0 defines a standard mechanism for configuring the Quality of Service for IP multicast sessions. It introduces the concept of a “Group Service Flow” for multicast traffic that references a Service Class Name that defines the QoS parameters for the service flow. This is critical for enabling QoS for clients behind a cable modem and not just cable modem attached clients, such as an eMTA. It greatly increased the ability of cable operator to improve QoS for new IP services in the home.

CMTS Forwarding Model

I have had many comments on previous posts about DOCSIS CMTSs and whether or not they

are Layer 2 or Layer 3 devices, so I believe it is important to clarify this on the DOCSIS 3.0

specification, which has not changed much from previous revisions of the spec. The CMTS

relays upstream and downstream data to a forwarder in the CMTS. The forwarder can function

as a Layer 2 or a Layer 3 device (per the OSI model). As a Layer 2 device, the CMTS forwarder

is acting similar to an IP switch and transparently moving packets based without altering the

packets in any way. As a Layer 3 device, the CMTS forwarder is now a Router and has value

add. It looks at MAC address, IP address, metrics to least-cost-path and routes packets

accordingly. In this mode, the CMTS may even bridge one IP domain to another and so the

source IP and MAC addresses can be changed when they arrive at the destination. The DOCSIS

specification leaves it up to the vendor (and sometimes the end user) as to whether Layer 2 or

Layer 3 forwarding is used. It is my recommendation that the CMTS forwarder always be

deployed as a Layer 3 device to segregate your DOCSIS network from your IP network for many

reasons, security being one of the biggest. One thing that does change in DOCSIS 3.0 is that the

MAC Domain is not considered to forward data packets from its upstream to its own

downstream channels; all upstream data packets are considered to be delivered to a CMTS

forwarder. DOCSIS 3.0

leaves most details of CMTS forwarder operation to CMTS vendor-specific implementation.

The MAC Domain

Another important concept that I have not covered before is that of the MAC domain. A MAC

domain consists of at least one downstream on one upstream port and contains all of the

functions necessary to operate a DOCSIS network (DOCSIS protocol, IP connectivity,

etc.). Now your typical MAC domain would be one downstream channel and four or more

upstream channels on a line card. Most line cards have more than one MAC domain because

they have more than one downstream and eight or more upstreams (say a 2×8 card for example

would have two MAC domains). It’s that simple. So if you have been confused when people

were talking about MAC domains, now you know the jargon.

Downstream MAC Domain Terminology – DOCSIS 3.0

A MAC Domain provides downstream DOCSIS data forwarding service using the set of

downstream channels associated with the MAC Domain. A downstream channel is defined as

either:

A “Downstream (RF) Channel”, representing a single-channel downstream RF signal on a Downstream RF Port of an Integrated CMTS; or

A “Downstream M-CMTS Channel”, representing a single-channel downstream RF signal at a remote Edge QAM that is reached via a DEPI tunnel from an M-CMTS Core.

Upstream MAC Domain Terminology

An “upstream channel” can be used to refer to either:

A “Physical Upstream Channel”; or “Logical Upstream Channel” of a Physical Upstream Channel.

A “Physical Upstream Channel” is defined as the DOCSIS RF signal at a single center frequency

in an upstream carrier path. Multiple “Logical Upstream Channels” can share the center

frequency of a Physical Upstream Channel, but operate in different subsets of the time domain.

Transmit opportunities for each Logical Upstream Channel are independently scheduled by the

CMTS.

So I have run out words for this article, but I have just touched on the beginning of DOCSIS 3.0

terminology, especially in the downstream. It is critical that you learn the differences in naming

conventions because a downstream is not just a downstream in DOCSIS 3.0 and you will

definitely want to keep your physical, primary, secondary and logicals straight when

communicating with others DOCSIS 3.0 channels.

DOCSIS 3.0 Tutorial – Downstream Channel Bonding

July 18, 2010 Brady 2 Comments

Downstream Channel Bonding is perhaps the ball bearings of

DOCSIS 3.0, enabling subscriber data speeds in excess of 160 Mbps (4 times that of previous

DOCSIS versions). While conceptually simple, the principle of combining multiple downstream

DOCSIS channels together to carry the same user data must have tight constraints in order to

preserve the integrity of the data and have the data arrive at the correct subscriber’s device and in

sequence. This article will cover both the physical layer aspects and DOCSIS protocol aspects

that enable channel bonding.

Physical Layer

Channel bonding simply means that the CMTS knows that there are four or more RF signals

within a 60 MHz passband (greater if more than four channels are bound). The 60 MHz window

is defined in section 6.3 of the DOCSIS 3.0 RFI and is really intended more for the cable modem

receiver than it is for the CMTS/eQAM transmitter. The CMTS/eQAM have very substantial

dynamic range when it comes to transmitting across a broad range of frequencies, however in

order to keep cable modem (CM) costs low, a broadband tuner is implemented in the CM. It was

determined that a 60 MHz bandwidth would be reasonable for cost effectiveness based upon

existing hardware at the time of the specification. These tuners are typically built into the silicon

of the DOCSIS cable modems are no longer discrete tuners as were previously designed in the

past. If more than four downstream channels are tranmistted as part of the Downstream Bonding

Group (DBG), then more than 60 MHz is permitted, but the cable modem must be able to tune to

at least four channels in a 60 MHz bandwidth, support of additional channels outside of the 60

MHz bandwidth is optional and now becoming more of a necessity for cable operators who are

commonly using eight channels in their DBGs – Note, remember the term Downstream Bonding

Group or DBG as it is common lingo for DOCSIS 3.0.

Another change in DOCSIS 3.0 from DOCSIS 1.x and 2.0 is the downstream operational

frequency range. The previous versions of DOCSIS operated down to 88 MHz and typically

topped out at 860 MHz. DOCSIS 3.0 starts at a higher frequency of 111 MHz and goes to 867

MHz as a requirement. Additionally, the specification has a recommendation that 999 MHZ

should be the high end frequency of DOCSIS 3.0, fully utilizing a 1 GHz plant. In a competitive

market, these recommendations are usually taken as “we had better do it to stay in business”,

which has been the case. The reason that DOCSIS 3.0 has a higher starting frequency of 111

MHz is because the upstream specification allows cable modems to transmit up to 88 MHz, the

rational for this will be covered in a later post. Otherwise, other physical characteristics of

DOCSIS 3.0 are similar to DOCSIS 1.x and 2.0. Any channel in the DBG can operate in either

64-QAM or 256-QAM mode. BER must be equal to or better than 1×10-8

and codeword error

rate (CER) must be less than or equal to 9×10-7

. Oh, maybe you have not heard about the new

spec. for code error rate before? Well this is new to the DOCSIS 3.0 specification and is

something that is quite over due since BER is nearly impossible to measure in a live DOCSIS

plant while CER can be obtained right from the CMTS. CER can be computed as follows:

,

Where:

Eu is the value of the count of code words with uncorrectable errors; Ec is the value of the count of code words with correctable errors and; C is the value of the count of code words without errors.

Keep an eye on CER to be a new metric for equipment manufacturers and test vendors to be

using moving forward in addition to BER and MER.

DOCSIS 3.0 Protocol

In a DOCSIS 3.0 network implementing downstream channel bonding, the DOCSIS CMTS

dynamically balances the data across the Downstream Bonding Group (DBG), which can consist

of four or more downstream channels. The reason this is done is to offer subscribers the best

quality of service across downstream channels with changing impairments and changing

congestion at the receive side. Each outgoing packet from the CMTS is tagged with a sequence

number. The sequence number becomes important for a number of reasons. Packets can be

dispersed across different downstream channels and can have different time delays in arriving at

the receiving cable modem. It is then the cable modem’s responsibility to re-synchronize the

incoming packets based upon the sequence numbers. TCP/IP windowing acknowledgments will

take care of any lost packets at Layer 3, however for UDP flows, such as voice and video, those

packets will be forever lost. Further, by dynamically distributing the packets across

downstreams, the CMTS can take advantage of statistical gains of many cable modems

connected to the DBG. This becomes especially critical when your system has a mixture of

legacy DOCSIS 1.x, 2.0 and 3.0 cable modems. The 1.x and 2.0 modems will all be receiving

data from only the Primary Channel of the DBG which the DOCSIS 3.0 modems will be able to

receive data from all four+ downstream channels in the DBG. So this dynamic prioritization is

in effect acting like upstream load balancing in the downstream.

If all downstream channels of the DBG are configured as Primary Downstreams, then DOCSIS

3.0 has another capability to load balance all legacy cable modems across the DBG. This is

called Downstream Channel Set (DCS) and is truly analogous to upstream load balancing. It is

highly recommended that when you are first turning on a DOCSIS 3.0 network with few

DOCSIS 3.0 modems and many legacy modems that you configure all downstream channels as

primary DOCSIS channels and enable DCS. There is a secondary benefit to configuring all

downstreams as primaries and that is you will be able to fully test each DOCSIS primary channel

using your legacy DOCSIS 2.0 hand-held test meters. What is the drawback then? When you

configure a DOCSIS channel as a primary it must carry all of the DOCSIS protocol overhead,

which is about a 15% to 20% loss of user data. So when you have the ability to later configure a

DOCSIS downstream channel to a secondary channel, the secondary channel no longer carriers

the excessive DOCSIS overhead, except for some minimal synchronization information, and so

you can now transport significantly more data to your end subscribers.

So let’s review the downstream terminology as a recap:

Local Downstream – Always Primary o Comes from the CMTS line card in M-CMTS architecture o Comes from the CMTS in I-CMTS architecture

Primary Downstream – Can come from Local Downstream or eQAM o Carries UCD, timing Sync, MAPs, etc. o Required for CMs to register on network Also carries PDU (subscriber data) o Has DOCSIS overhead, so you loose some subscriber data utilization

Secondary Downstream – Comes from eQAM or other bonded channel on I-CMTS o Only carries PDU – No UCD, timing Sync, MAPs, etc. o Cable modem cannot register on Secondary Downstream o Does not support legacy cable modems (non-DOCSIS 3.0) o Best subscriber data utilization capability

Some final take aways from this post

Downstream Channel Bonding has been the heart of DOCSIS 3.0 and is what has made it a huge

success. There have been a number of deployment issues along the way, but that is for another

post. From the physical layer standpoint, DOCSIS 3.0 is very similar to DOCSIS 1.x and

2.0. You will either be dealing with 64-QAM, 256-QAM or a mix of both in your downstream

bonding groups, which must be in a 60 MHz bandwidth. You still need to have good MER, BER

and now you can start learning and using CER as another tool.

Just as you may have been getting acquainted with upstream load balancing, this is a new a

valuable feature in DOCSIS 3.0. Not only are cable modems load balanced in 3.0, but the very

data packets themselves. At first this was done from a statistical standpoint to take advantage of

which modems may be pulling more traffic than others. In a live plant, however it has become

very evident that this feature is valuable for overcoming impairments which may occur on one

downstream channel that do not exist on another. Consider it a self-healing mechanism that can

come into play that gives you an opportunity to resolve a problem on that particular

frequency. It does not fix the plant, because your codeword errors and thus CER will go up

considerably on that channel, your overall data rate will drop for the DBG, but data will still go

through on the other channels at a maximum rate.

DOCSIS 3.0 Tutorial – Upstream Channel Bonding

August 22, 2010 Brady No comments

Since its inception, the DOCSIS 3.0 specification has supported upstream channel bonding. Most

CMTS vendors have not supported this in their firmware until the past year. In most cases the

firmware that is available is still in beta testing in cable operator labs to ensure cable modems

and networks can support it – why? The bonding of two, three or four upstream channels in an

RF DOCSIS network creates a lot of challenges such as finding enough RF spectrum, resolving

overloading return path lasers, isolation issues and much more. Some of these items have been

addressed in the Speeding Upstream Articles written by John Downey and me.

The focus of this article will be on the mechanics of upstream channel bonding and how it works

more from a DOCSIS protocol perspective. Much more detailed information can be found in the

DOCSIS 3.0 MULPIv3.0 document located in the Library, but this will provide a high level

overview for the layman who is curious about the basics. First lets understand that it is the cable

modem that is doing the channel bonding, remember in the upstream the cable modem transmits

data to the CMTS. Per DOCSIS 3.0, the CM can bond from two to four channels in the upstream

as coordinated by the CMTS. The CM is always controled by the CMTS.

Upstream channel bonding has a lot in common with downstream channel bonding. Like

downstream channel bonding, upstream bonding consists of two or more channels active as an

Upstream Bonding Group (UBG). In order to enable multiple upstream channels in a cable

modem, a special mode is turned on in the cable called Multiple Transmit Channel mode (MTC

mode). This is the basic distinction of whether the cable modem is functioning in DOCSIS 2.0

mode or DOCSIS 3.0 mode. Once in DOCSIS 3.0 mode, all of the power level variations

described in Figure 3 of Speeding Upstream Part II are in effect.

At a very basic level, the CMTS assigns each upstream channel with a service flow. The Best

Effort service flow is a just one of many “SID Clusters” that assigns this type of service flow to

each channel in the bonding group. Queuing in the cable modem buffers data and then

bandwidth REQs on any upstream from the CM allow the CMTS to load balance the CM by

sending back MAP Grants to any upstream channel. Why would load balancing be

necessary? For one the CMTS is managing many cable modems, some of which could be legacy

DOCSIS 1.x/2.0 modems using only one of the available channels in the bonding

group. Further, there could be impairments on some upstream channels seen by some

modems. Higher levels of service flows will be established for QoS by the CMTS, this will be

another SID Cluster and spread across the bonding group so that load balancing can also occur

for that particular QoS service flow to achieve the best possible performance in the case of traffic

or impairments.

A channel in a bonding group may become unusable due to say upstream impairments. In this

case, if the CMTS becomes aware of the outage, it will stop providing opportunities for CMs to

transmit on that channel (i.e. MAP grants). If the CM becomes aware, it will stop sending REQs

for data. Both the CM and CMTS will continue to do Initial Ranging (i.e. the cable modem will

continue to establish communication with the CMTS) until the channel become available. So

basically, if an UBG channel becomes unavailable, the DOCSIS 3.0 network will continue to

work with the remaining upstream bonded channels until the down channel is brought back

up. Its smart enough to know not to transmit any channel on the down channel.

So upstream channel bonding can be pretty straight forward. If you dig deeper you will find that

a host of advanced features are supported similar to DOCSIS 2.0 such as Dynamic Channel

Change, fragmentation, concatenation, etc. Of course these are beyond the scope of this blog

and I encourage you to read the spec if you are interested in this level of detail.

DOCSIS 3.0 Tutorial – DOCSIS Does IPv6

September 05, 2010 Brady 5 Comments

Everyone is familiar with Internet Protocol version 4 (IPv4) addresses. You probably even set

them up in your home network, such as 192.168.1.1 IPv4 is described in IETF publication RFC

791 (September 1981), which replaced the previous version RFC 760, dating back to January

1980. So its safe to say that IPv4 has been around for some time and serving us quite well. New

in DOCSIS 3.0 is the support for IPv6. Why do we need this new version? IPv6 has a vastly

larger address space than IPv4. This results from the use of a 128-bit address, whereas IPv4 uses

only 32 bits. Believe it or not, major cable operators are running out IP addresses. This is due to

more customers, not just for cable modems, but also for set top boxes and VoIP

eMTAs. Further, deployed in cable networks are IP devices such as power supplies with

embedded cable modems for monitoring voltage, temperature, current and more. All networks

are getting more IP devices requiring more and more IP addresses, so the 232

addresses allocated

in IPv4 are no longer sufficient and we turn to the 2128

addresses provided in IPv6.

IPv6 isn’t just your basic IP addressing scheme. It has some additional intelligence built into it

that, once enabled, will make networks (including DOCSIS networks) operate a little more

efficiently such as:

Stateless address autoconfiguration and network renumbering and

an automatic mechanism for forming the host identifier

The bullets above can be quite powerfull because a host server can change the prefix of an IPv6

IP address scheme and renumber a complete subnet in order to operate with adjacent

routers. This is something that would normally be a significant effort in an IPv4 network. IPsec

is also mandated as part of IPv6, which adds a layer of security through encryption. IPsec can be

added in IPv4 too, but it is something that the end user must put in as an afterthought. Having it

as a requirement in IPv6 is a step in the right direction to improving network security in the

future.

So now the question comes up, how does a DOCSIS 3.0 cable modem know when to use IPv4

and when to use IPv6? Well, the DOCSIS spec. defines it as follows: The CM performs IP

provisioning in one of four modes: IPv4 Only, IPv6 Only, Alternate Provisioning Mode (APM),

and Dual-stack Provisioning Mode (DPM). So the cable modem has four (4) options to pick

from during registration with a DHCP server. So let’s break each one down.

IPv4 Provisioning Mode

IPv4 Provisioning mode was discussed in detail in my blog on DOCSIS and Cable Modems –

How it works :: Cable Modem Registration. The following diagram is also a quick reminder of

the DHCP Discover, Offer process between a cable modem and a DHCP server:

Cable Modem IP Provisioning IPv4

Oh, and here are some handy fields that you probably are always looking up when setting up

config files – these are the required stuff for provisioning an IPv4 DOCSIS cable modem pulled

directly from the DOCSIS 3.0 specifcation:

The following fields MUST be present in the DHCPDISCOVER and DHCPREQUEST

messages from the CM:

The hardware type (htype) MUST be set to 1;

The hardware length (hlen) MUST be set to 6;

The client hardware address (chaddr) MUST be set to the 48 bit MAC address associated

with the RF interface of the CM;

The client identifier option MUST be included, using the format defined in [RFC 4361];

The parameter request list option MUST be included. The following option codes

(defined in [RFC 2132] and [RFC 4361]) MUST be included in the list:

o Option code 1 (Subnet Mask)

o Option code 2 (Time Offset)

o Option code 3 (Router Option)

o Option code 4 (Time Server Option)

o Option code 7 (Log Server Option)

o Option code 125 (DHCPv4 Vendor-Identifying Vendor Specific Information

Option)

Now I want to include the following information, which I have never covered in the past,

because a collegue of mine recently was inquiring about it. This deals with how a CPE

(Customer Premise Equipment) or any IP device connected to a cable modem gets an IP

address. While the DOCSIS specification I am about to quote is quite clear I want to warn you

that the implementation by vendors can vary greatly. First let me show you what the spec. says

as follows and then I’ll explain.

In order to assist the DHCP server in differentiating between a DHCPDISCOVER sent from a

CM and a DHCPDISCOVER sent from a CPE:

The CMTS DHCPv4 relay agent MUST support the DHCP Relay Agent Information

Option (RAIO) [RFC3046]. Specifically, the CMTS DHCPv4 relay agent MUST add an

RAIO to the DHCPDISCOVER message before relaying the message to a DHCP server.

The RAIO MUST include the 48 bit MAC address of the RF-side interface of the CM

generating or bridging the DHCPDISCOVER in the agent remote ID sub-option field

[RFC 3046].

If the CMTS is a router, the CMTS DHCPv4 relay agent MUST use a giaddr field to

differentiate between CM and CPE clients if they are to be provisioned in different IP

subnets. The DHCPv4 relay agent in a bridging CMTS SHOULD provide this function.

DHCPv4 Relay Agent CMTS Capabilities option, containing the value “3.0″ for the

CMTS DOCSIS Version Number [CANN DHCP-Reg].

Okay, so what the above bullets basically say is that if your CMTS is a router (which it had

better be if you want security – see my Hacking DOCSIS Cable Modems blog), then when a

CPE registers it’s IP, the cable modem is going to append it’s own MAC address along with a

“giaddr” What’s a giaddr? That is a gateway address that is configured in the CMTS running

config. It tells the CMTS how to route IP traffic for a particular IP subnet of cable modems –

this is important because each CMTS can have many IP subnets for cable modems, each on a

different line card. The giaddr helps the CMTS keep each block of cable modems straight

especially when routing traffic out to a main gateway port. So this is the ideal scenario, but in

my experience I have found that some vendors do make mistakes in this implementation and can

accidentally just pass on the CPE MAC address. Since this is a Layer 3 IP network it usually

does not cause any problems unless you trying to troubleshoot the root cause of an issue and

looking for the source cable modem MAC address. If the cable modem MAC address is not

attached to the DHCP Discover, realize this could be a vendor issue and you will need to dig

further.

IPv6 Provisioning Mode

For IPv6 provisioning mode to even occur, it must be communicated by the MAC Domain

Descriptor (MDD) message. Whoa, back up – this is new and I have never covered before, but it

is new to DOCSIS 3.0 also. The MDD is a message transmitted by the CMTS to all cable

modems on the downstream on a periodic basis, much like the UCD, but it contains more

information relevant to registration. Some of this useful information is the number of bonded

downstream and upstream channels, channel profile, early authentication (security), symbol

clock locking, and the IP Provisioning mode [0 = IPv4 Only, 1 = IPv6, only, 2 = Alternate

(APM), 3 = Dual-stack (DPM)]. So if a “1″ is included in the IP Provisioning Mode of the

MDD, then we have a “Go” for IPv6 provisioning. In which case the flow is a little different

from IPv4 and is a follows:

IPv6 Provisioning

So just by looking at the flow chart above, it should me immediately apparent that IPv6

provisioning is radically different that IPv4. A lot of items that were once the burden of a DHCP

server are now the burden of the cable modem or IP device. Some of these include:

Ensuring that the assigned IP address is not already in use, if so the CM must report this

event to the local log and restart provisioning

The CM MUST perform router discovery as specified in [RFC 4861] so that it is aware of

adjacent routers – if this fails, it must restart provisioning

Oh, and here are the list of goodies that are required for a valid DHCP IPv6 provisioning

message per the DOCSIS specification:

The CM sends a DHCPv6 Solicit message as described in [RFC 3315]. The Solicit message

MUST include:

A Client Identifier option containing the DUID (DHCP Unique Identifier) for this CM as

specified by [RFC3315]. The CM can choose any one of the rules to construct the DUID

according to section 9.1 of [RFC 3315];

An IA_NA (Identity Association for Non-temporary Addresses) option to obtain its IPv6

management address;

A Vendor Class option containing 32-bit number 4491 (the Cable Television

Laboratories, Inc. enterprise number) and the string “docsis 3.0″;

A Vendor-specific option containing:

1. TLV5 option (reference [CANN DHCP-Reg] ) containing the encoded TLV5s

describing the capabilities of CM information option in Annex C.1.3.1;.

2. Device ID option containing the MAC address of the HFC interface of the CM;

3. ORO option requesting the following vendor specific options:

o

1. Time Protocol Servers

2. Time Offset

3. TFTP Server Addresses

4. Configuration File Name

5. SYSLOG Server Addresses

A Rapid Commit option indicating that the CM is willing to perform a 2-message DHCPv6

message exchange with the server.

The CM MUST use the following values for retransmission of the Solicit message (see [RFC

3315] for details):

IRT (Initial Retransmission Time) = SOL_TIMEOUT

MRT (Maximum Retransmission Time) = SOL_MAX_RT

MRC (Maximum Retransmission Count) =4

MRD (Maximum Retransmission Duration) =0

The CM MUST use the values for retransmission of the Request message defined in [RFC 3315].

If the number of retransmissions is exhausted before the CM receives an Advertise or Reply

message from a DHCP server, the CM MUST consider IPv6 address acquisition to have failed.

The CM MUST support the Reconfigure Key Authentication Protocol as described in [RFC

3315].

The DHCPv6 server may be configured to use a 2 message Rapid Commit sequence. The DHCP

server and CM follow [RFC 3315] in the optional use of the Rapid Commit message exchange.

The DHCPv6 server responds to Solicit and Request messages with Advertise and Reply

messages (depending on the use of Rapid Commit). The Advertise and Reply messages may

include other configuration parameters, as requested by the CM or as configured by the

administrator to be sent to the CM. If any of the following options is

absent from the Advertise message, the CM MUST discard the message and wait for other

Advertise messages. If any of the following options is absent from the Reply message, the CM

MUST consider IPv6 address acquisition to have failed:

The IA_NA option received from the CM, containing the IPv6 management address for

the CM.

A Vendor-specific Information option containing the following sub-options (refer to

[CANN DHCP-Reg]):

1. Time Protocol Servers option

2. Time Offset option

3. TFTP Server Addresses option

4. Configuration File Name option

5. Syslog Server Addresses option

Okay, that was brutally painful. But I wanted to illustrate that while IPv6 has tremendous value

over IPv4, the transition from IPv4 to IPv6 is not going to be as easy as clicking a button in a

DHCP server – at least not yet in today’s DHCP servers. In fact as you make the transition, it is

highly likely that there will be growing pains and possible system outages as devices get stuck in

an IPv6 provisioning loop. This is one reason that the Alternate Provisioning Mode (APM) has

been provided.

Alternate Provisioning Mode

So as I just discussed, problems will arise during the transition to IPv6. So smart people have

developed the Alternate Provisioning Mode (APM) to help us through this. In the MDD, mode 2

would be identified indicating to the cable modem to use APM. In this case, the CM will first

attempt to provision with IPv6. In the event that IPv6 provisioning fails, for a multitude of

reasons, the cable modem will fall back and attempt provisioning using IPv4. This gives

operators a great opportunity to start trying out IPv6 without risking having subscribers lose

service. There will be a slight delay since there is a time in which two provisioning modes are

happening each time a modem boots, first IPv6 and then IPv4 if IPv6 fails. This can create

serious problems if there is a system-wide outage, in which case it would significantly increase

the load on a DHCP server and also the time for the network to be restored, assuming your IPv6

provisioning system is not working properly.

Dual-Stack Provisioning Mode

To build upon the potential time and outage issues with APM above, Dual-Stack Provisioning

Mode (DPM) is an available option and potentially the best of both worlds. In DPM, the cable

modem attempts simultaneous IPv6 and IPv4 provisioning. If both IPv6 and IPv4 are successful,

the CM can have dual IP addresses and maintain them in a parallel mode. If one fails, it will still

have a successful registration with the other. There can even be cases where portions of one

provision fails, but the overall provisioning is successful because the CM obtains everything it

needs as a hybrid provisioned device. Finally, since both IPv6 and IPv4 occur nearly

simultaneously, if there is system outage, there will be less overall impact to network restore

time since provisioning is back to a near parallel process for IPv6 and IPv4 instead of a serial

process as in APM.

Summary

IPv6 is in high demand in DOCSIS and IP networks and is now available through DOCSIS

3.0. Migration to IPv6 will not be seamless, but there are some features in the DOCSIS 3.0

specification to really enable operators to make the transition much easier provided they are

aware of the features and put them to use. There is no time like the present to adopt the features

and opportunities available in IPv6. Do your homework and make the leap.

For further reading on IPv6:

DOCSIS 3.0 Modems Readily Available

June 25, 2011 Brady 1 Comment

If you have been upgraded to DOCSIS 3.0 and

were issued a cable modem from your cable provider, you are probably paying a monthly fee for

that device. I have always been a strong advocate of owning my own cable modems because it

likely pay for itself over the two or three years you own it. Further, you will likely be able sell it

for a profit as I just did with my DOCSIS 2.0 modem last week (sold for just under $50).

The Motorola SURFboard® SB6121 DOCSIS 3.0 Cable Modem is available on Amazon.com

for $139.95 with free shipping. This is a latest gen modem from Motorola with the following

capabilities:

General

Plug and play installation

Front panel LEDs indicate status and simplify troubleshooting

Offers innovative, high-bandwidth data and multimedia services to customers

Backward-compatible to DOCSIS/EuroDOCSIS 1.x and 2.0

Earth-friendly

Advanced Services Ready

DOCSIS 3.0-certified (EuroDOCSIS 3.0-based)

Channel bonding of up to 4 downstream and 4 upstream channels

1 GHz capable tuners

Supports IPv4 and IPv6 to expand network addressing capabilities

Compatible with Windows, Macintosh, and UNIX computers.

GigE (RJ-45) data port enable flexible, high-speed connectivity with Auto Negotiate and

Auto MDIX

Reliable and Secure

Enhanced security: supports Advanced Encryption Services (AES) traffic encryption

Remotely configurable and monitorable using SNMP and TFTP

This is a great modem. Its likely going to be better than the budget modem provided by your

operator (Hitron) and it will put you in control of your premise equipment. You will need to

provide a new MAC address to you operator. It is also always good to make certain that your

cable provider will allow you to own your cable modem, rarely will they actually not allow this –

you just have to be diligent about with the discussion. This modem will also not support VoIP

with the cable provider, however I have long since switched to other non-cable VoIP

providers. More to come on this topic later.

Top 10 DOCSIS 3.0 Terms You Need to Know

September 05, 2011 Brady No comments

This is the speak you need to know when talking DOCSIS 3.0 to any DOCSIS Engineer or

specialist. It is important that you learn the full name, in many cases the acronym and also what

value the particular terminology plays in a DOCSIS 3.0 network as it will likely be crucial in

troubleshooting tough-to-diagnose DOCSIS impairments.

1. Primary Downstream Channel

A Primary Downstream Channel carries SYNC messages, MDD messages, UCD and MAP

messages which are required by each cable modem for registration on the DOCSIS network as

well as obtaining the necessary information to be able to transmit data on the upstream

channel(s) of the DOCSIS network. A Primary Downstream Channel is almost always

associated with the line card of a CMTS itself, but a Primary Downstream Channels can also be

configured on edge-QAMs (eQAMs) for M-CMTS architectures.

2. Non-Primary Capable Channel

A Non-Primary Capable Downstream Channel is a DOCSIS downstream channel that is not

defined as “Primary”. The non-primary capable downstream channel is primarily used for

transporting user data. Since it does not carry the MAC overhead of SYNC, UCD, MAP, etc. it

is more efficient by nearly 25% in transporting user data. Non-primary capable channels do

occasionally transmit MDD messages (see next #3 below). Note that non-primary capable

channels are often referred to as “secondary” channels.

3. MAC Domain Descriptor (MDD)

MDD messages are periodically transmitted on all Downstream Channel(s)

of DOCSIS 3.0 CMTSs. The MDD message contains the Downstream Channel ID of the

Primary Downstream Channel for the CMTS sending the MDD message. The value of the MDD

occurs when a cable modem locks onto a non-primary capable downstream channel, the cable

modem will read the MDD and know immediately where to tune to in order to lock on to the

Primary Downstream Channel. The MDD can also be used for load balancing cable modems in

a DOCSIS 3.0 network. The MDD also contains additional information for the cable modem,

such as available downstream channels, available upstream channels, upstream frequency range

and more.

4. Downstream Bonding Group (DBG)

The term “Downstream Bonding Group” is intended to refer to a set of two or more downstream

channels that are available to a cable modem. DBGs may either be statically provisioned by an

operator or dynamically determined by the CMTS for load balancing

purposes. The adjacency restrictions on DBGs are limited to within a 60 MHz bandwidth per

the DOCSIS specification. A DBG can span across more than one MAC domain, or CMTS line

card.

5. MAC Domain Downstream Service Group (MD-DS-SG)

The term “MAC Domain Downstream Service Group” (MD-DS-SG) refers to the set of

downstream channels from the same MAC Domain (or just on port on a CMTS line card) that

reaches a fiber node. In most cases, an operator will configure all downstream channels reaching

a fiber node to the same MAC Domain. Often times, MD-DS-SGs are configured such that they

are shared by many cable modems that have different upstream channels, or

more specifically, MAC Domain Upstream Service Groups (see next section). This is similar

to DOCSIS 1.x and 2.0 where a single downstream would service multiple upstreams.

6. MAC Domain Upstream Service Group (MD-US-SG)

The term “MAC Domain Upstream Service Group” (MD-US-SG) refers to the set of upstream

channels from the same MAC Domain that is reached by a single CM. The MD-US-SG set of

channels is available to many cable modems, but the downstream channels to those cable

modems may be different (this is the axiom of section 5 above). Further, even though a MD-US-

SG may be available to a particular cable modem, it does not mean that the cable modem will be

able to use each of the channels in the SG (service group). Impairments in the upstream may

prevent the CMTS from receiving all signals on all channels transmitted by the CM

available. This is not a problem for DOCSIS 3.0. The protocol is resilient enough so that the

modem will stay online with as many upstream channels as possible. Only throughput will

suffer.

7. Upstream Bonding Group (UBG)

Upstream Bonding Group or UBG is the total set of available RF transmit channels available to a

cable modem. The available number of transmit channels may be greater than those allocated by

the MD-US-SG because other modems may be using some channels. A UBG may span multiple

more than one MAC domain, or line cards on a CMTS. UBGs also will consist of physical RF

channels and logical channels. There can only be one channel associated with the center

frequency of an RF channel. However a physical channel can support multiple logical

channels. Logical channels can be used for A-TDMA modes of operation in addition to S-

CDMA mixed with non-S-CDMA modes of operation.

8. Early Authentication and Encryption (EAE)

EAE is enabled via a configuration in the CMTS and is then communicated to the CM in the

MDD message. If the CM receives an MDD message with EAE enabled, the CM will initiate

EAE during the registration process just after ranging and just before DHCP. EAE helps prevent

unauthorized CMs from accessing IP provisioning servers and provides confidentiality/privacy

for IP provisioning messages between the CM and CMTS. If there is a problem with the BPI+

certificates in a cable modem or the x.509 certificates or configuration in the CMTS, EAE could

be a root cause. However, enabling EAE greatly improves the overall security of

a DOCSIS network, thus reducing the probability of theft of service.

9. Alternate Provisioning Mode (APM)

The MDD message tells the cable modem what type of IP address (DHCP) provisioning it will

go through. The MDD has four options; IPv4 only, IPv6 only, APM and DPM. Alternative

Provisioning Mode (APM) tells the CM to try and provision using IPv6 first. If IPv6

provisioning is unsuccessful, either because IPv6 Address acquisition or the TFTP configuration

file download fails, the CM abandons IPv6 provisioning and attempts provisioning using

IPv4. APM is preferred over “IPv6 only” as it provides a fall-back mechanism to IPv4 and the

modem will ultimately come online, though it may take some additional time.

10. Dual-stack Provisioning Mode (DPM)

Dual-stack Provisioning Mode (DPM) is another configurable by the CMTS engineer and then

communicated by the MDD message to the CM. In this case, the CM attempts to acquire both

IPv6 and IPv4 addresses through DHCPv6 and DHCPv4 almost simultaneously. For the

acquisition of time-of-day and the download of a configuration file the CM prioritizes the use of

the IPv6 address over the IPv4 address. If the CM cannot obtain an IPv6 address, or if it cannot

download a configuration file using IPv6, it tries downloading it using IPv4. In this mode, the

CM can acquire IPv4 and the IPv6 addresses, if successfully acquired. The benefit of DPM over

APM is that the modem will acquire an IPv6 or IPv4 or both addresses in a faster time period

since it is dual-provisioning. This is especially crucial during a system-wide outage when

thousands of modems are coming back online. The downside of DPM is that if cable modems

are able to obtain both IPv6 and IPv4 address, there will be significant overhead of used

addresses and duplicate TFTP downloads. This will also have repercussions during a system

wide outage if the provisioning servers are unable to keep up with the extra utilization by the

CMs.

A top 10 list of DOCSIS 3.0 terminology is just a start, but this was provided in advance of my

next post where I will detail DOCSIS 3.0 cable modem registration. This differs somewhat from

legacy cable modem registration as described my DOCSIS 101 modem registration

blog here. But it was important to lay down some ground work first.

Top rated DOCSIS 3.0 cable modem – supported by most cable operators – check with your

local cable operator first.

Why are my DOCSIS 3.0 cable modems transmitting at 52.2

dBmV?

September 09, 2011 Brady 2 Comments

A recent question from a reader provided an interesting response – the answer: maximum

transmit power is 52.2 dBmV for DOCSIS 3.0 cable modems under certain conditions.

This will seem surprising if you have spent the past ten years working in a DOCSIS 1.x or 2.0

nerwork. Even if you have been working in a DOCSIS 3.0 environment with an unbonded

upstream for sometime, you would still be seeing maximum upstream transmit powers well in

excess of +58 dBmV. For DOCSIS 3.0 cablem modems, they are quite happy blasting out a hot

58.2 dBmV in 16-QAM mode.

Channel Bonding Changes Everything

Without getting into all of the details of how the CMTS makes the power calculations, the

CMTS must maintain a relatively constant total output power from a cable modem whether it is

transmitting with one channel or four channels. The transmit power will also vary depending

upon the cable modem’s modulation between QPSK up to 64-QAM. So there are multiple

combinations, but the DOCSIS 3.0 RFI specifications has a few tables to simplify these. Here a

couple of those as examples. First the single channel table:

DOCSIS 3.0 Cable Modem 1 Channel Transmit Power Levels

So this looks fairly similar to a typical legacy DOCSIS cable modem. Although notice the

QPSK Pmax transmit power which is as high as 61 dBmV. That’s 3 dB higher than the 58

dBmV of a legacy DOCSIS cable modem. This shouldn’t normally be a problem unless your

network is configured for QPSK station maintenance and you have a cable modem that is at

maximum transmit for its data burst descriptors, which are set to 16-QAM. The 16-QAM will

transmit at 58 dBmV and QPSK will transmit at 61 dBmV, which is a potential environment for

return path laser clipping. But hey, we only have one channel turned on now. Let’s look at the

four channel condition:

DOCSIS 3.0 Cable Modem 4 Channel Transmit Power

Now look at the significant decrease in power from the first table to the second table. For a one

channel QPSK to a four channel 64-QAM, the delta is as much as 10 dB. This could represent

two DOCSIS 3.0 cable modems operating on the same upstream channel, one running in

DOCSIS 2.0 mode with QPSK station maintenance and one with four bonded upstreams at 64-

QAM.

Now, back to the original title of my post and the question that was posed to me, why did a

reader see so many DOCSIS 3.0 cable modems transmitting at 52.2 dBmV? The answer is very

simple. They were transmitting at their maximum transmit level. That being in a 16-QAM

modulation with four bonded upstream channels. Since 52.2 dBmV is such an uncommon level

for most people to see it is understandable that this level would not come to mind as being a

maximum transmit level. But with DOCSIS 3.0 we all have to learn that the old rules do not

hold true and learn the new rules.

BTR | Rollouts Continue as DOCSIS 3.0 Matures

September 28, 2011 Brady No comments

The following article just released by Broadband Technology Report was written by Carl Weinschenk. A

number of us from the industry contributed to Carl’s piece, but it is a nice high-level summary article on

the status of DOCSIS 3.0. Below you will find a snippet of the article with a link to the BTR website for the

complete read. Enjoy.

Rollouts Continue as DOCSIS 3.0 Matures

There are just two areas of concern as the rollout of DOCSIS 3.0 continues: the upstream and the

downstream.

Observers seem satisfied that the cable industry is doing the best job possible of upgrading its

legacy platform. Planners recognize that they must be proactive if they hope to create a level the

playing field between their coaxial cable and the telcos’ fiber, so it is not a transition that is

happening without keen oversight.

This is not lost on cable industry thinkers, of course. Observers say planners are hard at work on

the next steps, which focus on engineering a gradual and non-disruptive switchover to IP

networking and bonding downstream and — in a particularly promising innovation — upstream

channels.

In the macro-picture, the long-term planning for DOCSIS 3.0 focuses on growing its support for

IP-based video. Just as the Internet has evolved in the past couple of years to be a conduit for

video, cable operators are working to support multiscreen and over-the-top (OTT) on their

broadband networks. It’s not really a matter of choice — it’s what their subscribers are telling

them they must do. <Read the complete article at BTR>

Posted on September 28, 2011 by Carl Weinschenk