How To Read A Junos Class-Of-Service Configuration

Quality of Service. Class of Service. What’s the difference?

Some say QoS refers to layer 3, while CoS refers to layer 2. Some say “quality of service” is the thing that you can give to traffic once it has been sorted into different “classes of service”. And some people say “why are you shouting this question through my bedroom window at 3am. How did you find my address. Who are you? What on earth made you think that this was an appropriate method by which to obtain an answer to this question. Please go away. Please. My wife is in tears, and so is my husband.”

Regardless of where you stand, Juniper uses the class-of-service hierarchy to configure almost all of its CoS/QoS/3am settings. The configuration is highly modular, which is great when you’re a service provider that wants to re-use the same pieces of configuration instead of having to type the same thing over and over. Configure it once, use it as many times as you like. Pretty nice for consistency, right?

The challenge with this modularity is that it can be quite difficult to read a configuration when you’re new to all this stuff. In your studies you will probably find that you understand each individual part as you learn it, but then you’ll come to read an actual production configuration, and you don’t know where to start.

In my experience, this is often due to the fact that production configs are never as clean as the nice examples we see in books and KB articles. In fact, I bet money that one of two things happens on your production devices:

– You work at a service provider which has decided to put every possible combination of object onto the box, in advance, “just in case it’s needed”, with the result being that the CoS config is absolutely massive, even though only 30 lines of it is actually used.

– You work in an enterprise where, over the years, loads of people have “had a go” at trying to make things work, and then haven’t deleted the objects they created in error or as experiments. The result, once again, is that the config is absolutely massive, even though only 30 lines of it is actually used.

If that sounds like your experience, then this blog post is for you.

 

WHAT THIS POST IS – AND WHAT IT ISN’T

In this post I’m going to give you a strategy that will help you read a Juniper CoS config.

Here’s the journey you’re going to go on through this post. First of all you want to find out how traffic is being classified as it comes in to an interface. That’s going to involve looking at your forwarding-classes (fancy word for your queues), and the method by which traffic is put into those queues. In addition, perhaps traffic is policed (ie dropped) before it even gets a chance to be queued.

Once the traffic is in those queues, you’re interested in seeing how the traffic is sent out of its outgoing interface. Each queue is serviced in a different way. If you imagine a block of time, then each queue will be scheduled for processing at a certain moment within that block of time. Hey, that’s why it’s called a scheduler! You’re going to see which schedulers deal with which queues (ie which forwarding classes), at which point you’ll then know which details to zoom in on, along with whether the QoS bit-markings in the packet or frame will be re-written as the traffic leaves the box.

You’re going to go around the houses a bit as you scroll up and down your config, but think of this journey as being like stabiliser wheels on a bike. Once you know what you’re looking for and why you’re looking for it, there are definitely quicker ways of reading a config than this. For now, focus on the logic. You will make your own shortcuts in time.

This blog is not a beginner’s guide to Junos Class of Service. If you’re at the stage where you don’t yet know the difference between a BA classifier and a multifield classifier, if you’re at the stage where you’re still getting your head around the difference between policing and shaping, or if you don’t yet know the difference between the “ipprec-compatibility” classifier vs the “dscp-default” classifier, I would recommend coming back to this post another day.

Instead, this post is for folks who are halfway along in their CoS studies. They’ve learned the parts, they’ve even half-remembered some of them, but they wouldn’t be confident reading a CoS configuration in production. For those folks, this post gives you a narrative that you can follow, by helping you to read the configuration in a certain order, and to help you to understand why you’re reading it in that order.

Finally, this post is not a complete guide to CoS config. There are plenty of bits that I’ve missed out. After all, entire books have been written on this subject, and this is just one blog post. But although some bits are missing, this post will definitely help you to learn the basic philosophy of what’s going on.

 

FORWARDING CLASSES AND QUEUES

What is the difference between a forwarding class and a queue?

A “queue” is the literal thing in hardware where you packets are stored on ingress, to be sent out on egress. Packets need to be put into this queue somehow, which is why you use classifiers to decide what traffic goes into what queue.

These queues have numbers to represent them: queue 0, queue 1, and so on. Your aim is to classify traffic, and put it into a queue.

However, it would be a real brain-ache to say “this traffic goes into queue 0, this traffic goes into queue 1”. So instead, you make named forwarding classes, and bind them to a queue. As it happens, there are four forwarding classes configured by default. “best-effort” is bound to queue 0, while “network-control” (traffic like BGP, OSPF, all that good stuff) is mapped to queue 3.

root@vMX1> show class-of-service forwarding-class    
Forwarding class              ID      Queue    Fabric priority  Policing priority
  best-effort                  0       0        low                normal 
  expedited-forwarding         1       1        low                normal
  assured-forwarding           2       2        low                normal
  network-control              3       3        low                normal

(I removed a few columns there so it fits better on the page. If anyone knows of a good WordPress plug-in to keep fixed-width text all on one line without word-wrapping, maybe by bringing in a horizontal scroll-bar, please let me know!)

A lot of Junos platforms offer 8 configurable queues, with 4 queues used by default like in that output. Some offer even more queues than that, but I’ve a terrible memory for such things.

So, the forwarding class is basically the name that you give a queue, right?

Well, yes, except… it is in fact possible on some hardware to bind two forwarding classes to the same queue. For example, some MX platforms offer 8 queues, but 16 forwarding classes.

What the documents don’t tell you is why you would ever want to do something as obviously bonkers as that, or what impact this has. That PDF I linked to says “Several forwarding classes can be assigned to the same queue (in that sense, different forwarding classes act as aliases for a particular queue)”. But why? What advantage does this bring? In my opinion, very little. What it does bring is a recipe for confusion and problems. If you choose to do this, be aware that you’re making things harder for your ops team to troubleshoot.

The only reasonable argument I’ve heard for this feature is in environments where some boxes have 8 queues and some have 16, this feature allows you to have more “consistent” config across all boxes. And that is true, and not unreasonable. But just aware that you haven’t magically doubled the number of hardware queues on your box by doing this.

Personally, I say: get rid of complexity, keep things simple. If you map one forwarding class to one queue, then you can reasonably say that a forwarding class is just a named alias for a queue.

 

HOW TO READ CONFIG: FORWARDING CLASSES

With that out of the way: you’ve got a configuration in front of you. How do you read it?

Let’s start with traffic coming into the router.

1) Start by reading the “set class-of-service forwarding-classes” hierarchy, because ultimately this is where you’ll find the queues that incoming traffic goes into. That lets you know what you’re dealing with at the start. That output above was from a box with no class-of-service config, but on yours you might see something like this:

set class-of-service forwarding-classes queue 0 BE_FORWARDING_CLASS
set class-of-service forwarding-classes queue 1 AF_FORWARDING_CLASS
set class-of-service forwarding-classes queue 2 EF_FORWARDING_CLASS
set class-of-service forwarding-classes queue 3 NETWORK_CONTROL_CLASS

That config basically does the same thing as the default forwarding classes that you saw a moment ago. I’m just changing the names to show you the philosophy, but you might give your forwarding classes names like “FC_VOIP” or “FC_MICROSOFT_TRAFFIC” or something. It’s a good idea to put something like “FC” at the beginning, because then you can easily read a config and spot when it refers to a forwarding-class.

You could alternatively do a “show class-of-service forwarding-class” to see what forwarding classes map to what queues, as you just saw. But this post isn’t about show commands. It’s about reading a config. So, with that in mind: let’s go to Step 2.

 

HOW TO READ CONFIG: CLASSIFYING TRAFFIC

2) Next, you’re going to find out how traffic is being classified – in other words, how Junos decides how to put traffic into whatever queue. There are three ways this can happen, and each method can either stand alone, or be used in combination with the other methods.

2a) It might be that the entire interface goes into one forwarding class. If so, you’ll see that under the “set class-of-service interfaces” hierarchy in the config. All incoming traffic on the interface will map directly to a single forwarding-class, something like this:

set class-of-service interfaces ge-0/0/2 unit 0 forwarding-class EF_FORWARDING_CLASS

Easy! You can go to Step 4 from here.

2b) It might be that a firewall filter is being used on the interface, in order to perform very precise, granular, “multifield” classification – so-called because you can classify traffic based on any field (ie header) in the packet or frame.

If this is happening, you’ll find the firewall filter under the regular “set interfaces ge-0/0/6 unit 0 family inet” stanza in the usual way that a firewall filter is deployed. The firewall filter on the interface will have terms in it like “If it’s from this IP block, put it into X forwarding-class. If it’s from this protocol, put it into Y forwarding-class”. Easy! You can go to Step 4 from here.

2c) The third way is “behavior aggregate”, which is where the box trusts that the markings in the packet itself are enough to decide which queue to put the traffic in. In this case you’ll want to know how the box is classifying traffic. Two outcomes here: either the box is using the default classifiers, or it’s using a custom one that has been added through configuration.

Check under the “set class-of-service interfaces” section in the configuration. Do you see a “classifier” stanza on the interface? Something like this:

set class-of-service interfaces ge-0/0/1 unit 0 classifiers dscp MY_CORE_CLASSIFIER

If you see something like this, you’ll want to take a look at what that custom classifier is doing. In that case, go to Step 3b.

However, if you don’t see anything like that the config, then the box is just using the default classifier on that interface. In that case, you can do a “show class-of-service interfaces” to see what classifiers are being used on your interface by default. If it’s a normal layer 3 interface, you’ll see something like this:

jcluser@vMX-addr-0> show class-of-service interface ge-0/0/0
Physical interface: ge-0/0/0, Index: 149
Maximum usable queues: 8, Queues in use: 4
Exclude aggregate overhead bytes: disabled
Logical interface aggregate statistics: disabled
  Scheduler map: <default>, Index: 2
  Congestion-notification: Disabled

  Logical interface: ge-0/0/0.0, Index: 340
Object                  Name                   Type                    Index
Classifier              ipprec-compatibility   ip                         13

Go to Step 3a where you’ll learn what “ipprec-compatibility” does.

 

CUSTOM CLASSIFIERS

3a) You’re at this step because there was no custom classifier, which means the default classifier is being used. Let’s see how to check what it does.

If you type “show class-of-service classifier”, you’ll be able to see all the different default classifiers for incoming traffic. Some of these classifiers look in the Ethernet 802.1q header, some of them look at the IPv4 TOS header, some of them look at the MPLS EXP bits, and so on.

You just saw that on a layer 3 IPv4 interface, the default classifier is called “ipprec-compatibility”, “ipprec” being short for “IP precedence”. When you use this, basically everything is put into best-effort, apart from stuff tagged with bits that mean network-control. Only the first three bits of the TOS field are used, unlike DSCP which uses six bits. Check it out:

jcluser@vMX-addr-0> show class-of-service classifier name ipprec-compatibility 

Classifier: ipprec-compatibility, Code point type: inet-precedence, Index: 13
  Code point         Forwarding class                    Loss priority
  000                best-effort                         low         
  001                best-effort                         high        
  010                best-effort                         low         
  011                best-effort                         high        
  100                best-effort                         low         
  101                best-effort                         high        
  110                network-control                     low         
  111                network-control                     high 

Cool! You can go to Step 4 now.

3b) You’re at this step because you saw a classifier at the “set class-of-service interface” stanza. Perhaps something like this:

set class-of-service interfaces ge-0/0/1 unit 0 classifiers dscp MY_CORE_CLASSIFIER

In that case, look for the classifier itself in the “set class-of-service classifiers” section, where you might see something like this:

set class-of-service classifiers dscp MY_CORE_CLASSIFIER import default

set class-of-service classifiers dscp MY_CORE_CLASSIFIER forwarding-class BE_FORWARDING_CLASS loss-priority high code-points 000001 

set class-of-service classifiers dscp MY_CORE_CLASSIFIER forwarding-class EF_FORWARDING_CLASS loss-priority high code-points 101111

This config imports the default DSCP classifier (which you can see with the command “show class-of-service classifier type dscp”), and tweaks it just for those two code-points. Effectively the logic says “if the traffic has bit marking 101111, then put it into the EF_FORWARDING_CLASS queue, with a high loss priority”.

“Loss priority” is an odd term, which actually means “loss probability”. In other words: in times of congestion, how likely is this traffic to be dropped? This loss probability comes into play if you’ve got a drop-profile, which you’ll see later.

Hey there: binary is hard to read, right? The code points at the end of the custom classifier can be tricky for your ops team to read. To make your life easier, you can define a “code-point-alias”, which gives a name to a particular code-point. You can then use that name in your custom classifier, like this:

set class-of-service code-point-aliases dscp VOIP 101110

set class-of-service classifiers dscp MY_CORE_CLASSIFIER forwarding-class EF_FORWARDING_CLASS loss-priority low code-points voip

Nice nice nice! Let’s carry on reading.

 

POLICING

4) Once traffic has been classified, there is a chance that it might also be policed. Remember that policers are hard drops, as opposed to shapers which tend to hold traffic back in order to send it when bandwidth becomes available. As a general rule, you police inbound, and shape outbound.

Policers are defined in the “set firewall policer” hierarchy. Most of the time, policers are then referenced in firewall filters. So once again, check the actual interface itself, under the “set interfaces” hierarchy. You’ll see the named policer in the firewall filter config, at which point you can go and see what it does

You could police traffic in this way even if you’re using a different classifying method. For example, you might have a BA classifier under the “set class-of-service interfaces” hierarchy, but there could also be a firewall filter on the actual interface which policies different kinds of traffic inbound.

There is a possibility that a policer has also been applied to the entire interface. You’d do this when you want to do general rate-limiting, rather than the kind of CoS you’re learning about in this post. If there is one, it will be applied directly to the interface, like this:

set interface ge-0/0/4 unit 0 family inet policer input 100M_POLICER

Right! Now that you know what traffic has been put into what forwarding class, it’s time to see how the traffic is taken out of that queue, and sent out of an interface. This is done by something called a scheduler. Time for Step 5!

 

HOW TO READ CONFIG: SCHEDULING

5) A scheduler defines how a queue will be processed – the bandwidth it gets, the chances of traffic in the queue being dropped, stuff like that.

A scheduler is just an object that in itself doesn’t do anything, in the same way that just defining a firewall filter doesn’t actually do anything until you use it. The scheduler “comes to life” when you use a scheduler-map to bind a scheduler to a forwarding class. In other words, the scheduler-map says “traffic in forwarding class A will be processed with scheduler ABC, traffic in forwarding class B will use scheduler XYZ”. The scheduler-map will list each forwarding-class, and the scheduler assigned to it.

Each interface will have its own scheduler-map, whether it be just the default one, or a custom one.

Go to “set class-of-service interfaces ge-0/0/3” (I’m just making these interfaces up as I go along, lol) and look for the scheduler-map. Do you see one? Maybe something like this?

set class-of-service interfaces ge-0/0/3 scheduler-map WAN_INTERFACE_SCHEDULER_MAP

If not, then the interface is using the default scheduler map. You can confirm this with a “show class-of-service interface” for the interface. The default is easy: assuming the default forwarding-class names, best-effort gets 95% of traffic, network-control gets 5%, and then the other two queues get a tiny tiny fraction just in case traffic does end up being put into those queues somehow. That won’t happen with the default classifiers, but it might happen if you’ve made your own.

if there’s no scheduler-map configured on the interface, your job is almost done! Go to Step 9 to check one final thing. However, if there is a custom scheduler-map on the interface, go to Step 6.

6) Under “set class-of-service interfaces ge-0/0/3” where you saw your scheduler-map, was it configured under the physical interface or the logical interface? If it’s under the logical, then that means the “per-unit-scheduler” command will have been configured under “set interfaces ge-0/0/3”. Just something to bear in mind, because you’ll want to know whether the scheduler-map is applying to the entire interface, or whether each unit has its own scheduler-map on it.

Go to “set class-of-service scheduler-maps” to see what your scheduler-map is doing. Here’s an example:

set class-of-service scheduler-maps WAN_INTERFACE_SCHEDULER_MAP forwarding-class EF_FORWARDING_CLASS scheduler EF_SCHEDULER

set class-of-service scheduler-maps WAN_INTERFACE_SCHEDULER_MAP forwarding-class BE_FORWARDING_CLASS scheduler BEST_EFFORT_SCHEDULER

set class-of-service scheduler-maps WAN_INTERFACE_SCHEDULER_MAP forwarding-class AF_FORWARDING_CLASS scheduler AF_SCHEDULER

set class-of-service scheduler-maps WAN_INTERFACE_SCHEDULER_MAP forwarding-class NETWORK_CONTROL_CLASS scheduler NETWORK_CONTROL_SCHEDULER

I’ve worked in places where, for historical template reasons, a customer router might have a bajillion schedulers defined, but they’re only actually using like four of them in a scheduler-map. In such a situation, if you were to go straight to the schedulers themselves, you’d have no idea which ones were actually worth your time reading. So instead, look at the scheduler-map first, see which schedulers you care about, and then read the config of just those.

Pick one scheduler at random, and go to Step 7.

7) The “set class-of-service schedulers” stanza is your next port of call. Let’s look at a couple of examples, and see what’s going on:

set class-of-service schedulers EF_SCHEDULER transmit-rate percent 20
set class-of-service schedulers EF_SCHEDULER buffer-size percent 20
set class-of-service schedulers EF_SCHEDULER priority high

Seasoned QoS engineers who don’t actually need to read this post but are reading it anyway for some reason will look at that example and going “What?? 20% buffer to the EF queue?” Yes. 20%. What are you gonna do about it, nerd? Have a cry on the internet?

Some of this config is intuitive just by reading it. If this scheduler is applied to a queue, then that queue gets 20% of the total bandwidth. The transmit rate of 20% is only in times of congestion. By default, the queue can still use more bandwidth than this if the bandwidth is available. You can override this with the word “exact”, or alternatively you could add a shaping rate to the queue, which you do by adding it to the scheduler:

set class-of-service schedulers EF_SCHEDULER shaping-rate 20m

As for the priority, there are five levels: strict-high, high, medium-high, medium-low, and low. If you’re unsure about what the different priority levels mean, or what the difference between high and strict-high is, this doc explains it.

Let’s look at a slightly different scheduler:

set class-of-service schedulers BEST_EFFORT_SCHEDULER transmit-rate percent 30

set class-of-service schedulers BEST_EFFORT_SCHEDULER priority medium-low

set class-of-service schedulers BEST_EFFORT_SCHEDULER drop-profile-map loss-priority high protocol any drop-profile HIGH_DROP_CHANCE

This is where the famous Random Early Detection comes in. This one applies a named “drop-profile” to certain kinds of traffic, in this case traffic from any protocol (as opposed to just tcp or just udp) with a high loss-priority (which remember really means “loss probability”, which was set by the classifier as the packet entered the queue). If you see one of these, go to Step 8. Otherwise, go to Step 9.

8) Drop profiles define what traffic should be dropped, and in what circumstances.

As with a lot of these pieces, they’re just objects which don’t automatically do anything just by making them. Instead, the are “enabled” when they’re used in a scheduler. And thanks to the modular nature of the config, the same drop-profile can be used by many different schedulers on many different interfaces. This is very helpful indeed on service provider PE boxes with thousands of customers. You only really need to configure a few different drop-profiles, which you can then reuse to give all of your customers a consistent QoS offering.

Let’s take a look at the one that was being used in that last scheduler.

set class-of-service drop-profiles HIGH_DROP_CHANCE interpolate fill-level [50 70 90]
set class-of-service drop-profiles HIGH_DROP_CHANCE interpolate drop-probability [10 40 80]

This says “When the queue is 50% full, the traffic has a 10% chance of being dropped. When it’s 70% full, it has a 40% chance of being dropped. And when it’s 90% full, it has an 80% chance of being dropped.” However, there’s a bit more to the story.

“Interpolate” means that if a queue is full at a percentage in between 50% and 70%, the chance of traffic drops rises in relation to each percent’s fill level. So in this example, when the queue is 25% full, the traffic has a 60% chance of being dropped. If you were to draw it out on a graph, the drop-probability would be smooth, real smooth, smooth like Sade.

This is as opposed to the “staircase method” which just goes up in steps.

set class-of-service drop-profile LOW_DROP_CHANCE fill-level 50 drop-probability 5
set class-of-service drop-profile LOW_DROP_CHANCE fill-level 60 drop-probability 10
set class-of-service drop-profile LOW_DROP_CHANCE fill-level 70 drop-probability 15

In that example, traffic at 55% fill level would still only have a 5% chance of being dropped.

You’re almost there! There’s just one final question to ask, which is: are there any rewrite rules as traffic leaves the interface?

 

HOW TO READ CONFIG: REWRITE RULES

9)  By default, traffic leaves the box with whatever TOS/DSCP/etc markings it came in on. It doesn’t matter if it came in tagged with TOS 000 (best effort) but was then sorted into your expedited forwarding queue. That just means that your box is choosing to treat it with high priority in the moment. By default, it still goes out with 000 in the TOS field, or 000000 in the DSCP, or whatever.

In that example, if you want every other hop to also treat packet traffic as EF too, you need a rewrite rule. And as ever, you’ll find it under “set class-of-service interfaces”. if there’s a rewrite rule, it will look like this:

set class-of-service interfaces ge-0/0/0 unit 0 rewrite-rules dscp REWRITE_DSCP_OUTBOUND

You then make a named rewrite rule, which will probably be quite long, but here’s three example lines of config that rewrites traffic in the EF and BE queues as the packets leave the router:

set class-of-service rewrite-rules dscp REWRITE_DSCP_OUTBOUND forwarding-class EF_FORWARDING_CLASS loss-priority high code-point ef

set class-of-service rewrite-rules dscp REWRITE_DSCP_OUTBOUND forwarding-class BE_FORWARDING_CLASS loss-priority low code-point be

set class-of-service rewrite-rules dscp REWRITE_DSCP_OUTBOUND forwarding-class BE_FORWARDING_CLASS loss-priority high code-point cs1

Not sure what bit markings “ef”, “be”, and “cs1” represent? Give this a read.

 

NOW YOU KNOW THE JOURNEY, HERE’S A SHORTCUT

As you read that config, you skipped around the place a bit. But you probably noticed that a lot of the key information was in “set class-of-service interfaces”. As you get more confident, you can go straight there and use it as a spring-board to other things. Remember to also check under the actual interface itself for any firewall filters and policers!

Alternatively, you could do something like “show configuration | display set | match xe-0/0/0.1234” (for example) and make that your starting point. Once you’ve done that, hopefully that journey you just went on will empower you to know where to start looking first.

Combine that with a “show class-of-service interface xe-0/0/0.1234” and you’ll be good to go. That command will show you the classifier, scheduler-map, and rewrite rules on that particular interface.

 

THAT’S IT!

To round things off, here’s a way to improve things in your organisation.

Because Junos CoS config is modular, there is a strong chance that there will be a ton of irrelevant and unused config on your production boxes, which just makes the config longer. If that sounds like your workplace, here’s something you can suggest. Instead of putting all config on all boxes “just in case it’s needed”, a cleaner idea would be to keep all those pieces on some kind of central deployment platform, that can add them just when they’re needed. This will certainly help your operations team to troubleshoot things. A configuration that is a third of the size and just as functional can only be a good thing in the heat of the moment!

Did you enjoy this class-of-service journey? Hopefully you’ve come out of the other side with a clearer idea of how to find what you want. Grab a few configs from boxes in your workplace, and try following along. Did it make sense? Let me know in the comments! In the mean time, if you enjoyed this post, please share it with your colleagues or on your favourite social media of choice, or maybe email it directly to colleagues and pals who you think will enjoy it?

If you fancy some more learning, take a look through my other posts. I’ve got plenty of cool new networking knowledge for you on this website, especially covering Juniper tech and service provider goodness.

It’s all free for you, although I’ll never say no to a donation. This website is 100% a non-profit endeavour, in fact it costs me money to run. I don’t mind that one bit, but it would be cool if I could break even on the web hosting, and the licenses I buy to bring you this sweet sweet content.

If you’re on Mastodon, follow me to find out when I make new posts. (2024 edit: I’m also on BlueSky nowadays too. I was once on Twitter, but I’ve given up on it, on account of… well, I don’t need to finish that sentence, do I.)

Be sure to unfollow everyone else, so that you can be guaranteed to spot my glorious posts, which are of course the most important posts on that web site, as compared to everyone else’s posts which are hot garbage.

Thank you for reading!