MPLS VPNs AND JUNOS ROUTING POLICY: LESSONS IN TROUBLESHOOTING JUNIPER ROUTERS

Hey there: you smell good! That’s probably because you’ve just read Part 1 of this two-part blog post, where we learned all about BGP communities, and how route-targets are used in MPLS VPNs. You also smell so good that I think you know the basics of setting up Junos routing policy.

Well, now you know the theory, let’s look at a problem ticket that came in to us recently. Put on your learning hat! Then, take it off, make yourself a cup of coffee, put your hat on, take it off again, put it on again, take it off again, and burn it. It never suited you anyway.

NOW WE KNOW THE THEORY, LET’S LOOK AT AN INTERESTING PROBLEM!

A ticket came in the other day. We had an existing MPLS customer – let’s call them Barry’s Ice Cream (a company founded by their CEO, Barry Ice-Cream) who had taken out a UCaaS solution with us. It stands for Unified Communications as a Service – basically a cloud-based phone system. All our customers with UCaaS also have MPLS networks.

If this were a brand new customer, we’d create the MPLS VPN from scratch, via a template with all the relevant config, including import and export targets for the customer’s VRF, the management IPs, and the UCaaS solution IPs.

So, our engineer logs onto the PE routers facing the UCaaS platform, and sees that this is actually an existing MPLS customer. Aha, they think: in that case, I can just add in the lines of config that are missing, which import and export the UCaaS prefixes. Job done!

By the end, the VRF config looked something like the config below. I say “something like” because of course I’ve removed any similarities to my company’s internal communities and naming conventions. However, don’t mistake me for someone who is security conscious: I will gladly tell you every single password to every single system of my employer, in exchange for three bags of Maltesers.

chris.parker@PE_Router_1> show configuration routing-instances BARRYS-ICE-CREAM-VRF
    interface ge-1/0/1.0;
    instance-type vrf;
    route-distinguisher 10.58.255.1:37
    vrf-import [ MPLS-MANAGEMENT-IMPORT BARRYS-ICE-CREAM-IMPORT SHARED-UCAAS-IMPORT ];
    vrf-export [ MPLS-MANAGEMENT-EXPORT BARRYS-ICE-CREAM-EXPORT SHARED-UCAAS-EXPORT ];
    [edited for brevity]

Here’s the sad twist in the story, though: even though our plucky engineer added in the config, they were still unable to access their UCaaS solution from the usual management problem.

You know what that means? IT’S TROUBLESHOOTING TIME, BABY!!!!!!!!!!!!!

THE SOLUTION: JUNOS ROUTING POLICY!

So, our engineer told me they’d looked at the prefixes being learned on the PE router hosting the management platform, and couldn’t see our new customer’s range in there. That saves time pinging around the place: the route isn’t even being learned in the first place. In that case, is the prefix being advertised at all by the UCaaS PE router? (That question is what we in the dramatic arts call “foreshadowing”).

Let’s look at our config again, because we’re going to learn a little bit about Junos routing policy.

Look at the config above again. Notice that the vrf-import and vrf-export config lines each contain three policies, which are each analysed one after the other. Let’s look at the config for the three export policies. Notice that the first policy has no “from” statement, which in Junos routing policy means “match anything”.

funtimes@PE_Router_1> show configuration policy-options policy-statement BARRYS-ICE-CREAM-EXPORT
term default {
     then {
          community add target-BARRYS-ICE-CREAM;
          accept;
     }
}

funtimes@PE_Router_1> show configuration policy-options policy-statement MPLS-MANAGEMENT-EXPORT
term cpe-tail {
     from {
          prefix-list-filter MPLS-MANAGEMENT-PREFIXES orlonger;
     }
     then {
          community add target-MPLS-MANAGEMENT;
     }
}

funtimes@PE_Router_1> show configuration policy-options policy-statement SHARED-UCAAS-EXPORT
term UCAAS {
     from {
          prefix-list-filter UCAAS-PREFIXES orlonger;
     }
     then {
          community add SHARED-UCAAS;
     }
}

Looks simple, right? We can see that each statement either matches everything, or matches a list of IPs defined in a prefix-list somewhere. There’s no “deny” statements here.

Or… is there? Take a look at the config again. Can you spot any differences between these three policies?

Did you see it? The first one has an action of “accept”, whereas the other ones don’t. In Junos, if you don’t have an explicit action in a policy like this, then the default action is to just look at the next policy in the list. So in fact, in those other two policies, after adding on the communities, there’s an implied action of accept.

But here’s the catch: if you explicitly say “then accept”, what you’re actually saying is “accept, and stop looking at further policies. “Accept” is what’s known as a terminating action. Leave it out, and the router just carries on looking at the next policy. Add it in, and the policy lookup comes to a halt.

The policy BARRYS-ICE-CREAM-EXPORT has a terminating action in it. Unfortunately, our engineer put that policy in the middle of the chain, and that policy happens to match everything. So, our third policy, SHARED-UCAAS-EXPORT, never even gets looked at. D’oh!!

Luckily, the solution is simple: change the order to be [ MPLS-MANAGEMENT-EXPORT SHARED-UCAAS-EXPORT BARRYS-ICE-CREAM-EXPORT ]. If we make this change, here’s what happens: MPLS-MANAGEMENT-EXPORT is analysed. A community may or may not be added, but either way the policy has no terminating action, so regardless of whether or not the prefix matches the prefix list, the policy passes the prefix to the next policy, SHARED-UCAAS-EXPORT. The same thing happens again: regardless of whether or not it matches, the prefix is finally passed to BARRYS-ICE-CREAM-EXPORT, which actually has a terminating action.

So, in this case, the fix was really easy! All we have to do is just move SHARED-UCAAS-EXPORT to be somewhere before BARRYS-ICE-CREAM-EXPORT. And then, suddenly, we find not only that our customer’s Layer 3 MPLS VPN works perfectly, but that we can manage their devices from our management platform, too. Don’t you love a happy ending?

THAT’S IT!

Understanding Junos routing policy is a key skill. There’s a lot of subtleties to it, and understanding it can give you incredible power to do some very complicated things. I hope this blog has shown you a simple but effective way that you can do something very cool – and how to spot when it goes wrong!

If you’re on Mastodon, follow me to find out when I make new posts, plus you’ll be in an exclusive club of people who can proudly say that they followed me before IPv4 was shut down for good. (2024 edit: I’m also on BlueSky nowadays too. I was once on Twitter, but I’ve given up on it, on account of… well, I don’t need to finish that sentence, do I.)

And of course, if you really enjoyed this post then I’d love you to share it on your favourite social media of choice. Or, why not print it and send it to Santa? Sending him such fantastic content will surely only increase your chances of being added to his good list in 2019.

If you fancy some more learning, take a look through my other posts. I’ve got plenty of cool new networking knowledge for you on this website, especially covering Juniper tech and service provider goodness.

It’s all free for you, although I’ll never say no to a donation. This website is 100% a non-profit endeavour, in fact it costs me money to run. I don’t mind that one bit, but it would be cool if I could break even on the web hosting, and the licenses I buy to bring you this sweet sweet content.