BFD ON JUNOS! BIDIRECTIONAL FORWARDING DETECTION – JUNIPER-CONFIG, MULTI-VENDOR EXPLANATION!

In some ways, humans and routers are exactly alike. After all, we both need “keepalives” to keep going.

Having said that, if you were being super-picky and precise about it, I suppose you could say that humans and routers are also different, in that human keepalives include air, water and food, whereas router keepalives involve sending very small electrical signals to each other over some kind of wired or wireless media.

Actually, now I think about it, I see that humans and routers are completely different, so please forget absolutely everything I’ve said so far. Instead, read this:

 

WHAT IS BFD?

Many networking protocols are slow to detect failure. For example, IS-IS often has a default hold-time of 27 seconds. OSPF has a dead interval of 40 seconds. Now, these default times can be altered to be quicker. And depending on the kind of network it’s connected to, there may be other default timers that are even quicker.

Unfortunately though, it’s often the case that the smallest Hello time configurable on a protocol is 1 second, which means the hold time needs to be a minimum of 3 seconds. And if you’re running a lot of protocols on the box, that means you’re adjusting lots of protocols to each send more hello packets. That might not sound like a lot of strain – but scale it out to a lot of protocols and a lot of interfaces, and it can be a genuine problem.

If only there was a way to send sub-second hellos – and have one single stream of hellos that can be shared by all protocols. IF ONLY. IF ONLY THIS WAS POSSIBLE.

Well, guess what: it is! Please say hello to BFD, or Bidirectional Forwarding Detection.

This protocol has the ability to send multiple “hello” messages a second. This single BFD session can then be used by almost any other protocol running on the box. If the BFD session goes down, all the protocols using the session can also tear themselves down immediately. On more powerful boxes, you can even configure sub-second failure detection!

There are two reasons that BFD is able to act so quickly. The first is that it has a very low overhead. A BFD message is basically just a very small UDP message sent in one direction, from Router A to Router B. As long as both boxes are sending BFD messages to each other, the protocols relying on BFD will stay up.

The other reason is that a single BFD session can be used by multiple protocols. If a box is running IS-IS, OSPF, BGP, LDP, RSVP and more, all of those protocols could share the one single BFD session. The protocols still send their hello messages, but those protocol hello message can be left to their default slow timers. Instead, all the requirements for quicker messages can be bundled up into the one single session. This is far more efficient than tuning a dozen protocols on a box to all suddenly send their hello messages once a second.

By the way, this blog post uses Juniper configuration and terminology, and we’re going to look at Juniper output. But the underlying protocol is an open standard, and the actual behaviour of BFD is the same regardless of who you’re using. So if you’re not familiar with Juniper but you still want to know how BFD works, don’t worry: this post has you covered.

 

HOW DOES BFD WORK?

BFD is very easy to configure. There are just two essential pieces to know about:

• The “minimum-interval“, which specifies the smallest amount of time a box is able to send and receive BFD packets.
• The “multiplier“, which is how long the router wants its neighbour to wait before the neighbour assumes the BFD session is down. Again, these times don’t have to match.

Neither of these numbers actually have to match at both ends. For example, if Router A is set to a minimum-interval of 300ms, and Router B is set to 500ms, then both boxes will agree to send BFD to each other using the longer, slower timer. This makes sense when you think that Router B has probably been deliberately set to a slower time because it can’t handle 300ms. Router A can send faster, but it sees that Router B can’t, so Router A slows itself down.

Similarly, if Router A is configured with a multiplier of 3, and Router B is configured with a multiplier of 4, then that’s totally fine. However, this piece is a bit more complicated, so let’s look at an example.

If the minimum-interval is 300ms on both boxes, and the multiplier configured on Router A is 3, then Router B will wait 900ms (300 x 3) before it declares an adjacency down. If the multiplier configured on Router B is 4, then Router A will wait 1200ms (300 x 4) before it tears things down.

In other words, by configuring a multiplier on Router A, what we’re actually doing is getting Router A to indicate to Router B how powerful Router A is. Router A is powerful enough to be able to confidently send BFD regularly, and it lets Router B know this via its multiplier setting. By configuring the multiplier on a box, you’re giving that box the ability to tell its neighbours “this is the multiplier I’d like you to use, because it’s what I’m able to support.”

You could think of it as if Router A is saying something like this: “I am Powerful and Strong and Cool enough to be able to reliably send BFD packets. If you miss three packets from me, you know something has gone wrong. Therefore, Router B, please set your multiplier to 3.”

By contrast, think of Router B as saying to Router A “I am also cool, but I’m not as strong and powerful. There’s a chance that my routing engine might become overloaded for a brief time. So do me a favour, Router A, and set your multiplier to 4. This gives me a bit of freedom to not send a few BFD packets if I ever become overwhelmed for a short while.”

 

WHAT DOES BFD LOOK LIKE IN PRACTICE?

Let’s say we added these two lines of configuration to two boxes, Router A and Router B. These two boxes were already running IS-IS, and the adjacency was already up. I then added this on Router A, to turn BFD on for a particular interface:

set protocols isis interface ge-0/0/0.0 family inet bfd-liveness-detection minimum-interval 400
set protocols isis interface ge-0/0/0.0 family inet bfd-liveness-detection multiplier 3

(NOTE: On some older Juniper boxes you won’t see the “family inet” piece in this configuration.)

At this point, Router B is not talking BFD. However, even though Router A is not receiving any BFD from Router B, the IS-IS adjacency stays up!

Why does this happen? The reason is that BFD is smart: it waits until it receives a reply before it decides that the BFD session is actually up. And the BFD isn’t considered “down” until it has first come up.

Once the configuration is committed on both boxes, we see this:

root@Router_A> show bfd session
                                                  Detect   Transmit
Address                  State     Interface      Time     Interval  Multiplier
10.1.2.1                 Up        ge-0/0/0.0     1.200     0.400        3 

1 sessions, 1 clients
Cumulative transmit rate 2.5 pps, cumulative receive rate 2.5 pps

Notice that you can see the interval, the multiplier, and the total time the session will wait for no replies from the neighbor (3 x 400ms = 1.2 seconds).

You can also see “1 clients”. If we were to add BFD to another protocol (for example OSPF) on ge-0/0/0.0, which talks to the router at 10.1.2.1, we’d see this change to “2 clients”. Try it in a lab of your own, and see what happens!

 

WHAT DO BFD PACKETS LOOK LIKE?

Like this:

There’s a lot of nonsense in the packet, but here’s the bits to look out for:

• This is a regular UDP packet, on port 3784.
• The source and destination IPs are clear to see in the IP header: they’re the IPs of the physical interfaces, not loopback IPs or anything like that.
• In the actual payload (labelled in this picture as the “BFD Control message”) you can see that the session state is “Up”, meaning that this router can see BFD packets coming in from the other router too.
• We see a multiplier of 3, and both TX and RX (transmit and receive) timers of 400ms. It’s possible to set the transmit and receive timers to be different, but you generally would want to keep them the same. The “minimum-interval” command above sets both receive and transmit at the same time.

It’s fairly simple when you only look at the bits that matter!

 

CHANGING BFD TIMERS DOES NOT TEAR ANY SESSIONS DOWN

Updating BFD timers on either of these two routers will have no impact on the service.

In fact, even if the change is done on one box at a time, with a result that each box has different timers for a while during the process of the change, then BFD will still carry on with no problems!

To understand why this is the case, imagine that those two boxes from earlier, Router A and Router B, are still talking BFD to each other, with 400ms timers and a multiplier of 3. Then imagine that we reconfigure Router A to have a 700ms timer, and a multiplier of 4. In other words, Router A and Router B now have different timers and multipliers.

We’ll come back to the multiplier in a moment. For now, let’s focus on what happens to the timers. Here’s the step-by-step process of the negotiation:

• Router A starts sending new BFD messages to Router B, which include the new updated timer and multiplier. However, for the time being, Router A keeps sending at the old, faster speed.
• Router B sees that the timer from Router A (700ms) is larger than the one in Router B’s configuration (400ms). In other words, Router A is asking for slower times.
• Router B replies with the same BFD timers that it was sending before, because nothing has changed in Router B’s config. However, Router B also sets a specific bit in its header to show that it’s received the requested change.
• At this point, Router A can now safely start sending at the slower speed, safe in the knowledge that Router B won’t take these slower message as an indication of a problem.
• All the time, Router B still keeps on telling Router A that it’s willing to do 400ms. In other words, both ends tell the other what they’re capable of, and they then negotiate to the slowest speed.

This all happens gracefully, and no adjacencies are torn down. Aww, that’s cute! They’re working together like best friends!

Router B also sees that the multiplier it’s receiving from Router A has changed. Remember, when we configure it on Router A, this actually changes the multiplier that Router B uses. So, Router B changes the multiplier it uses – but once again, it makes the change gracefully, and nothing goes down.

 

WHAT DOES IT LOOK LIKE WHEN BFD SETTINGS ARE DIFFERENT ON EACH BOX?

Let’s actually now configure Router A to have a minimum-interval of 700, and a multiplier of 4.

root@Router_A> show configuration protocols isis interface ge-0/0/0.0 | display set
set protocols isis interface ge-0/0/0.0 family inet bfd-liveness-detection minimum-interval 700
set protocols isis interface ge-0/0/0.0 family inet bfd-liveness-detection multiplier 4

Then let’s keep Router B with its existing config: a minimum-interval of 400, and a multiplier of 3.

root@Router_B> show configuration protocols isis interface ge-0/0/0.0 | display set
set protocols isis interface ge-0/0/0.0 family inet bfd-liveness-detection minimum-interval 400
set protocols isis interface ge-0/0/0.0 family inet bfd-liveness-detection multiplier 3

What would we expect the result of this to be?

On Router A below we see that the hold time is 2100ms – in other words, 700ms x 3! That’s because it’s Router B that actually want Router A to use a multiplier of 3. However, notice that Router A lists a multiplier of 4 here. This is where it really gets confusing: Router A isn’t showing us the multiplier it’s using – it’s showing us the multiplier it’s sending.

root@Router_A> show bfd session
                                                  Detect   Transmit
Address                  State     Interface      Time     Interval  Multiplier
10.1.2.2                 Up        ge-0/0/0.0     2.100     0.700        4

1 sessions, 1 clients
Cumulative transmit rate 1.4 pps, cumulative receive rate 1.4 pps

I think a very valid question to ask the boffins that configure vendor CLIs is “Why couldn’t you have called that the Sending Multiplier, instead of just the Multiplier”. Oh well, too late now!

Similarly, on Router B we see that it has also negotiated to use the 700ms time, even though it’s configured for 400ms. As for its detect time, it’s 2800ms – in other words, it’s taking the negotiated timer, and combining it with the multiplier that Router A advertised, which is four. Below you can see that Router B actually show a multiplier of 3 – which again, is what Router B is sending to Router A. Good grief!

root@Router_B> show bfd session
                                                  Detect   Transmit
Address                  State     Interface      Time     Interval  Multiplier
10.1.2.1                 Up        ge-0/0/0.0     2.800     0.700        3

1 sessions, 1 clients
Cumulative transmit rate 1.4 pps, cumulative receive rate 1.4 pps

It gets a little confusing, so don’t worry if it takes you a while to get your head around it. The main thing to understand is that you can change BFD timers and multipliers to make them bigger, without it impacting service: all changes happen gracefully, and no neighborships or adjacencies or LSPs are torn down.

 

SHOULD I USE BFD IN MY NETWORK?

Now that’s the million-dollar question. The short answer is “yes”. The long answer is “yes, but there’s some caveats”.

There’s two key questions you have to ask yourself:

• First, is your hardware powerful (but probably expensive), or not-so-powerful (but probably affordable)?
• Second, how many interfaces will you be running it on?

To give examples of Juniper hardware: their MX series of routers is a truly mighty beast, and can comfortably handle BFD with pretty aggressive timers, on lots of interfaces. By contrast, their ACX series is designed specifically to be run fairly light, and in certain circumstances it can struggle if the BFD timers are anything less than 500ms.

If you’ve got 40 interfaces that are all running OSPF, putting BFD on all of them might be okay, or it might be disastrous. It all depends on the hardware you’re using.

A problem with copper Ethernet interfaces is that they can sometimes stay “up”, even when they’re really down. However, some fibre presentations do actually go down the very split-second they see that the light source has disappeared. With that in mind, you might choose to not run BFD on those interfaces, because they’ll go down immediately. This is a good strategy if you’re scared about running BFD on too many interfaces. Be aware though that this only protects you from physical fibre problems. If the routing engine of the box at the other end has crashed, you might not notice it if you’re not running BFD.

 

BONUS BFD FACT

If you ever delete BFD config from one end, but keep the BFD config on the other end, what will happen to the adjacency? You might instinctively think that it will go down. After all, if one end suddenly stops sending BFD messages to the other end… well, that’s bad, right?

But in fact, the adjacency will still stay up! The reason for this is because the router without BFD will send a huge flood of about a dozen “BFD admin-down” messages to the other box, to say that BFD has been deleted by the user. This lets the other router know that the BFD hasn’t disappeared because of a fault, and this allows the IS-IS adjacency to carry on normal. The fact that it’s all UDP means that this message can’t be sent reliably, hence the tidal wave of admin-down messages that will be sent. If at least one of them doesn’t get to the other end, then frankly you’ve got bigger problems on your hands!

 

THAT’S IT!

Did you enjoy this journey down BFD lane? If so then I’d love you to share this post on your favourite social media of choice. The more readers I get, the more I’m inspired to keep on writing even more cool new posts for you.

If you’re on Mastodon, follow me to find out when I make new posts. (2024 edit: I’m also on BlueSky nowadays too. I was once on Twitter, but I’ve given up on it, on account of… well, I don’t need to finish that sentence, do I.)

And if you fancy some more learning, take a look through my other posts. I’ve got plenty of cool new networking knowledge for you on this website, especially covering Juniper tech and service provider goodness.

It’s all free for you, although I’ll never say no to a donation. This website is 100% a non-profit endeavour, in fact it costs me money to run. I don’t mind that one bit, but it would be cool if I could break even on the web hosting, and the licenses I buy to bring you this sweet sweet content.

Thank you for reading!