Introduction
Recently I carried out tests in labs to evaluate the FlowSpec implementation on MX960 router with TRIO MPC cards. I used a 12.3 Junos release.
Those tests have covered:
- IPv4 blackholing traffic feature
- IPv4 rate-limiting traffic feature
- IPv4 traffic redirection for traffic mitigation (redirect within a VRF)
All tests have been passed with success with just a limitation on traffic redirection presented below. This post presents, how FlowSpec is implemented at RE and PFE level. It presents also the configuration template for Redirecting Traffic through a specific VRF. The “remarking” feature has not been tested.
FlowSpec’s theory
BGP FlowSpec is defined within the RFC 5575: “Dissemination of Flow Specification Rules”. This RFC describes a new NLRI that allows to convey flow specifications (@IP src ; DST ; proto ; port number …) and traffic Action/Rules associated (rate-limiting, redirect, remark …). For IPv4 purposes, RFC defines the following AFI/SAFI value: AFI=1 SAFI=133. FlowSpec uses two existing BGP “containers” to convey both Flow specifications and rules associated.
Indeed, FLOW specifications are encoded within the MP_REACH_NLRI and MP_UNREACH_NLRI attributes. Rules (Actions associated) are encoded in Extended Community attribute.
Within the MP_REACH_NLRI a flow specification is made of a set of components, each component of the flow is combined with each other with an AND operator. Below we describe the different components we have to describe a specific flow (encoded on the NLRI variable field). The Next-Hop length and Next-hop value should be set to 0 (in case of the RFC 5575). Note is should be set other than 0 in this case: BGP Flow-Spec Extended Community for Traffic Redirect to IP Next Hop (draft-simpson-idr-FlowSpec-redirect-02.txt)
NLRI FlowSpec Validation:
FlowSpec BGP update received should pass Validation Steps before their installation from the Adj-RIB-IN to LOC-RIB. Next-Hop validation must not be taken into account, because NH FlowSpec (RFC 5575) is always set to 0. But other Validations have to be done before installation (extracted from RFC):
a) The originator of the flow specification matches the originator of the best-match unicast route for the destination prefix embedded in the flow specification.
b) There are no more specific unicast routes, when compared with the flow destination prefix, that have been received from a different neighboring AS than the best-match unicast route, which has been determined in step a).
These validations can cause some problems when you use for example external Server to inject FlowSpec updates. Implementations may de-activate these validation steps
FlowSpec components ?
Notice from the RFC: “Flow specification components must follow strict type ordering. A given component type may or may not be present in the specification, but if present, it MUST precede any component of higher numeric type value.”
Type 1: Destination prefix component
Type 2: Source prefix component
Type 3: IP Protocol component
The option byte is defined as following:
- E bit: end of option (Must be set to 1 for the last Option)
- A bit: AND bit, if set the operation between several [option/value] is AND, if unset the operation is a logical OR. Never set for the first Option
- Len: If 0 the following value is encoded in 1 byte ; if 1 the following value is encoded in 2 bytes
- Lt bit: less than comparison between the Data and the given value
- Gt bit: greater than comparison between the Data and the given value
- Eq bit: equal comparison between the Data and the given value
Type 4: Port number component
Type 5: Destination port number component
Type 6: Source port number component
Type 7: ICMP Type component
Type 8: ICMP Code component
Type 9: TCP Flags component
The option byte is defined as following:
- E bit: end of option (Must be set to 1 for the last Option)
- A bit: AND bit, if set the operation between several [option/value] is AND, if unset the operation is a logical OR. Never set for the first Option
- Len: If 0 the following value is encoded in 1 byte ; if 1 the following value is encoded in 2 bytes
- NOT bit: logical negation operation between Data and the given value
- m bit: match operation between the Data and the given value
Type 10: Packet Length component
Type 11: DSCP Value component
Type 12: Fragment component
After the flow definition, Traffic Actions (rules) are encoded as Extended Community Attribute (see RFC 4360)
Extented Community – RFC 4360
There are 4 types of “Action”, each of them has a dedicated Extended Community TYPE. The tab below lists the current Actions available:
Traffic-rate action:
Used for discard or rate-limit a specific flow. Discard action is actually a rate equal to zero. The remaining 4 octets carry the rate (in Bytes/sec) information in IEEE floating point [IEEE.754.1985] format. A nice link to troubleshoot encoded rate can be find here: http://www.h-schmidt.net/FloatConverter/IEEE754.html
Traffic-action action:
Used to trigger specific processing the corresponding flow. Only the last 2 Bits of the 6 bytes are currently defined as following:
- Terminal Action (bit 47): When this bit is set, the traffic filtering engine will apply any subsequent filtering rules (as defined by the ordering procedure). If not set, the evaluation of the traffic filter stops when this rule is applied.
- Sample (bit 46): Enables traffic sampling and logging for this flow specification.
Redirect action:
Traffic redirection allows to specify a “route-target” community which will be handled by the router to redirect a Flow to a specific VRF.
Traffic-marking action:
Used to force a flow to be re-writted with a specific DSCP value when it leaves the routers.
Flow Specification and JUNOS
Junos allows 2 kind of Flow Routes. First of all, static flow routes which are configured like this:
#Example: Rate-limiting flow (@IP source 10.1.1.1/32 DNS traffic) at 15Kbits/s
sponge@bob# show routing-options flow
route static-flow1 {
match {
source 10.1.1.1/32;
protocol udp;
port 53;
}
then rate-limit 15k;
}
BGP flow routes family is configured as following. Associated to the flow family you can add a policy-statement to disable the validation process of the route (allows direct installation of FlowSpec routes in the BGP LOC-RIB and bypass the FLOW SPEC routes validation process described in chapter 1)
[edit protocols bgp]
group FLOWSPEC {
type internal;
local-address 10.1.1.3;
family inet {
unicast;
flow {
no-validate NO-VAIDATION;
}
}
neighbor 10.1.1.2 {
description FS-SERVER;
}
}
[edit policy-options]
policy-statement NO-VAIDATION {
term 1 {
then accept;
}
}
Flow routes (BGP or static) are installed within the table “inetflow.0”.
sponge@bob> show route table inetflow.0 detail
inetflow.0: 1 destinations, 1 routes (2 active, 0 holddown, 0 hidden)
*,10.1.1.1,proto=17,port=53/term:2 (1 entry, 1 announced)
*Flow Preference: 5
Next hop type: Fictitious
Address: 0x904d5e4
Next-hop reference count: 2
State: <Active>
Local AS: 65000
Age: 38
Validation State: unverified
Task: RT Flow
Announcement bits (2): 0-Flow 1-BGP_RT_Background
AS path: I
AS path: Recorded
Communities: traffic-rate:0:1875
As I said previously, Next-Hop is always Fictitious (because it is set to 0). The Flow Specification is encoded in an n-turple (the BGP FLOW routes) and rules as an extended-community. Here, the rule type is traffic-rate used for rate-limiting or discarding (rate=0) a specific flow.
Junos then converts this route on a well-known firewall-filter and pushes the filter update on every PFE in both directions (Input / Output). This Firewall Filter is called “__FlowSpec_default_inet__”.
This is not an hidden filter and you can have access to it via the cli command:
sponge@bob> show firewall filter __FlowSpec_default_inet__
Filter: __FlowSpec_default_inet__
Counters:
Name Bytes Packets
*,10.1.1.1,proto=17,port=53 0 0
Policers:
Name Bytes Packets
15K_*,10.1.1.1,proto=17,port=53 0 0
Actually, flow spec firewall filter is a dynamic FW filter which combines in one FW filter all the Flow Specification routes (as a “from” criteria) and their associated rules (as a “then” criteria). Each route can be view as a term.
FlowSpec firewall filter is always at the end of the Firewall Filter list. In other words, you can use before your own Firewall Filter(s) applied in both directions via the command “set interface xxx family inet filter [input|input-list|output|output-list]”. Your FW filter will be always analyzed first before the flow spec FW filter.
But the flow spec Firewall Filter will be always analyzed even if a previous static firewall filter has a “then” action to ACCEPT. So, you can never bypass the “__FlowSpec_default_inet__” filter. Also remember that FLOW SPEC Firewall Filter is applied in both direction at PFE level.
At PFE Level you can use this command to check how the “__FlowSpec_default_inet__” is programed.
ADPC0(bob vty)# show filter
Program Filters:
---------------
Index Dir Cnt Text Bss Name
-------- ------ ------ ------ ------ --------
[…]
17000 52 0 4 4 __default_arp_policer__
57008 104 288 16 16 __cfm_filter_shared_lc__
65024 104 144 36 36 __FlowSpec_default_inet__
65280 52 0 4 4 __auto_policer_template__
65281 104 0 16 16 __auto_policer_template_1__
[…]
16777216 104 288 36 36 fnp-filter-level-all
ADPC0(bob vty)# show filter index 65024 program
Program Filters:
---------------
Index Dir Cnt Text Bss Name
-------- ------ ------ ------ ------ --------
65024 104 144 36 36 __FlowSpec_default_inet__
Action directory: 2 entries (104 bytes)
0: accept counter 0 policer 0
-> 7:
1: accept
-> 8:(bss location 3:)
Counter directory: 1 entry (144 bytes)
0: Counter name "*,10.1.1.1,proto=17,port=53": 1 reference
Policer directory: 1 entry (176 bytes)
0: Policer name "15K_*,10.1.1.1,proto=17,port=53": 1 reference
Bandwidth Limit: 1875 bytes/sec.
Burst Size: 15000 bytes.
discard
Program instructions: 9 words
0: match protocol != 17 -> 8:
set icmp-type
match source-port == 53 -> 5:
set destination-port
match destination-port != 53 -> 8:
5: match source-address != 0x0a010101 -> 8:
terminate -> action index 0
8: terminate -> action index 1
As presented in chapter 1 ; the second useful Action type is “redirect”. A redirect action is also encoded in a specific extended community. In Junos you can then use this specific community to redirect a specific traffic within a VRF. Very useful for traffic mitigation or traffic inspection (IDP).
A flow route with the redirect action example:
sponge@bob> show route 10.1.1.0/30 detail
inetflow.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
10.1.1.0,*,proto=17/term:1 (1 entry, 1 announced)
*BGP Preference: 170/-101
Next hop type: Fictitious
Address: 0x9041484
Next-hop reference count: 1
State: <Active Int Ext>
Local AS: 65000 Peer AS: 65000
Age: 48
Validation State: unverified
Task: BGP_65000.81.253.192.94+31919
Announcement bits (1): 0-Flow
AS path: ?
AS path: Recorded
Communities: redirect:65000:123456
Accepted
Localpref: 100
Router ID: 81.253.192.94
The only limitation on Junos when you redirect traffic is that the server which will be in charge of mitigation or inspection or anything else must not be directly connected to the same router attached to the VRF. Indeed, Junos pushes FlowSpec routes towards all PFE. So if you dynamically redirect a traffic to a server to perform “traffic processing”, the “coming back” traffic will be again redirect and so on… In this configuration you will experience a forwarding loop.
The diagram below shows this limitation and 2 workarounds (design based).
On Junos traffic redirection is configured as following:
[edit protocols bgp]
group FLOWSPEC {
type internal;
local-address 10.1.1.3;
family inet {
unicast;
flow {
no-validate NO-VAIDATION;
}
}
neighbor 10.1.1.2 {
description FS-SERVER;
}
}
[edit policy-options]
policy-statement NO-VAIDATION {
term 1 {
from community redirect;
to instance PROCESSING-VRF;
then accept;
}
term 2 {
then accept;
}
}
community redirect membersredirect:65000:123456;
[edit routing-instances]
PROCESSING-VRF {
instance-type vrf;
interface ge-2/2/1.0;
route-distinguisher 12.2.2.2:1234;
vrf-target target:65000:123456;
routing-options {
static {
defaults {
resolve;
}
# DEFAULT route if Server in charge of processing traffic is directly attached
# to the VRF
route 0.0.0.0/0 next-hop 10.128.2.10;
}
}
}