SDN控制器Floodlight源码学习(八)--转发模块(Forwarding)

很久没更新了，今天来学习下Forwarding模块,先看看官网上对这个模块的说明:

Description
Forwarding will forward packets between two devices. The source and destination devices will be classified by the IDeviceService.
How it works
Since Floodlight is designed to work in networks that contain both OpenFlow and non-OpenFlow switches Forwarding has to take this into account. The algorithm will find all OpenFlow islands that have device attachment points for both the source and destination devices. FlowMods will then be installed along the shortest path for the flow. If a PacketIn is received on an island and there is no attachment point for the device on that island the packet will be flooded.

官网上主要就是说，Forwarding模块能够找到源和目的设备之间的最短路径，并通过下发流表项的方式让交换机来对数据包进行转发。下面我们深入的学习下该模块.

Forwarding作为一个实现了IOFMessageListener的模块，表示它处于进入控制器的数据包的处理队列中，我们先来看receive函数的功能:

@Override
    public Command receive(IOFSwitch sw, OFMessage msg, FloodlightContext cntx) {
    switch (msg.getType()) {
        case PACKET_IN:
            IRoutingDecision decision = null;
            if (cntx != null) {
                decision = RoutingDecision.rtStore.get(cntx, IRoutingDecision.CONTEXT_DECISION);
            }

            return this.processPacketInMessage(sw, (OFPacketIn) msg, decision, cntx);
        default:
            break;
        }
        return Command.CONTINUE;
    }

通过代码可以看出，Forwarding模块只处理packet_in消息，首先Forwarding模块会从RoutingDecision模块的上线文环境中获取路由策略，并传入processPacketInMessage函数中进行处理，然后我们看看processPacketInMessage模块:

@Override
    public Command processPacketInMessage(IOFSwitch sw, OFPacketIn pi, IRoutingDecision decision, FloodlightContext cntx) {
        Ethernet eth = IFloodlightProviderService.bcStore.get(cntx, IFloodlightProviderService.CONTEXT_PI_PAYLOAD);
        // We found a routing decision (i.e. Firewall is enabled... it's the only thing that makes RoutingDecisions)
        if (decision != null) {
            if (log.isTraceEnabled()) {
                log.trace("Forwarding decision={} was made for PacketIn={}", decision.getRoutingAction().toString(), pi);
            }
            switch(decision.getRoutingAction()) {
            case NONE:
                // don't do anything
                return Command.CONTINUE;
            case FORWARD_OR_FLOOD:
            case FORWARD:
                doForwardFlow(sw, pi, decision, cntx, false);
                return Command.CONTINUE;
            case MULTICAST:
                // treat as broadcast
                doFlood(sw, pi, decision, cntx);
                return Command.CONTINUE;
            case DROP:
                doDropFlow(sw, pi, decision, cntx);
                return Command.CONTINUE;
            default:
                log.error("Unexpected decision made for this packet-in={}", pi, decision.getRoutingAction());
                return Command.CONTINUE;
            }
        } else { // No routing decision was found. Forward to destination or flood if bcast or mcast.
            if (log.isTraceEnabled()) {
                log.trace("No decision was made for PacketIn={}, forwarding", pi);
            }

            if (eth.isBroadcast() || eth.isMulticast()) {
                doFlood(sw, pi, decision, cntx);
            } else {
                doForwardFlow(sw, pi, decision, cntx, false);
            }
        }

        return Command.CONTINUE;
    }

1.进入函数以后，首先判断是否存在路由策略，如果存在，则根据路由策略的类型（NONE、FORWARD_OR_FLOOD、FORWARD、MULTICAST、DROP）来对数据包进行处理。
2.如果不存在路由策略，则判断二层报文是否为广播和多播，如果是广播和多播则进行doFlood，如果为单播则进行doForwardFlow。
以上处理的流程可以通过下图体现:

下面我们进一步看看每个具体的处理代码
1.doForwardFlow 转发函数

protected void doForwardFlow(IOFSwitch sw, OFPacketIn pi, IRoutingDecision decision, FloodlightContext cntx, boolean requestFlowRemovedNotifn) {
        //获取源端口
        OFPort srcPort = OFMessageUtils.getInPort(pi);
        //获取源IP地址
        DatapathId srcSw = sw.getId();

        //获取目标设备
        IDevice dstDevice = IDeviceService.fcStore.get(cntx, IDeviceService.CONTEXT_DST_DEVICE);
        //获取源设备
        IDevice srcDevice = IDeviceService.fcStore.get(cntx, IDeviceService.CONTEXT_SRC_DEVICE);

        //如果没有目标设备，则广播 
        if (dstDevice == null) {
            log.debug("Destination device unknown. Flooding packet");
            doFlood(sw, pi, decision, cntx);
            return;
        }

        //如果源设备没有，直接返回
        if (srcDevice == null) {
            log.error("No device entry found for source device. Is the device manager running? If so, report bug.");
            return;
        }

        //如果配置为允许ARP广播同时以太网是ARP类型，则广播
        /* Some physical switches partially support or do not support ARP flows */
        if (FLOOD_ALL_ARP_PACKETS && 
                IFloodlightProviderService.bcStore.get(cntx, IFloodlightProviderService.CONTEXT_PI_PAYLOAD).getEtherType() 
                == EthType.ARP) {
            log.debug("ARP flows disabled in Forwarding. Flooding ARP packet");
            doFlood(sw, pi, decision, cntx);
            return;
        }

        //如果发送pack_in消息的端口不是节点端口，那么只做转发
        /* This packet-in is from a switch in the path before its flow was installed along the path */
        if (!topologyService.isEdge(srcSw, srcPort)) {  
            log.debug("Packet destination is known, but packet was not received on an edge port (rx on {}/{}). Flooding packet", srcSw, srcPort);
            doFlood(sw, pi, decision, cntx);
            return; 
        }   

        //获取目标设备所关联的端口        
        for (SwitchPort ap : dstDevice.getAttachmentPoints()) {
            if (topologyService.isEdge(ap.getNodeId(), ap.getPortId())) {
                dstAp = ap;
                break;
            }
        }   

        if (dstAp == null) {
            log.debug("Could not locate edge attachment point for destination device {}. Flooding packet");
            doFlood(sw, pi, decision, cntx);
            return; 
        }

        //查看目标端口和源端口是不是同一个端口
        if (sw.getId().equals(dstAp.getNodeId()) && srcPort.equals(dstAp.getPortId())) {
            log.info("Both source and destination are on the same switch/port {}/{}. Dropping packet", sw.toString(), srcPort);
            return;
        }           

        U64 flowSetId = flowSetIdRegistry.generateFlowSetId();
        U64 cookie = makeForwardingCookie(decision, flowSetId);
        //找到从源到目标的路径
        Path path = routingEngineService.getPath(srcSw, 
                srcPort,
                dstAp.getNodeId(),
                dstAp.getPortId());

        Match m = createMatchFromPacket(sw, srcPort, pi, cntx);

        if (! path.getPath().isEmpty()) {
            if (log.isDebugEnabled()) {
                log.debug("pushRoute inPort={} route={} " +
                        "destination={}:{}",
                        new Object[] { srcPort, path,
                                dstAp.getNodeId(),
                                dstAp.getPortId()});
                log.debug("Creating flow rules on the route, match rule: {}", m);
            }

            //设置流表
            pushRoute(path, m, pi, sw.getId(), cookie, 
                    cntx, requestFlowRemovedNotifn,
                    OFFlowModCommand.ADD);  

            for (NodePortTuple npt : path.getPath()) {
                flowSetIdRegistry.registerFlowSetId(npt, flowSetId);
            }
        } /* else no path was found */
    }

2.doFlood 泛洪函数

protected void doFlood(IOFSwitch sw, OFPacketIn pi, IRoutingDecision decision, FloodlightContext cntx) {
        OFPort inPort = OFMessageUtils.getInPort(pi);
        //构建packout消息
        OFPacketOut.Builder pob = sw.getOFFactory().buildPacketOut();
        List<OFAction> actions = new ArrayList<OFAction>();
        //获取交换机的广播域端口
        Set<OFPort> broadcastPorts = this.topologyService.getSwitchBroadcastPorts(sw.getId());

        if (broadcastPorts.isEmpty()) {
            log.debug("No broadcast ports found. Using FLOOD output action");
            broadcastPorts = Collections.singleton(OFPort.FLOOD);
        }

        //设置action
        for (OFPort p : broadcastPorts) {
            if (p.equals(inPort)) continue;
            actions.add(sw.getOFFactory().actions().output(p, Integer.MAX_VALUE));
        }
        pob.setActions(actions);
        // log.info("actions {}",actions);
        // set buffer-id, in-port and packet-data based on packet-in
        pob.setBufferId(OFBufferId.NO_BUFFER);
        OFMessageUtils.setInPort(pob, inPort);
        pob.setData(pi.getData());

        if (log.isTraceEnabled()) {
            log.trace("Writing flood PacketOut switch={} packet-in={} packet-out={}",
                    new Object[] {sw, pi, pob.build()});
        }
        messageDamper.write(sw, pob.build());

        return;
    }

3.doDropFlow 丢掉流函数

protected void doDropFlow(IOFSwitch sw, OFPacketIn pi, IRoutingDecision decision, FloodlightContext cntx) {
        OFPort inPort = OFMessageUtils.getInPort(pi);
        Match m = createMatchFromPacket(sw, inPort, pi, cntx);
        OFFlowMod.Builder fmb = sw.getOFFactory().buildFlowAdd();
        List<OFAction> actions = new ArrayList<OFAction>(); // set no action to drop
        U64 flowSetId = flowSetIdRegistry.generateFlowSetId();
        U64 cookie = makeForwardingCookie(decision, flowSetId); 

        /* If link goes down, we'll remember to remove this flow */
        if (! m.isFullyWildcarded(MatchField.IN_PORT)) {
            flowSetIdRegistry.registerFlowSetId(new NodePortTuple(sw.getId(), m.get(MatchField.IN_PORT)), flowSetId);
        }

        log.info("Dropping");
        fmb.setCookie(cookie)
        .setHardTimeout(FLOWMOD_DEFAULT_HARD_TIMEOUT)
        .setIdleTimeout(FLOWMOD_DEFAULT_IDLE_TIMEOUT)
        .setBufferId(OFBufferId.NO_BUFFER) 
        .setMatch(m)
        .setPriority(FLOWMOD_DEFAULT_PRIORITY);

        FlowModUtils.setActions(fmb, actions, sw);

        /* Configure for particular switch pipeline */
        if (sw.getOFFactory().getVersion().compareTo(OFVersion.OF_10) != 0) {
            fmb.setTableId(FLOWMOD_DEFAULT_TABLE_ID);
        }

        if (log.isDebugEnabled()) {
            log.debug("write drop flow-mod sw={} match={} flow-mod={}",
                    new Object[] { sw, m, fmb.build() });
        }
        boolean dampened = messageDamper.write(sw, fmb.build());
        log.debug("OFMessage dampened: {}", dampened);
    }

以上为Forwarding 模块大致的功能，那么下面通过一个实验来熟悉这个模块,实验的用例为下图:

实验环境说明:
一台控制器(192.168.56.1)，三台OF交换机，S1下有两台主机，IP地址分别为192.168.1.1(h1)、192.168.1.3(h3)。S2下有一台主机，IP地址为192.168.1.2(h2)。
实验步骤
1.取消Forwarding模块加载,观察联通情况

从实验可以看出,再控制器不加载Forwarding模块的情况下，ping 192.168.1.2直接返回Destination Host Unreachable,再通过arp -a命令看到h1没有得到h2的MAC地址。证明OF交换机并没有进行数据包的转发。
2.恢复Forwarding模块的加载，观察网络情况
恢复Forwarding模块加载以后，同样我们在h1上ping h2,可以看到h1已经能够获取h2的mac地址.

同样我在每个接口进行抓包，抓包的情况如下:
s1-eth1接口抓包

s1-eth2接口抓包

s1-eth3接口抓包

s2-eth2接口抓包

OF交换机到控制器接口抓包

从这些抓包的情况可以分析出h1 ping h2的工作情况:
step1:h1查看本地mac地址表，没有找到h2的mac,则发送ARP消息到s1交换机。
step2:s1收到ARP消息以后，发现流表中不存在对应项，则通过packet_in消息将ARP消息发送到控制器.
step3:控制器收到packet_in消息后，通过Forwarding模块进行处理，发现为ARP消息，则通过doFlood函数进行处理，形成packet_out消息，并下发至s1。
step4:s1收到packet_out消息后，将arp通过除入端口之外的其它端口进行转发。
step5:ARP数据包进入s3,s3则重复step2-step3的操作，将数据包转发到s2.
step6:s2重复step2-step3的操作，将ARP包发送至h2 .
step7:h2收到arp后，产生响应，数据包逆向返回到h1（与ARP过程相反）.
注意：在控制器通过packet_out消息控制OF交换机转发数据包以后，还会下发FLOW_MOD消息，在相应的交换机的流表中添加流表项，这样下次数据包到来的时候，OF交换机就可以通过流表进行数据转发，而不需要告诉控制器了,下图为s1-s3交换机中的流表项:

这样我们基本上了解了Forwarding模块的工作原理.