openwrt 网桥 brctl配置linux bridge及虚拟bridge实现

有线网卡和无线网卡上网实现

Linux网关模式下将有线LAN和无线LAN共享网段实现局域网内互联:

思路其实很简单:就是将虚拟出一个bridge口,将对应的有线LAN和无线LAN都绑定在这个虚拟bridge口上,并给这个bridge口分配一个地址,其他子网微机配置网关为bridge口的地址便可以了。当然,因为是设备是网关模式,路由和nat也是必须的了。如果设备本身便是网桥模式,那么路由和nat便可以省掉了。

    brctl addbr bridge

    brctl addif bridge eth0

    brctl addif bridge ath0

    ifconfig eth0  0.0.0.0

    ifconfig bridge 10.0.0.1netmask 255.255.255.0 up

    添加iptables -t nat -A POSTROUTING -o eth1-j SNAT --to 192.168.2.173

    将有线和无线都设置为10.0.0.*网段,即可通过网上邻居进行访问

 

参考:https://blog.csdn.net/u011956172/article/details/38581247

1 man brctl命令与安装

1.1 ubuntu下安装:

 
  1. root@localhost:/etc/init.d# apt-get install bridge-utils

  2. Reading package lists... Done

  3. Building dependency tree

  4. Reading state information... Done

  5. The following NEW packages will be installed:

  6. bridge-utils

  7. 0 upgraded, 1 newly installed, 0 to remove and 105 not upgraded.

  8. Need to get 29.2 kB of archives.

  9. After this operation, 146 kB of additional disk space will be used.

  10. Get:1 http://us.archive.ubuntu.com/ubuntu/ trusty/main bridge-utils amd64 1.5-6ubuntu2 [29.2 kB]

  11. Fetched 29.2 kB in 1s (21.6 kB/s)

  12. Selecting previously unselected package bridge-utils.

  13. (Reading database ... 60229 files and directories currently installed.)

  14. Preparing to unpack .../bridge-utils_1.5-6ubuntu2_amd64.deb ...

  15. Unpacking bridge-utils (1.5-6ubuntu2) ...

  16. Processing triggers for man-db (2.6.7.1-1ubuntu1) ...

  17. Setting up bridge-utils (1.5-6ubuntu2) ...

1.2 命令:

NAME
       brctl - ethernet bridge administration
SYNOPSIS
       brctl [command]
DESCRIPTION
       brctl is used to set up, maintain, and inspect the ethernet bridge configuration in the linux kernel.
       An  ethernet bridge is a device commonly used to connect different networks of ethernets together, so that these ether‐
       nets will appear as one ethernet to the participants.
       Each of the ethernets being connected corresponds to one physical interface in the bridge. These  individual  ethernets
       are bundled into one bigger ('logical') ethernet, this bigger ethernet corresponds to the bridge network interface.
INSTANCES
       brctl  addbr <name> creates a new instance of the ethernet bridge. The network interface corresponding to
       the bridge will be called <name>.
       brctl delbr <name> deletes the instance <name> of the ethernet bridge. The network interface  corresponding
       to the bridge must be down before it can be deleted!
       brctl show shows all current instances of the ethernet bridge.
PORTS
       Each  bridge has a number of ports attached to it.Network traffic coming in on any of these ports will be forwarded to
       the other ports transparently, so that the bridge is invisible to the rest of the network (i.e. it will not show up  in
       traceroute(8) ).
       brctl  addif <brname> <ifname> will make the interface <ifname> a port of the bridge <brname>. This means
       that all frames received on <ifname> will be processed as if destined for the bridge.  Also,  when  sending  frames  on
       <brname>, <ifname> will be considered as a potential output interface.
       brctl delif <brname> <ifname> will detach the interface <ifname> from the bridge <brname>.
       brctl show <brname> will show some information on the bridge and its attached ports.
AGEING
       The  bridge  keeps  track  of ethernet addresses seen on each port.When it needs to forward a frame, and it happens to
       know on which port the destination ethernet address (specified in the frame) is located, it can 'cheat'  by  forwarding
       the frame to that port only, thus saving a lot of redundant copies and transmits.
       However,  the ethernet address location data is not static data. Machines can move to other ports, network cards can be
       replaced (which changes the machine's ethernet address), etc.
       brctl showmacs <brname> shows a list of learned MAC addresses for this bridge.
       brctl setageing <brname> <time> sets the ethernet (MAC) address ageing time, in seconds. After <time>  seconds  of  not
       having  seen  a frame coming from a certain address, the bridge will time out (delete) that address from the Forwarding
       DataBase (fdb).
       brctl setgcint <brname> <time> sets the garbage collection interval for the bridge <brname>  to  <time>  seconds.  This
       means that the bridge will check the forwarding database for timed out entries every <time> seconds.
SPANNING TREE PROTOCOL
      ...
NOTES
       brctl(8) replaces the older brcfg tool.

 

 
  1. Usage: brctl [commands]

  2. commands:

  3. addbr <bridge> add bridge

  4. delbr <bridge> delete bridge

  5. addif <bridge> <device> add interface to bridge

  6. delif <bridge> <device> delete interface from bridge

  7. hairpin <bridge> <port> {on|off} turn hairpin on/off

  8. setageing <bridge> <time> set ageing time

  9. setbridgeprio <bridge> <prio> set bridge priority

  10. setfd <bridge> <time> set bridge forward delay

  11. sethello <bridge> <time> set hello time

  12. setmaxage <bridge> <time> set max message age

  13. setpathcost <bridge> <port> <cost> set path cost

  14. setportprio <bridge> <port> <prio> set port priority

  15. show [ <bridge> ] show a list of bridges

  16. showmacs <bridge> show a list of mac addrs

  17. showstp <bridge> show bridge stp info

  18. stp <bridge> {on|off} turn stp on/off

 

2 实例

2.1 实例1

 
  1. user@compute02:~$ brctl show

  2. bridge name bridge id STP enabled interfaces

  3. qbr48450f11-e0 8000.3e04f36e5929 no qvb48450f11-e0

  4. tap48450f11-e0

  5. qbr850fef60-99 8000.fa8822149d71 no qvb850fef60-99

  6. tap850fef60-99

  7. qbr9fd0b8b9-7a 8000.662b93760507 no qvb9fd0b8b9-7a

  8. tap9fd0b8b9-7a

  9. virbr0 8000.000000000000 yes

  10. user@compute02:~$

  11. user@compute02:~$ brctl show qbr9fd0b8b9-7a

  12. bridge name bridge id STP enabled interfaces

  13. qbr9fd0b8b9-7a 8000.662b93760507 no qvb9fd0b8b9-7a

  14. tap9fd0b8b9-7a

  15. user@compute02:~$

  16. user@compute02:~$ brctl showmacs qbr9fd0b8b9-7a

  17. port no mac addr is local? ageing timer

  18. 1 66:2b:93:76:05:07 yes 0.00

  19. 1 fa:16:3e:3a:43:db no 0.54

  20. 2 fa:16:3e:51:0a:68 no 2.14

  21. 1 fa:16:3e:64:b0:ce no 0.14

  22. 2 fe:16:3e:51:0a:68 yes 0.00

  23. user@compute02:~$

  24. user@compute02:~$ ifconfig | grep 66\:2b

  25. qbr9fd0b8b9-7a Link encap:Ethernet HWaddr 66:2b:93:76:05:07

  26. qvb9fd0b8b9-7a Link encap:Ethernet HWaddr 66:2b:93:76:05:07

  27. user@compute02:~$ ifconfig | grep 16\:3e\:

  28. tap48450f11-e0 Link encap:Ethernet HWaddr fe:16:3e:3a:43:db

  29. tap850fef60-99 Link encap:Ethernet HWaddr fe:16:3e:64:b0:ce

  30. tap9fd0b8b9-7a Link encap:Ethernet HWaddr fe:16:3e:51:0a:68

2.2 实例2

在linux上还可以通过如下方式创建tap:
添加虚拟网卡tap
tunctl -b
tap0 -------> 执行上面使命就会生成一个tap,后缀从0,1,2依次递增  
激活创建的tap
ip link set tap0 up
这里tap可以桥接到网桥上,成为网桥的一个端口。
显示网桥信息
 brctl show    
添加网桥
brctl addbr virbr0
激活网桥
ip link set virbr0 up
将tap0虚拟网卡添加到指定网桥上。
brctl addif br0 tap0
在创建kvm虚拟机时,可以指定虚拟机连接在哪个端口上。请看如下举例:
qemu-kvm -m 4096 -smp 4 -hda /home/linux_hzhsan.qcow2 -net nic,model=e1000,macaddr=53:02:43:BC:F8:C1 -net tap,ifname=tap0,script=no,downscript=no &
就像一台物理主机(对应这里的虚拟主机)用网线直接连接到交换机的一个网口上(这网口就对应上面的tap0)
这个tap0在创建的虚拟机中,会有对应的一个虚拟物理网卡(在虚拟客户机中可以看到eth0)

3 相关理解

        网桥是在内核中虚拟出来的,可以将主机上真实的物理网卡(如eth0,eth1),或虚拟的网卡(tap0,tap1,vnet0,vnet1)桥接上来。桥接上来的网卡就相当于网桥上的端口。 端口收到的数据包都提交给这个虚拟的”网桥“,让其进行转发。
        网桥其实是不用配置ip的,给网桥配制ip地址,是为了方便网桥所在的主机和网桥所桥接的网卡(包括虚拟网卡)进行通信。一般会配成同一个网段的ip。桥接在网桥上的网卡(包括物理网卡和虚拟网卡)不能配制ip地址。但是其对应的虚拟机可以配制ip地址,如果和网桥的ip是同一个网段的话,网桥所在的物理主机和这个网桥上桥接的网卡所对应的虚拟主机就可以进行通信了(能够ping通)。
在给linux创建虚拟机时,为了方便主机与其虚拟机进行通信,常用到网桥,并且常给网桥配制ip地址。


注意:必须给网桥br0配制IP地址,且保持和tap0所对应的虚拟机网卡上配制的ip地址在同一网段,这样才能从主机中直接访问到虚拟机。

 

参考:

1 bridge By Linux Foundatio... - November 19, 2009 - 10:23am         

原文:http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge
A bridge is a way to connect twoEthernet segments together in a protocol independent way. Packets are forwarded based on Ethernet address, rather than IP address (like a router). Since forwarding is done at Layer 2, all protocols can go transparently through a bridge.

The Linux bridge code implements a subset of the ANSI/IEEE 802.1d standard.[1]. The original Linux bridging was first done in Linux 2.2, then rewritten by Lennert Buytenhek. The code for bridging has been integrated into 2.4 and 2.6 kernel series.

Contents

 

Bridging and Firewalling

 

A Linux bridge is more powerful than a pure hardware bridge because it can also filter and shape traffic. The combination of bridging and firewalling is done with the companion projectebtables.

Status

 

The code is updated as part of the 2.4 and 2.6 kernels available at kernel.org.

Possible future enhancements are:

  • Document STP filtering
  • Netlink interface to control bridges (prototype in 2.6.18)
  • STP should be in user space
  • Support RSTP and other 802.1d STP extensions

Downloading

 

Bridging is supported in the current 2.4 (and 2.6) kernels from all the major distributors. The required administration utilities are in the bridge-utils package in most distributions. Package releases are maintained on the Download page.

You can also build your own up to date version by getting the latest kernel fromkernel.org and build the utilities based from the source code in bridge-utils GIT repository.

 $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/shemminger/bridge-utils.git
 $ cd bridge-utils
 $ autoconf
 $ ./configure

Kernel Configuration

 

You need to enable bridging in the kernel. Set "networking -> 802.1d Ethernet Bridging" to either yes or module

Manual Configuration

Network cards

Before you start make sure both network cards are set up and working properly.Don't set the IP address, and don't let the startup scripts run DHCP on the ethernet interfaces either. The IP address needs to be set after the bridge has been configured.

The command ifconfig should showboth network cards, and they should be DOWN.

Module loading

 

In most cases, the bridge code is built as a module. If the module is configured and installed correctly, it will get automatically loaded on the first brctl command.

If your bridge-utilities have been correctly built and your kernel and bridge-module are OK, then issuing a brctl should show a small command synopsis.

# brctl
# commands:
        addbr           <bridge>                add bridge
        delbr           <bridge>                delete bridge
        addif           <bridge> <device>       add interface to bridge
        delif           <bridge> <device>       delete interface from bridge
        setageing       <bridge> <time>         set ageing time
        setbridgeprio   <bridge> <prio>         set bridge priority
        setfd           <bridge> <time>         set bridge forward delay
        sethello        <bridge> <time>         set hello time
        setmaxage       <bridge> <time>         set max message age
        setpathcost     <bridge> <port> <cost>  set path cost
        setportprio     <bridge> <port> <prio>  set port priority
        show                                    show a list of bridges
        showmacs        <bridge>                show a list of mac addrs
        showstp         <bridge>                show bridge stp info
        stp             <bridge> <state>        turn stp on/off

Creating a bridge device

The command

 brctl addbr "bridgename"

creates a logical bridge instance with the name bridgename. You will need at least one logical instance to do any bridging at all. You can interpret the logical bridge as a container for the interfaces taking part in the bridging. Each bridging instance is represented by a new network interface.

The corresponding shutdown command is:

 brctl delbr bridgename

Adding devices to a bridge

The command

 brctl addif bridgename device

adds the network device device to take part in the bridging of "bridgename." All the devices contained in a bridge act as one big network. It is not possible to add a device to multiple bridges or bridge a bridge device, because it just wouldn't make any sense! The bridge will take a short amount of time when a device is added to learn the Ethernet addresses on the segment before starting to forward.

The corresponding command to take an interface out of the bridge is:

 brctl delif  bridgename device

Showing devices in a bridge

The brctl show command gives you a summary about the overall bridge status, and the instances running as shown below:

 # brctl addbr br549
 # brctl addif br549 eth0
 # brctl addif br549 eth1
 # brctl show
 bridge name     bridge id               STP enabled     interfaces
 br549           8000.00004c9f0bd2       no              eth0
                                                         eth1

Once a bridge is running the brctl showmacs will show information about network addresses of traffic being forwarded (and the bridge itself).

 # brctl showmacs br549
 port no mac addr                is local?       ageing timer
   1     00:00:4c:9f:0b:ae       no                17.84
   1     00:00:4c:9f:0b:d2       yes                0.00
   2     00:00:4c:9f:0b:d3       yes                0.00
   1     00:02:55:1a:35:09       no                53.84
   1     00:02:55:1a:82:87       no                11.53
  ...

The aging time is the number of seconds a MAC address will be kept in the forwarding database after having received a packet from this MAC address. The entries in the forwarding database are periodically timed out to ensure they won't stay around forever. Normally there should be no need to modify this parameter, but it can be changed with (time is in seconds).

  # brctl setageing bridgename time

Setting ageing time to zero makes all entries permanent.

Spanning Tree Protocol

 

If you are running multiple or redundant bridges, then you need to enable the Spanning Tree Protocol (STP) to handle multiple hops and avoid cyclic routes.

 # brctl stp br549 on

You can see the STP parameters with:

 # brctl showstp br549
 br549
  bridge id              8000.00004c9f0bd2
  designated root        0000.000480295a00
  root port                 1                    path cost                104
  max age                  20.00                 bridge max age           200.00
  hello time                2.00                 bridge hello time         20.00
  forward delay           150.00                 bridge forward delay      15.00
  ageing time             300.00                 gc interval                0.00
  hello timer               0.00                 tcn timer                  0.00
  topology change timer     0.00                 gc timer                   0.33
  flags
 
 eth0 (1)
  port id                8001                    state                   forwarding
  designated root        0000.000480295a00       path cost                100
  designated bridge      001e.00048026b901       message age timer         17.84
  designated port        80c1                    forward delay timer        0.00
  designated cost           4                    hold timer                 0.00
  flags
 
 eth1 (2)
  port id                8002                    state                   disabled
  designated root        8000.00004c9f0bd2       path cost                100
  designated bridge      8000.00004c9f0bd2       message age timer          0.00
  designated port        8002                    forward delay timer        0.00
  designated cost           0                    hold timer                 0.00
  flags

STP tuning

 

There are a number of parameters related to theSpanning Tree Protocol that can be configured. The code autodetects the speed of the link and other parameters, so these usually don't need to be changed.

Bridge priority

 

Each bridge has a relative priority and cost. Each interface is associated with a port (number) in the STP code. Each has a priority and a cost, that is used to decide which is the shortest path to forward a packet. The lowest cost path is always used unless the other path is down. If you have multiple bridges and interfaces then you may need to adjust the priorities to achieve optimium performance.

  # brctl setbridgeprio bridgename priority

The bridge with the lowest priority will be elected as the root bridge. The root bridge is the "central" bridge in the spanning tree.

Path priority and cost

 

Each interface in a bridge could have a different speed and this value is used when deciding which link to use. Faster interfaces should have lower costs.

 # brctl setpathcost bridge port cost

For multiple ports with the same cost there is also a priority

Forwarding delay

 

Forwarding delay time is the time spent in each of the Listening and Learning states before the Forwarding state is entered. This delay is so that when a new bridge comes onto a busy network it looks at some traffic before participating.

 #  brctl setfd bridgename time

Hello time

 

Periodically, a hello packet is sent out by the Root Bridge and the Designated Bridges. Hello packets are used to communicate information about the topology throughout the entire Bridged Local Area Network.

 # brctl sethello bridgename time

Max age

 

If a another bridge in the spanning tree does not send out a hello packet for a long period of time, it is assumed to be dead. This timeout is set with:

 # brctl maxage  bridgename time

Multicast (IGMP) snooping

 

IGMP snooping support is not yet included in bridge-utils or iproute2, but it can be easily controlled through sysfs interface. For brN, the settings can be found under /sys/devices/virtual/net/brN/bridge.

multicast_snooping

This option allows the user to disable IGMP snooping completely. It also allows the user to reenable snooping when it has been automatically disabled due to hash collisions. If the collisions have not been resolved however the system will refuse to reenable snooping.

multicast_router

This allows the user to forcibly enable/disable ports as having multicast routers attached. A port with a multicast router will receive all multicast traffic.

The value 0 disables it completely. The default is 1 which lets the system automatically detect the presence of routers (currently this is limited to picking up queries), and 2 means that the ports will always receive all multicast traffic.

Note: this setting can be enabled/disable on a per-port basis, also through sysfs interface (e.g. if eth0 is some bridge's active port, then you can adjust /sys/...../eth0/brport/multicast_router)

hash_{max,elasticity}

These settings allow the user to control the hash elasticity/max parameters. The elasticity setting does not take effect until the next new multicast group is added. At which point it is checked and if after rehashing it still can't be satisfied then snooping will be disabled.

The max setting on the other hand takes effect immediately. It must be a power of two and cannot be set to a value less than the current number of multicast group entries. This is the only way to shrink the multicast hash.

remaining multicast_* options

These allow the user to control various values related to IGMP snooping.

More details about the options, some discussions and rationale can be found inhttp://thread.gmane.org/gmane.linux.network/153338

Sample setup

 

The basic setup of a bridge is done like:

 # ifconfig eth0 0.0.0.0
 # ifconfig eth1 0.0.0.0
 # brctl addbr mybridge
 # brctl addif mybridge eth0
 # brctl addif mybridge eth1 
 # ifconfig mybridge up

This will set the host up as a pure bridge, it will not have an IP address for itself, so it can not be remotely accessed (or hacked) via TCP/IP.

Optionally you can configure the virtual interface mybridge to take part in your network. It behaves like one interface (like a normal network card). Exactly that way you configure it, replacing the previous command with something like:

 # ifconfig mybridge 192.168.100.5 netmask 255.255.255.0

If you want your bridge to automatically get its IP address from the ADSL modem via DHCP (or a similar configuration), do this:

 # ifconfig eth0 0.0.0.0
 # ifconfig eth1 0.0.0.0
 # brctl addbr mybridge
 # brctl addif mybridge eth0
 # brctl addif mybridge eth1 
 # dhclient mybridge

If you do this many times, you may end up with lots of dhclient processes. Either kill them impolitely or learn aboutomshell(1).

Configuration with /etc/net

In /etc/net we first configure two ethernet devices port0 and port1:

 # cat >> /etc/net/iftab
 port0 mac 00:13:46:66:01:5e
 port1 mac 00:13:46:66:01:5f
 ^D
 # mkdir /etc/net/ifaces/port0
 # cat > /etc/net/ifaces/port0/options
 TYPE=eth
 MODULE=via-rhine
 # mkdir /etc/net/ifaces/port1
 # cat > /etc/net/ifaces/port1/options
 TYPE=eth
 MODULE=via-rhine
 ^D

Then we describe the bridge:

 # mkdir /etc/net/ifaces/mybridge
 # cat > /etc/net/ifaces/mybridge/options
 TYPE=bri
 HOST='port0 port1'
 ^D
 # cat > /etc/net/ifaces/mybridge/brctl
 stp AUTO on
 ^D

Now we can use "ifup mybridge" to bring it up. port0 and port1 will be brought up automatically.

FAQ

What does a bridge do?

A bridge transparently relays traffic between multiple network interfaces. In plain English this means that a bridge connects two or more physical Ethernets together to form one bigger (logical) Ethernet.

Is it protocol independent?

Yes. The bridge knows nothing about protocols, it only sees Ethernet frames. As such,the bridging functionality is protocol independent, and there should be no trouble relaying IPX, NetBEUI, IP, IPv6, etc.

Why is this code better than a switch?

Please note that this code wasn't written with the intent of having Linux boxes take over from dedicated networking hardware. Don't see the Linux bridging code as a replacement for switches, but rather as an extension of the Linux networking capabilities. Just as there are situations where a Linux router is better than a dedicated router (and vice versa), there are situations where a Linux bridge is better than a dedicated bridge (and vice versa).

Most of the power of the Linux bridging code lies in its flexibility. There is a whole lot of bizarre stuff you can do with Linux already (read Linux Advanced Routing and Traffic Control document to see some of the possiblities), and the bridging code adds some more filter into the mix.

One of the most significant advantages of a Linux solution over a dedicated solution that come to mind is Linux' extensive firewalling capabilities.It is possible to use the full functionality of netfilter (iptables) in combination with bridging, which provides way more functionality than most proprietary offerings do.

Why is this code worse than a switch?

In order to act a a bridge,the network device must be placed into promiscuous mode which means it receives all traffic on a network. On a really busy network, this can eat significant bandwidth out of the processor, memory slowing the system down. The answer is tosetup either a separate dedicated Linux box as the bridge, or use a hardware switch.

What is the performance of the bridge?

The performance is limited by the network cards used and the processor. A research paper was done by James Yu at Depaul University comparing Linux bridging with a Catalyst switchYu-Linux-TSM2004.pdf

My bridge does not show up in traceroute!

It's not supposed to. The operation of a bridge is (supposed to be) fully transparent to the network,the networks that a bridge connects together are actually to be viewed as one big network. That's why the bridge does not show up in traceroute; the packets do not feel like they are crossing a subnet boundary.

For more information on this, read a book about TCP/IP networking.

It doesn't work!

It says: "br_add_bridge: bad address" when I try to add a bridge!

Either your kernel is old (2.2 or earlier), or you forgot to configure Ethernet bridging into your kernel.

No traffic gets trough (except ARP and STP)

Your kernel might have ethernet filtering (ebtables, bridge-nf, arptables) enabled, and traffic gets filtered. The easiest way to disable this is to go to /proc/sys/net/bridge. Check if the bridge-nf-* entries in there are set to 1; in that case, set them to zero and try again.

 # cd /proc/sys/net/bridge
 # ls
 bridge-nf-call-arptables  bridge-nf-call-iptables
 bridge-nf-call-ip6tables  bridge-nf-filter-vlan-tagged
 # for f in bridge-nf-*; do echo 0 > $f; done

Does bridging work on 2.2?

The base kernel for 2.2, did not support the current bridging code. The original development was on 2.2, and there used to be patches available for it. But these patches are no longer maintained.

Are there plans for RSTP (802.1w) support?

Yes, work is being done to integrate RSTP support in a future 2.6 release. The code was done for a version of 2.4 and needs to be cleaned up, tested and updated.

What can be bridged?

Linux bridging is very flexible;the LAN's can be either traditional Ethernet device's, or pseudo-devices such as PPP, VPN's or VLAN's.The only restrictions are that the devices:

  • All devices share the same maximum packet size (MTU). The bridge doesn't fragment packets.
  • Devices must look like Ethernet. i.e have 6 byte source and destination address.
  • Support promiscuous operation. The bridge needs to be able to receive all network traffic, not just traffic destined for its own address.
  • Allow source address spoofing. The bridge must be able to send data over network as if it came from another host.

Can I do bridging in combination with netfilter/iptables?

Yes. The code for this is available in most kernels. See ebtables project.

Does it work with Token Ring,FDDI, or Firewire?

No, the addressing and frame sizes are different.

I keep getting the message retransmitting tcn bpdu!

It means that your Linux bridge is retransmitting a Topology Change Notification Bridge Protocol Data Unit (so now you know what the letters are for :-). Seriously, there is probably another switch (or Linux bridge) nearby that isn't complying to the rules of the spanning tree protocol (which is what bridges speak).

In each bridged local area network, there is one 'master bridge', which is also called the root bridge. You can find out which bridge this is using brctl.

When the topology of a bridged local area network changes (f.e. somebody unplugs a cable between two bridges), the bridge which detects this sends a topology change notification to the root bridge. The root bridge will respond to this by setting a 'topology changed' bit in the hello packets it sends out for the next X seconds (X usually being 30). This way, all bridges will learn of the topology change, so that they can take measures like timing out learned MAC entries faster for example.

After having sent out a topology change notification, if a bridge does not find the 'topology changed' bit set in the hello packets received (which in essence serves as the 'acknowledgment' of the topology change notification), it concludes that the topology change notification was lost. So it will retransmit it. However, some bridges run lobotomized implementations of the Spanning Tree Protocol which causes them not to acknowledge topology change notifications. If you have one of those bridges as your root bridge, all of the other bridges will keep retransmitting their topology changed notifications. Which will lead to these kinds of syslog messages.

There are a number of things you can do:

  • Find out which bridge is the root bridge, find out where it is located, and what internetworking software it runs. Please report this info to the mailing list (or to me directly), so that I can keep a blacklist.
  • Force the linux bridge to be the root bridge. See what the priority of the current root bridge is, and use the brctl 'setbridgeprio' command to set the priority of the linux bridge to 1 lower. (The bridge with the lowest priority always becomes the root bridge.)
  • Disable the spanning tree protocol on your linux bridge altogether. In this case, watch out for bridging loops! If you have loops in your topology, and if no bridge in the loop is running the spanning tree protocol, mayhem will come your way, as packets will be forwarded forever. Don't Do This(TM).

It doesn't work with my regular Ethernet card!

Unfortunately, some network cards have buggy drivers that fail under load. The situation is improving, so having a current kernel and network driver can help. Also try swapping with another brand.

Please report all problems to the Bridge mailing list:bridge@osdl.org. If your network card doesn't work (even without bridging) then try the Linux networking mailing listlinux-net@vger.kernel.org

It doesn't work with my Wireless card!

This is a known problem, and it is not caused by the bridge code. Many wireless cards don't allow spoofing of the source address. It is a firmware restriction with some chipsets. You might find some information in the bridge mailing list archives to help. Has anyone found a way to get around Wavelan not allowing anything but its own MAC address? (answer by Michael Renzmann (mrenzmann at compulan.de))

Well, for 99% of computer users there will never be a way to get rid of this. For this function a special firmware is needed. This firmware can be loaded into the RAM of any WaveLAN card, so it could do its job with bridging. But there is no documentation on the interface available to the public. The only way to achieve this is to have a full version of the hcf library which controls every function of the card and also allows accessing the card's RAM. To get this full version Lucent wants to know that it will be a financial win for them, also you have to sign an NDA. So be sure that you won't most probably get access to this peace of software until Lucent does not change its mind in this (which I doubt never will happen).

If you urgently need to have a wireless LAN card which is able to bridge, you should use one of those having the prism chipset onboard (manufactured by Harris Intersil). There are drivers for those cards available at www.linux-wlan.com (which is the website from Absoval), and I found a mail that says that there is the necessary firmware and an upload tool available for Linux to the public. If you need additional features of an access point you should also talk to Absoval.

I still don't understand!!

Doing full bridging of wireless (802.11) requires supportingWDS. The current implementation doesn't do it.

It is possible to do limited wireless to Ethernet functionality with some wireless drivers. This requires the device to be able to support a different sender address and source address. That is what WDS provides.

There are ways to make it work, but it is not always straightforward and you probably won't get it right without a pretty solid understanding of 802.11, it's modes, and the frame header format.

I get the error 'too much work in interrupt'

This is because the network card is getting lots of packets. There are a few things you can try. First, build the driver with NAPI support (if it isn't on by default). NAPI means the driver will do the receive processing at soft IRQ, not at the low level interrupt.

If the driver doesn't support NAPI, you can try to increase the amount of work a driver will attempt to do in an interrupt. For 3c59x this is done with the option max_interrupt_work (so add something like 'options 3c59x max_interrupt_work=10000' to your /etc/modules.conf file), other cards might have similar options.

Does DHCP work over/through a bridge?

The bridge will forward DHCP traffic (broadcasts) and responses. You can also use DHCP to set the local IP address of the bridge pseudo-interface.

One common mistake is that the default bridge forwarding delay setting is 30 seconds. This means that for the first 30 seconds after an interface joins a bridge, it won't send anything. This is because if the bridge is being used in a complex topology, it needs to discover other bridges and not create loops. This problem was one of the reasons for the creation of Rapid Spanning Tree Protocol (RSTP).

If the bridge is being used standalone (no other bridges near by). Then it is safe to turn the forwarding delay off (set it to zero), before adding interface to a bridge. Then you can run DHCP client right away.

 # brctl setfd br0 0
 # brctl addif br0 eth0
 # dhclient eth0

Contact Info

 

The code is currently maintained by Stephen Hemminger<shemminger@osdl.org> for both 2.4 and 2.6 Linux. Bridge bugs and enhancements are discussed on the Bridge mailing list<bridge@osdl.org>. The list is open to anyone interested, use the web mailman interface http://lists.osdl.org/mailman/listinfo/bridge to subscribe.

External Links

 

 

2

Linux网关模式下将有线LAN和无线LAN共享网段实现局域网内互联:

 

brctl addbr bridge

 

brctl addif bridge eth0

 

brctl addif bridge ath0

 

ifconfig eth0 0.0.0.0

 

ifconfig bridge 10.0.0.1 netmask 255.255.255.0 up

 

添加iptables -t nat -A POSTROUTING -o eth1-j SNAT --to 192.168.2.173

 

将有线和无线都设置为10.0.0.*网段,即可通过网上邻居进行访问

 

当然了,要是Linux可以工作在网桥模式,必须安装网桥工具bridge-utils,运行命令:

 

yum install bridge-utils

 

或者下载bridge-utils-1.4.tar.gz进行安装,步骤如下:

 

编译安装bridge-utils

 

(1)进入到/usr/src目录下,下载bridge-utils-1.4.tar.gz :

 

#cd /usr/src

 

#wget http://launchpad.net/bridgeutils/

 

main/1.4/+download/bridge-utils-

 

1.4.tar.gz

 

(2)解压缩:

 

#tar zxvf bridge-utils-1.4.tar.gz

 

进入bridge-utils-1.4目录:

 

#cd bridge-utils-1.4

 

(3)编译安装:

 

#autoconf

 

生成configure文件:

 

#./configure

 

#make

 

#make install

 

编译安装完成。最后将命令brctl复制到/sbin下:

 

#cp/usr/local/sbin/brctl/sbin

 

=========================================================================

 

下面是参考的一片文章:

 

有五台主机。其中一台主机装有linux ,安装了网桥模块,而且有四块物理网卡,分别连接同一网段的其他主机。我们希望其成为一

 

个网桥,为其他四台主机(IP分别为192.168.1.2 ,192.168.1.3,192.168.1.4,192.168.1.5) 之间转发数据包。同时,为了方便管

 

理,希望网桥能够有一个IP(192.168.1.1),那样管理员就可以在192.168.1.0/24网段内的主机上telnet到网桥,对其进行配置,

 

实现远程管理。

 

前一节中提到,网桥在同一个逻辑网段转发数据包。针对上面的拓扑,这个逻辑网段就是192.168.1.0/24网段。我们为这个逻辑网段一个名称,br0。首先需要配置这样一个逻辑网段。

 

# brctl addbr br0 (建立一个逻辑网段,名称为br0)

 

实际上,我们可以把逻辑网段192.168.1.0/24看作使一个VLAN ,而br0则是这个VLAN的名称。

 

建立一个逻辑网段之后,我们还需要为这个网段分配特定的端口。在Linux中,一个端口实际上就是一个物理网卡。而每个物理网卡

 

的名称则分别为eth0,eth1,eth2,eth3。我们需要把每个网卡一一和br0这个网段联系起来,作为br0中的一个端口。

 

# brctl addif br0 eth0 (让eth0成为br0的一个端口)

 

# brctl addif br0 eth1 (让eth1成为br0的一个端口)

 

# brctl addif br0 eth0 (让eth2成为br0的一个端口)

 

# brctl addif br0 eth3 (让eth3成为br0的一个端口)

 

网桥的每个物理网卡作为一个端口,运行于混杂模式,而且是在链路层工作,所以就不需要IP了。

 

# ifconfig eth0 0.0.0.0

 

# ifconfig eth1 0.0.0.0

 

# ifconfig eth2 0.0.0.0

 

# ifconfig eth3 0.0.0.0

 

然后给br0的虚拟网卡配置IP:192.168.1.1。那样就能远程管理网桥。

 

# ifconfig br0 192.168.1.1

 

给br0配置了IP之后,网桥就能够工作了。192.168.1.0/24网段内的主机都可以telnet到网桥上对其进行配置。

 

以上配置的是一个逻辑网段,实际上Linux网桥也能配置成多个逻辑网段(相当于交换机中划分多个VLAN)。

 

另外一篇有助理解的文章:http://www.2cto.com/os/201202/118320.html

 

=====================================================================

 

[root@xenserver ~]# brctl--help

 

Usage: brctl [commands]

 

commands:

 

addbr <bridge> add bridge

 

delbr <bridge> delete bridge

 

addif <bridge> <device> add interface to bridge

 

delif <bridge> <device> delete interface from bridge

 

setageing <bridge> <time> set ageing time

 

setbridgeprio <bridge> <prio> set bridge priority

 

setfd <bridge> <time> set bridge forward delay

 

sethello <bridge> <time> set hello time

 

setmaxage <bridge> <time> set max message age

 

setpathcost <bridge> <port>< cost> set path cost

 

setportprio <bridge> <port>< prio> set port priority

 

show show a list of bridges

 

showmacs <bridge> show a list of mac addrs

 

showstp <bridge> show bridge stp info

 

stp <bridge> {on|off} turn stp on/off

 

====================================================================

 

addbr bridge的名称 #添加bridge;

 

delbr bridge的名称 #删除bridge;

 

addif bridge的名称device的名称#添加接口到bridge;

 

delif bridge的名称device的名称#从bridge中删除接口

 

setageing bridge的名称时间 #设置老化时间,即生存周期

 

setbridgeprio bridge的名称 优先级#设置bridge的优先级

 

setfd bridge的名称时间 #设置bridge转发延迟时间

 

sethello bridge的名称时间 #设置hello时间

 

setmaxage bridge的名称时间 #设置消息的最大生命周期

 

setpathcost bridge的名称 端口 权重#设置路径的权值

 

setportprio bridge的名称 端口 优先级#设置端口的优先级

 

show #显示bridge列表

 

showmacs bridge的名称 #显示MAC地址

 

showstp bridge的名称 #显示bridge的stp信息

 

stp bridge的名称{on|off} #开/关stp

 

设置linux让网桥运行 配置网桥

 

1.我们需要让linux知道网桥,首先告诉它,我们想要一个虚拟的以太网桥接口:(这将在主机bridge上执行,不清楚的看看测试场景)

 

root@bridge:~> brctl addbr br0

 

2.其次,我们不需要STP(生成树协议)等。因为我们只有一个路由器,是绝对不可能形成一个环的。我们可以关闭这个功能。(这样也可以减少网络环境的数据包污染):

 

root@bridge:~> brctl stp br0 off

 

3.经过这些准备工作后,我们终于可以做一些立竿见影的事了。我们添加两个(或更多)以太网物理接口,意思是:我们将他们附加到刚生成的逻辑(虚拟)网桥接口br0上。

 

root@bridge:~> brctl addif br0 eth0

 

root@bridge:~> brctl addif br0 eth1

 

4.现在,原来我们的两个以太网物理接口变成了网桥上的两个逻辑端口。那两个物理接口过去存在,未来也不会消失。要不信的话,去看看好了。.现在他们成了逻辑网桥设备的一部分了,所以不再需要IP地址。下面我们将这些IP地址释放掉

 

root@bridge:~> ifconfig eth0 down

 

root@bridge:~> ifconfig eth1 down

 

root@bridge:~> ifconfig eth0 0.0.0.0 up

 

root@bridge:~> ifconfig eth1 0.0.0.0 up

 

好了!我们现在有了一个任何IP地址都没有的box w/o了。好了,这下如果你想通过TP配置你的防火墙或路由器的话,你就只能通过本地的控制端口了。你不会告诉我你的机器上连串行端口都没有吧?

 

注:上面红色部分其实是可选的,在试验中,我发现,就算不把原有的网卡地址释放掉,网桥也能工作!但是,为了更规范,或者说

 

为了避免有什幺莫名其妙的问题,最好还是按要求做,执行这四步吧!

 

5.最后,启用网桥root@bridge:~> ifconfig br0 up

 

可选: 我们给这个新的桥接口分配一个IP地址

 

root@bridge:~> ifconfig br0 10.0.3.129

 

或者把最后这两步合成一步:

 

root@bridge:~> ifconfig br0 10.0.3.129 up

 

就是多一个up!

 

这下我们做完了 。

 

关闭网桥命令

 

brctl delif ena eth1;

 

brctl delif ena eth0;

 

ifconfig ena down;

 

brctl delbr ena;

 

摘自 zjl_1026_2001的专栏

 

什么是网桥

 

网桥是一种在链路层实现中继,对帧进行转发的技术,根据MAC分区块,可隔离碰撞,将网络的多个网段在数据链路层连接起来的网络设备。

 

Linux网桥配置命令:brctl

 

在Linux中配置网络一般使用 brctl 命令,使用此命令首先要安装:bridge-utils软件包。

 

[inbi@debian~]#apt-get install bridge-utils

 

[inbi@debian~]#modprobe bridge

 

[inbi@debian~]#echo "1">/proc/sys/net/ipv4/ip_forward

 

#安装bridge-utils软件包,并加载bridge模块和开启内核转发。

 

[inbi@debian~]#brctl

 

#直接输入brctl命令将显示帮助信息!

 

Usage: brctl [commands]

 

commands:

 

addbr

<bridge>

add bridge

delbr

<bridge>

delete bridge

addif

<bridge> <device>

add interface to bridge

delif

<bridge> <device>

delete interface from bridge

setageing

<bridge> <time>

set ageing time

setbridgeprio

<bridge> <prio>

set bridge priority

setfd

<bridge> <time>

set bridge forward delay

sethello

<bridge> <time>

set hello time

setmaxage

<bridge> <time>

set max message age

setpathcost

<bridge> <port> <cost>

set path cost

setportprio

<bridge> <port> <prio>

set port priority

show

 

show a list of bridges

showmacs

<bridge>

show a list of mac addrs

showstp

<bridge>

show bridge stp info

stp

<bridge> {on|off}

turn stp on/off

 

增加网桥

 

[inbi@debian~]#brctl addbr br0

 

#增加一个网桥

 

[inbi@debian~]#ifconfig eth0 0.0.0.0 promisc

 

[inbi@debian~]#ifconfig eth1 0.0.0.0 promisc

 

[inbi@debian~]#brctl addif br0 eth0 eth1

 

#将两块已有的网卡添加到网桥,此时这两个网卡工作于混杂模式,所以不需要IP了,因为网桥是工作在链路层的。

 

[inbi@debian~]#brctl show

 

#查看已有网桥

 

你也可以为 br0 设置一个IP,已访问这台机器。

 

[inbi@debian~]#ifconfig br0 10.10.1.1 netmask 255.255.0.0 up

 

删除网桥

 

[inbi@debian~]#brctl delif br0 eth0 eth1

 

#增加网桥中的接口

 

[inbi@debian~]#brctl delbr br0

 

#删除网桥

 

关闭生成树

 

[inbi@debian~]#brctl stp br0 off

 

#关闭生成树协议,减少数据包污染,因为我这里只有一个路由器哦!

 

配置桥开机激活

 

[inbi@debian~]#echo "modprobe bridge">>/etc/rc.local

 

#开机加载 bridge 模块,或者echo "bridge">>/etc/modules

 

[inbi@debian~]#cp /etc/network/interfaces /etc/network/interfaces.default

 

#备份下,方便以后使用啊!

 

[inbi@debian~]#vim /etc/network/interfaces

 

auto lo eth0 eth1 br0

 

iface lo inet loopback

 

iface br0 inet static

 

address 10.10.10.1

 

netmask 255.255.0.0

 

gateway 10.10.10.254

 

pre-up ip link set eth0 promisc on

 

pre-up ip link set eth1 promisc on

 

pre-up echo "1">/proc/sys/net/ipv4/ip_forward

 

bridge_ports eth0 eth1

 

#配置eth0 eth1 br0开机启动,eth0,eth1未设置IP信息,在启动br0网卡时,开启了eth0,eth1的混杂模式,并桥接了它们。

 

3 Linux下的虚拟Bridge实现 

Linux下的Bridge也是一种虚拟设备,这多少和vlan有点相似,它依赖于一个或多个从设备。与VLAN不同的是,它不是虚拟出和从设备同一层次的镜像设备,而是虚拟出一个高一层次的设备,并把从设备虚拟化为端口port,且同时处理各个从设备的数据收发及转发,再加上netfilter框架的一些东西,使得它的实现相比vlan复杂得多。

1.Bridge的功能框图

    它是Linux下虚拟出来bridge设备,Linux下可用brctl命令创建br设备,如下

brctl addbr brname

然后添加port,并进行相应配置,就可以使用了

brctl addif brname eth0

brctl addif brname eth1

ifconfig brname IP up

ifconfig eth0 0.0.0.0 up

ifconfig eth1 0.0.0.0 up

可见br设备是建立在从设备之上的(这些从设备可以是实际设备,也可以是vlan设备等),并且可以为br准备一个IP(br设备的MAC地址是它所有从设备中最小的MAC地址),这样该主机就可以通过这个br设备与网络中的其它主机通信了(详见发送功能框图)。

另外它的从设备被虚拟化为端口port,它们的IP及MAC都不再可用,且它们被设置为接收任何包,最终由bridge设备来决定数据包的去向:接收到本机、转发、丢弃(详见接收功能框图)。

 

        发送功能                                        接收功能

 

    简单的数据结构框图如下所示。

2.Bridge设备的创建

    和Vlan一样,bridge也被当成一个module加载进内核,它的module_init()函数和vlan差不多,进行一些namespace的注册,特殊的是它还注册了一个netfilter_ops,在内核全局的HOOK函数表中增加了7个函数,其中5个的pf=Bridge,另两个的pf分别为INET、INET6,它们主要用于bridge中的netfilter操作(后面会细讲)。

    最后,也是我们这最关心的是,它注册了一个ioctl函数br_ioctl_deviceless_stub(),该ioctl函数和vlan的一样,都会作为sock_ioctl()的特殊情况被调用。映射到应用层,它应该是对某个socket插口进行ioctl操作,详见brctl源码。该ioctl函数中最主要的就是br_add_bridge(net,buf),用于创建bridge设备,如下图所示:

该函数调用netdev_alloc(),申请net_device(),并分配私有空间net_bridge()结构,指明初始化函数为br_dev_setup(),最后register_netdev()把该设备组册进内核,可见bridge设备和一般的设备差不多。

主要看其中br_dev_setup(),首先初始化设备的type、flags为bridge,然后最关键的是设置其dev->netdev_ops = br_netdev_ops,即内核为bridge设备准备好了一套通用的驱动函数,这个直接关系到bridge的工作方法,后面再细讲。然后初始化私有空间net_bridge()结构,设置bridge的本地设备及从设备的list(当然这是还没有从设备加进来),然后设置了桥的group_address,即上一节所说的特殊的MAC地址,最后还初始化了timer相关的。

 

    这时bridge还不完整,还需添加port从设备,由命令brctl addif brname portdev完成,但要注意,虽然还是brctl命令,但此时的操作对象是已经存在的bridge设备,映射到内核中就是br_netdev_ops->ioctl()中的br_add_if()(它是br设备的ioctl操作,和之前那个sock_ioctl的分支不是一个层次上的)。至于怎么从应用层直接操作底层的net_device设备的,可以参见brctl源码,以后再看吧,先看看这里的br_add_if(),如下图:

    首先判断dev从设备必须不是loopback,不是bridge,不是其他bridge的port,且要是ethernet设备,才能继续;然后根据br、dev选择一个index号,并分配一个新的net_bridge_port结构,初始化之,并将它加入bridge的port_list中;最后br的一些物理参数,其MAC地址为所有从设备中MAC最小的(由上一节知,从设备被设置成全接收模式,其IP和MAC都没有了),且其MTU也为所有从设备中最小的。

    上面设置br的相关参数,下面还要设置从设备,首先使dev->master=br_dev(实际上就是构成上一节数据结构中的索引关系);然后设置dev->prive_flags加上IFF_BRIDGE_PORT,这样它就不能再作为其他br的从设备了;最后也是最关键的,设置dev->rx_handler为br_handler_frame(),为数据接收作准备。

3.Bridge设备的发送流程

    前面也讲过了,Linux下的bridge设备,对下层而言是一个桥设备,进行数据的转发(实际上对下也有接收能力,下一节讲)。而对上层而言,它就像普通的ethernet设备一样,有自己的IP和MAC地址,那么上层当然可以把它加入路由系统,并利用它发送数据啦,并且很容易想到,它的发射函数最终肯定是利用某个从设备的驱动去完成实际的发送的,这个和VLAN是相通的。具体看代码:

    上层根据目的IP地址,路由选择了该br_dev设备发送,并且由ARP缓存中得到了对应的目的MAC,填写在了skb中,然后启动了发送流程dev_queue_xmit(skb)。因为此时的skb->dev为br_dev,无queue,直接去调用br设备的发送函数,该函数就是br_netdev_ops中定义的br_dev_xmit(skb,br_dev)。

    该函数首先根据目的MAC地址,确定是广播还是单播,这里仅讨论单播时,根据DMAC在net_bridge的fdb_hash中找到相应的net_bridge_fdb_entry项,并索引到对应的端口net_bridge_port。最后利用该端口的从设备来发送数据,注意,这里是直接调用dev->ops->ndo_start_xmit(skb,dev)的,一放面这里的dev已经是从设备了,另一方面,这里没有像VLAN中那样重定位skb->dev,并重启发送流程dev_queue_xmit(),是因为一个从设备只能作为一个bridge的port,没有其它身份,不存在竞争问题。

4.Bridge设备的接收流程

    和VLAN一样,实际接收由硬件设备完成,最终通过netif_receive_skb(skb)函数提交给上层,而在该函数中会处理vlan、bridge这类特殊设备。与LVAN的仅是把skb设备重定位以实现对上层透明的要求不同,Bridge接受过程复杂得多,因此专门注册了一个函数来处理,即前面提到的rx_handler(),它被注册在port设备的net_device结构中(这算是port设备失去自身IP、MAC的一个补偿吧J),如下图所示。只有作为bridge的从设备才会注册rx_handler(),并在这里执行,处理桥接,普通的设备不会执行到这里。

    这里两种典型的返回值是RX_CONSUMED、RX_PASS,后者表示处理了一下回来,继续之前的流程,实际上就是对应的接收功能框图中的第一种情况;前者表示该skb已不再属于这个从设备了,而是被提交给了br设备,所以本次netif_receive_skb()就不用管啦,直接goto out,这里还要再分两种情况,一是转发的,这时br就真的充当了桥的角色,二是由br提交给上层的,这时br充当的是一个以太网设备,如前面所述。

要处理这么多情况,代码需设计得很巧妙,这里的rx_handler被设置为br_handle_frame(**pskb),看具体代码:

    功能框架清楚了,代码流程就清楚了,就不细看了,有几个注意的地方:一是bridge的端口处于FORWARD或LEARNING状态没多大区别,只是FORWARD要多执行一个二层防火墙,所有用了一个中间没有break的switch结构;二是要时刻记着,bridge本身除了有转发端口外,自己也是一个设备,广播(多播)时也要发一份给自己,且是以br_dev的身份递交上层;三是所有的HOOK函数都会比较复杂,因为内核的netfilter框架建立在网络层,而bridge在链路层就转发了,相当于跳过了netfilter,所以在这些hook中都会去调用INET域的hook函数。

5.小结

    通过对Bridge和vlan的学习,了解了网络栈底层的工作方式,发送这个主动过程相对简单,而接收过程则相对复杂,用到BH模型,NAPI等。

    Vlan和bridge功能有所不同,但相似处很多,更重要的是:它们都对上层透明,所以不会牵扯到协议域的问题。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值