Summary of typical IPMP Configurations:
Steps to Follow
Contents:
1. Production and test interfaces in the same IP subnet
1.1 With default router
1.2 Without default router
1.3 With dedicated hosts acting as test targets with "host-routes"
1.4 Configuration example for 1.1, 1.2 and 1.3
2. Production and test interfaces in different IP subnets but the same
physical network
2.1 With default router in production subnet and test subnet
2.2 With default router in production subnet but without default router in test subnet
2.3 With dedicated hosts acting as test targets with "host-routes"
2.4 Configuration example for 2.1, 2.2 and 2.3
2.5 Configuration example for 2.2 if you use IPMP on the test subnet
A. Start script. for adding static "host routes" on Solaris 8 and 9
B. Adding persistent "host routes" via 'route -p' option on Solaris 10
C. Summary
Note: This document does not cover the basic knowledge of IP Multipathing. If you would need it, please search in Oracle Documentation.
All IPMP patches mentioned below should be installed.
OS
Release Sparc Intel Comments
-----------------------------------------------------------------------------------
7 or earlier N/S N/S IPMP was introduced with Solaris[TM] 8 U2
-----------------------------------------------------------------------------------
8 108528 108529 kernel update patch (in.mpathd, ip, icmp, ifconfig)
116965 116966 ip/arp/tcp/udp/tun patch
109900 109901 /etc/init.d/network and /sbin/ifparse patch
109902 109903 in.ndpd patch
110378 110396 mipagent patch Mobile IP
116991 116992 rcm_daemon patch
-----------------------------------------------------------------------------------
9 112233 112234 Kernel Patch
114344 119435 arp, dlcosmk, ip, and ipgpc Patch
" 114348 /usr/sbin/in.routed patch
112912 N/P libinetcfg Patch
113073 114733 ufs and fsck patch
-----------------------------------------------------------------------------------
10 118833-36 118855-36 kernel patch
120011-14 120012-14 kernel patch (min. required for IPMP patch 141532)
141532 141533 in.mpathd patch
10 U4 (8/07) 127111 127112 kernel patch
10 U5 (5/08) 137111 137112 kernel patch
10 U6 (10/08) 138888 138889 kernel patch
10 U7 (5/09) 141414 141415 kernel patch
10 U8 (10/09) 142900 142901 kernel patch
10 U9 (9/10) 144488 144489 kernel patch
-----------------------------------------------------------------------------------
Key
N/S - Product not supported on this OS
N/P - No patch applicable for this OS
Legend:
IPMP system with IPMP group
T target host
p network/host partof IP address of interface in production subnet
t network/host part of IP address of interface in test subnet
--- data link subnet (i.e broadcast domain) with one IP subnet
=== data link subnet (i.e broadcast domain) with two (or more) IP subnets
Good to know:
The operation of IP Multipathing (in.mpathd) depends on the routing configuration. Therefore in.mpathd always refers to the routing-table (IRE-cache) to distinguish which test partner(s) are going to be used. Test partners are required for deciding if the interface is working properly.
in.mpathd by default chooses the default router as single test-target for probing. If no default router exists for the test-interface ip address, arbitrary hosts on the link are detected by sending out "all hosts" multicast packets (224.0.0.1) on the wire to detect its test-partners. An "all routers" multicasts (224.0.0.2) will never be sent! The first five hosts that are responding to the echo packets are chosen as targets for probing. In such a non-default router environment, the in.mpathd always tries to find five probe targets via an "all hosts" multicast.
The in.mpathd detects failures and repairs by sending out 'icmp-echo' probes (like pinging) from all interfaces that are part of the IPMP group. If there are no replies to five consecutive probes, the interface is considered to have failed and performs a failover of the network access to another interface in the IPMP group. The probing rate depends on the failure detection time which is defined in /etc/default/mpathd. By default, failure detection time is 10 seconds. Thus the five probes will be sent within the failure detection time.
1. Production and test interfaces in the same IP subnet
1.1 With default router
+----------------+
| default router |
+-------o--------+
| p=t:172.20.20.1
|
|
----------+-+------------------------+-------------------------
| | p=t:172.20.20/24
| |
p:172.20.20.10 | |
t:172.20.20.210 | | t:172.20.20.220
+---o-o---+
| IPMP |
+---------+
IPMP uses the default router as a single probe target if it is configured to the same IP subnet. Each test interface of the IPMP group will send ICMP requests to the default router only. To get the configuration, IPMP looks to the routing table and is independent of the /etc/defaultrouter file. There will be NO "all hosts" multicast (224.0.0.1) sent out.
Advantages:
- Easiest configuration for IPMP.
Disadvantages:
- If the default router is down then IPMP failover does not work anymore. The in.mpathd does NOT send out multicasts to get other probe targets, therefore all interfaces in the IPMP group get the state "failed". You can ignore this bug/feature when you have a default router which is 100% online! Please look to RFE 4431511 and 4489960 for further information.
- If you have a lot of IPMP groups, the default router has to reply to many ICMP requests. Take care about the default router. Do not overload the default router.
- The default router has to reliably answer ICMP echo requests. (e.g. firewalls sometimes do not)
1.2 Without default router
+-----+ +-----+
| T 1 | ... | T 5 |
+--o--+ +--o--+
| p=t:172.20.20.110 | p=t:172.20.20.150
| |
| |
-----------+-+--------------+---------------------+-------
| | p=t:172.20.20.0/24
| |
p:172.20.20.10 | |
t:172.20.20.210 | | t:172.20.20.220
+---o-o---+
| IPMP |
+---------+
IPMP dynamically determines five arbitrary hosts on the link via an "all hosts" multicast (224.0.0.1). At least, you will need one probe target to get IPMP working but be aware that one probe target is not reliable enough. As long as there are less than five targets available, the in.mpathd daemon will continuously sent out "all hosts" multicasts to get a complete list of five probe targets.
Advantages:
- easiest configuration for IPMP in a subnet without a default router.
- very reliable due to the five targets.
Disadvantages:
- a subnet without an default router is very rare.
1.3 With dedicated hosts acting as test targets with "host-routes"
+----------------+ +-----+ +-----+
| default router | | T 1 | ... | T 5 |
+-------o--------= +--o--+ +--o--+
| | p=t:172.20.20.110 | p=t:172.20.20.150
| | |
| | |
-----------+-+--------------+---------------------+-------
| | p=t:172.20.20.0/24
| |
p:172.20.20.10 | |
t:172.20.20.210 | | t:172.20.20.220
+---o-o---+
| IPMP |
+---------+
Solaris 8 and 9 - Some "host routes" will be defined with a start script. in /etc/rc2.d/S70staticroutes (refer to appendix A). When IPMP refers to the routing table it will choose the first five defined "host routes" as probe targets. This is due to the fact that normally the "host routes" are before the default router in the routing table. If you have less than five "host routes", the default router (when available) will be used as a probe target too.
Examples:
- Configuration with host1, host2 ... hostN (with N=5 or N>5), defaultrouter :
==> The first five hosts (host1 ... host5) will be defined as target, not the default router - Configuration with less than 5 hosts : for instance, host1, host2, defaultrouter :
==> The three systems (host1, host2, defaultrouter) will be defined as target.
The in.mpathd continuously tries to get five probe targets from the routing table but it will NOT send "all hosts" multicasts!
Solaris 10 - Use the "route -p" option to add persistent routes to the kernel (refer to appendix B). No additional script. will be necessary.
Advantages:
- The default router is not a single point of failure anymore because you still have some other "host routes" as probe targets.
- Higher availability.
Disadvantages:
- More administrative work to do.
- Due to static configuration you should regularly check that the probe targets are available.
1.4 Configuration examples for 1.1, 1.2 and 1.3
/etc/hosts
127.0.0.1 localhost
172.20.20.10 host10 loghost
172.20.20.210 host10-test-qfe0
172.20.20.220 host10-test-qfe4
/etc/hostname.qfe0
host10 netmask + broadcast + group ipmp0 up \
addif host10-test-qfe0 netmask + broadcast + deprecated -failover up
/etc/hostname.qfe4
host10-test-qfe4 netmask + broadcast + deprecated -failover group ipmp0 up
ifconfig output:
inet 172.20.20.10 netmask ffffff00 broadcast 172.20.20.255
groupname ipmp0
ether 8:0:20:e8:88:dc
qfe0:1: flags=1000843 mtu 1500 index 3
inet 172.20.20.210 netmask ffffff00 broadcast 172.20.20.255
qfe4: flags=9040843 mtu 1500 index 4
inet 172.20.20.220 netmask ffffff00 broadcast 172.20.20.255
groupname ipmp0
ether 8:0:20:e8:89:34
Note: The example describes the setup of an active-active IPMP configuration which is best practices. But you can also configure an active-standby configuration. You simple have to add the "standby" flag to the /etc/hostname.qfe4 file. More Details available in Technical Instruction Document: 1012450.1.
2. Production and test interfaces in different IP subnets
If you don't have enough additional IP addresses available, you can configure the IPMP test interfaces to a different IP subnet than your production network (e.g. 192.168., 10. ..). But you have to make sure that there are always enough test partners in that test network to respond to the ICMP probes. In such a configuration in.mpathd will use the test IP addresses only as source address for the outgoing probe packets.
Note: The in.mpathd only checks the test network. If there is no responding probe target in that IP subnet the IPMP group will fail even though the production subnet is still available.
2.1 With default router in production subnet and test subnet
+----------------+
| default router |
+-------o--------+
| p:172.20.20.1
| t:192.168.1.1
|
----------+-+------------------------+-------------------------
| | p:172.20.20.0/24
| | t:192.168.1.0/24
p:172.20.20.10 | |
t:192.168.1.210 | | t:192.168.1.220
+---o-o---+
| IPMP |
+---------+
IPMP uses the default router as a single probe target if it is configured to the same IP subnet. Each test interface of the IPMP group will send ICMP requests to the default router only. To get the configuration, IPMP looks to the routing table and is independent of the /etc/defaultrouter file. There will be NO "all hosts" multicast (224.0.0.1) sent out.
Advantages:
- No additional production IP addresses required for the test interfaces
Disadvantages:
- The interface of the default router has to reside in both the production AND test subnet.
- Exceptional configuration of default router.
- All other disadvantages already mentioned in section 1.1
2.2 With default router in production subnet net but without default router in test subnet
+----------------+ +-----+ +-----+
| default router | | T 1 | ... | T 5 |
+-------o--------= +--o--+ +--o--+
p:172.20.20.1 | | p:172.20.20.110 | p:172.20.20.150
| | t:192.168.1.110 | t:192.168.1.150
| | |
-----------+-+--------------+---------------------+-------
| | p:172.20.20.0/24
| | t:192.168.1.0/24
p:172.20.20.10 | |
t:192.168.1.210 | | t:192.168.1.220
+---o-o---+
| IPMP |
+---------+
IPMP dynamically determines five arbitrary hosts on the link via an "all hosts" multicast (224.0.0.1). At least, you will need one probe target to get IPMP working but be aware that one probe target is not reliable enough. As long as there are less than five targets available, the in.mpathd daemon will continuously sent out "all hosts" multicasts to get a complete list of five probe targets.
Advantages:
- easiest IPMP configuration if you don't have enough additional IP addresses available in the production subnet.
- very reliable due to the five targets.
Disadvantages:
- the probe targets must be available before you can setup the IPMP host.
- more administrative work because you have to setup the probe targets as an additional interface in the test subnet.
Recommendation: Setup an additional logical network interface on target host.
(e.g. add a new interface to /etc/hostname.qfe0 with the 'addif' option)
If your test partners are on systems which run also IPMP you must add an logical network interface NOT flagged deprecated and NOT flagged nofailover. An address associated with this interface will then be used as source address by responding properly to "all hosts" multicast packets which are used for the automatic IPMP test partner detection. Be aware that Solaris 8 does NOT require the additional logical network interface in the test subnet on the target host. So, in case of an upgrade from Solaris 8 to Solaris[TM] 9 or higher you have to change your configuration. - Please refer to 2.5 for a detailed example.
2.3 with dedicated hosts acting as test targets with "host-routes"
+----------------+ +-----+ +-----+
| default router | | T 1 | ... | T 5 |
+-------o--------= +--o--+ +--o--+
p:172.20.20.1 | | p:172.20.20.110 | p:172.20.20.150
| | t:192.168.1.110 | t:192.168.1.150
| | |
-----------+-+--------------+---------------------+-------
| | p:172.20.20.0/24
| | t:192.168.1.0/24
p:172.20.20.10 | |
t:192.168.1.210 | | t:192.168.1.220
+---o-o---+
| IPMP |
+---------+
Solaris 8 and 9 - Some "host routes" will be defined with a start script. in /etc/rc2.d/S70staticroutes (refer to appendix A). When IPMP refers to the routing table it will choose the first five defined "host routes" as probe targets. This is due to the fact that normally the "host routes" are before the default router in the routing table. If you have less than five "host routes", the default router (when available in the test subnet) will be used as a probe target too.
Examples:
Please refer to the examples of section 1.3.
Solaris 10 - Use the "route -p" option to add persistent routes to the kernel (refer to appendix B). No additional script. will be necessary.
Advantages:
- test interfaces don't need IP addresses of the production subnet
- all others already mentioned in section 1.3
- all others already mentioned in section 1.3
- all others already mentioned in section 2.2
2.4 Configuration examples for 2.1, 2.2 and 2.3
/etc/hosts
127.0.0.1 localhost
172.20.20.10 host10 loghost
192.168.1.210 host10-test-qfe0
192.168.1.220 host10-test-qfe4
/etc/hostname.qfe0
host10 netmask + broadcast + group ipmp0 up \
addif host10-test-qfe0 netmask + broadcast + deprecated -failover up
/etc/hostname.qfe4
host10-test-qfe4 netmask + broadcast + deprecated -failover group ipmp0 up
ifconfig output:
inet 172.20.20.10 netmask ffffff00 broadcast 172.20.20.255
groupname ipmp0
ether 8:0:20:e8:88:dc
qfe0:1: flags=1000843 mtu 1500 index 3
inet 192.168.1.210 netmask ffffff00 broadcast 192.168.1.255
qfe4: flags=9040843 mtu 1500 index 4
inet 192.168.1.220 netmask ffffff00 broadcast 192.168.1.255
groupname ipmp0
ether 8:0:20:e8:89:34
2.5 Configuration example for 2.2 if you use IPMP on the test subnet
/etc/hosts
127.0.0.1 localhost
#
# IPMP active IP address:
#
172.20.20.10 host10 loghost
#
# IPMP test interfaces
#
192.168.1.210 host10-test-ce0
192.168.1.220 host10-test-ce1
#
# additional active interface in the 'test subnet'
# to be able to respond to probe packets from other
# IPMP setups (See 2.2). Will also failover.
#
192.168.1.211 host10-prod-ce0
/etc/hostname.ce0
host10 netmask + broadcast + group ipmp0 up \
addif host10-prod-ce0 netmask + broadcast + up \
addif host10-test-ce0 netmask + broadcast + deprecated -failover up
/etc/hostname.ce1
host10-test-ce1 netmask + broadcast + deprecated -failover group ipmp0 up
ifconfig output:
inet 172.20.20.10 netmask ffffff00 broadcast 172.20.20.255
groupname ipmp0
ether 8:0:20:e2:da:d5
ce0:1: flags=1000843 mtu 1500 index 2
inet 192.168.1.211 netmask ffffff00 broadcast 192.168.1.255
ce0:2: flags=9040843 mtu 1500 index 2
inet 192.168.1.210 netmask ffffff00 broadcast 192.168.1.255
ce1: flags=9040843 mtu 1500 index 3
inet 192.168.1.220 netmask ffffff00 broadcast 192.168.1.255
groupname ipmp0
ether 8:0:20:e2:da:d6
A. Start script. for adding static "host routes" on Solaris 8 and 9
----------- Begin of start script. /etc/init.d/ipmp.targets --------------
#!/sbin/sh
# /etc/rc2.d/S70ipmp.targets /etc/init.d/ipmp.targets
# Copyright (c) 2005 by Sun Microsystems, Inc.
# All rights reserved.
#
#ident "@(#)ipmp.targets 1.0.0
#
# Edit the following IPMP test TARGETS to suit your needs.
# To install:
# 1) cp ipmp.targets /etc/init.d
# 2) perform. edits on the script. as required (e.g: add TARGETS)
# 3) chmod 744 /etc/init.d/ipmp.targets
# 4) chown root:sys /etc/init.d/ipmp.targets
# 5) ln /etc/init.d/ipmp.targets /etc/rc2.d/S70ipmp.targets
#
TARGETS="192.168.85.117 192.168.85.127 192.168.85.137"
case "$1" in
'start')
/usr/bin/echo "Adding static routes for use as IPMP targets"
for target in $TARGETS; do
/usr/sbin/route add -host $target $target
done
;;
'stop')
/usr/bin/echo "Removing static routes for use as IPMP targets"
for target in $TARGETS; do
/usr/sbin/route delete -host $target $target
done
;;
esac
----------- End of start script. /etc/init.d/ipmp.targets --------------
See also:
System Administration Guide: IP Services > Configuring IPMP Groups > Configuring Target Systems
B. Adding persistent "host routes" via 'route -p' option on Solaris 10
Example:
In this example, the active interface on the system is 10.8.118.104
The test host targets are 5 systems that are external to this physical system.
The IP address used for the default router is the same IP address that was in the /etc/defaultrouter file.
# rm /etc/defaultrouterNote: If you define 5 host routes then the default router will NOT be used as a probe target, if you define less than 5 host routes the default router will be used as an additional probe target.
# route -p add -host 10.8.118.1 10.8.118.1 -static
# route -p add -host 10.8.118.3 10.8.118.3 -static
# route -p add -host 10.8.118.40 10.8.118.40 -static
# route -p add -host 10.8.118.41 10.8.118.41 -static
# route -p add -host 10.8.118.193 10.8.118.193 -static
# route -p add default 10.8.118.248 -static
C. Summary
It's not easy to give a general recommendation because it depends on the network infrastructure which the customers have. Therefore you have to discuss the various possibilities with your customer. Maybe the most used setups are 1.1 and 2.2.Product
Solaris 8 Operating System
Solaris 9 Operating System
Solaris 10 Operating System
来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/339291/viewspace-691495/，如需转载，请注明出处，否则将追究法律责任。
转载于:http://blog.itpub.net/339291/viewspace-691495/
3898
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
