ipvsadm(LVS)的Weight ActiveConn InActConn

The output of ipsvadm lists connections, either as

  • ActiveConn - in ESTABLISHED state
  • InActConn - any other state

With LVS-NAT, the director sees all the packets between the client and the realserver, so always knows the state of tcp connections and the listing from ipvsadm is accurate. However for LVS-DR, LVS-Tun, the director does not see the packets from the realserver to the client. Termination of the tcp connection occurs by one of the ends sending a FIN (see W. Richard Stevens, TCP/IP Illustrated Vol 1, ch 18, 1994, pub Addison Wesley) followed by reply ACK from the other end. Then the other end sends its FIN, followed by an ACK from the first machine. If the realserver initiates termination of the connection, the director will only be able to infer that this has happened from seeing the ACK from the client. In either case the director has to infer that the connection has closed from partial information and uses its own table of timeouts to declare that the connection has terminated. Thus the count in the InActConn column for LVS-DR, LVS-Tun is inferred rather than real.

Entries in the ActiveConn column come from

  • service with an established connection. Examples of services which hold connections in the ESTABLISHED state for long enough to see with ipvsadm are telnet and ftp (port 21).

Entries in the InActConn column come from

  • Normal operation

    • Services like http (in non-persistent i.e. HTTP /1.0 mode) or ftp-data(port 20) which close the connections as soon as the hit/data (html page, or gif etc) has been retrieved (<1sec). You're unlikely to see anything in the ActiveConn column with these LVS'ed services. You'll see an entry in the InActConn column untill the connection times out. If you're getting 1000connections/sec and it takes 60secs for the connection to time out (the normal timeout), then you'll have 60,000 InActConns. This number of InActConn is quite normal. If you are running an e-commerce site with 300secs of persistence, you'll have 300,000 InActConn entries. Each entry takes 128bytes (300,000 entries is about 40M of memory, make sure you have enough RAM for your application). The number of ActiveConn might be very small.
  • Pathological Conditions (i.e. your LVS is not setup properly)

    • identd delayed connections: The 3 way handshake to establish a connection takes only 3 exchanges of packets (i.e. it's quick on any normal network) and you won't be quick enough with ipvsadm to see the connection in the states before it becomes ESTABLISHED. However if the service on the realserver is under authd/identd, you'll see an InActConn entry during the delay period.

    • Incorrect routing (usually the wrong default gw for the realservers):

      In this case the 3 way handshake will never complete, the connection will hang, and there'll be an entry in the InActConn column.

Usually the number of InActConn will be larger or very much larger than the number of ActiveConn.

Here's a LVS-DR LVS, setup for ftp, telnet and http, after telnetting from the client (the client command line is at the telnet prompt).

director:# ipvsadm
IP Virtual Server version 0.2.8 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP lvs2.mack.net:www rr
-> RS2.mack.net:www Route 1 0 0
-> RS1.mack.net:www Route 1 0 0
TCP lvs2.mack.net:0 rr persistent 360
-> RS1.mack.net:0 Route 1 0 0
TCP lvs2.mack.net:telnet rr
-> RS2.mack.net:telnet Route 1 1 0
-> RS1.mack.net:telnet Route 1 0 0

showing the ESTABLISHED telnet connection (here to realserver RS2).

Here's the output of netstat -an | grep (appropriate IP) for the client and the realserver, showing that the connection is in the ESTABLISHED state.

client:# netstat -an | grep VIP
tcp 0 0 client:1229 VIP:23 ESTABLISHED

realserver:# netstat -an | grep CIP
tcp 0 0 VIP:23 client:1229 ESTABLISHED

Here's immediately after the client logs out from the telnet session.

director# ipvsadm
IP Virtual Server version 0.2.8 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP lvs2.mack.net:www rr
-> RS2.mack.net:www Route 1 0 0
-> RS1.mack.net:www Route 1 0 0
TCP lvs2.mack.net:0 rr persistent 360
-> RS1.mack.net:0 Route 1 0 0
TCP lvs2.mack.net:telnet rr
-> RS2.mack.net:telnet Route 1 0 0
-> RS1.mack.net:telnet Route 1 0 0

client:# netstat -an | grep VIP
#ie nothing, the client has closed the connection

#the realserver has closed the session in response
#to the client's request to close out the session.
#The telnet server has entered the TIME_WAIT state.
realserver:/home/ftp/pub# netstat -an | grep 254
tcp 0 0 VIP:23 CIP:1236 TIME_WAIT

#a minute later, the entry for the connection at the realserver is gone.

Here's the output after ftp'ing from the client and logging in, but before running any commands (like `dir` or `get filename`).

director:# ipvsadm
IP Virtual Server version 0.2.8 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP lvs2.mack.net:www rr
-> RS2.mack.net:www Route 1 0 0
-> RS1.mack.net:www Route 1 0 0
TCP lvs2.mack.net:0 rr persistent 360
-> RS1.mack.net:0 Route 1 1 1
TCP lvs2.mack.net:telnet rr
-> RS2.mack.net:telnet Route 1 0 0
-> RS1.mack.net:telnet Route 1 0 0

client:# netstat -an | grep VIP
tcp 0 0 CIP:1230 VIP:21 TIME_WAIT
tcp 0 0 CIP:1233 VIP:21 ESTABLISHED

realserver:# netstat -an | grep 254
tcp 0 0 VIP:21 CIP:1233 ESTABLISHED

The client opens 2 connections to the ftpd and leaves one open (the ftp prompt). The other connection, used to transfer the user/passwd information, is closed down after the login. The entry in the ipvsadm table corresponding to the TIME_WAIT state at the realserver is listed as InActConn. If nothing else is done at the client's ftp prompt, the connection will expire in 900 secs. Here's the realserver during this 900 secs.

realserver:# netstat -an | grep CIP
tcp 0 0 VIP:21 CIP:1233 ESTABLISHED
realserver:# netstat -an | grep CIP
tcp 0 57 VIP:21 CIP:1233 FIN_WAIT1
realserver:# netstat -an | grep CIP
#ie nothing, the connection has dropped

#if you then go to the client, you'll find it has timed out.
ftp> dir
421 Timeout (900 seconds): closing control connection.

http 1.0 connections are closed immediately after retrieving the URL (i.e. you won't see any ActiveConn in the ipvsadm table immediately after the URL has been fetched). Here's the outputs after retreiving a webpage from the LVS.

director:# ipvsadm
IP Virtual Server version 0.2.8 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP lvs2.mack.net:www rr
-> RS2.mack.net:www Route 1 0 1
-> RS1.mack.net:www Route 1 0 0

client:~# netstat -an | grep VIP

RS2:/home/ftp/pub# netstat -an | grep CIP
tcp 0 0 VIP:80 CIP:1238 TIME_WAIT
阅读更多
想对作者说点什么?

博主推荐

换一批

没有更多推荐了,返回首页