故障表现
-
连接认证成功后就断开: ssh closed by remote host
-
详细的信息
-
服务端开debug模式看到
-
同网络的ssh连接正常
从 10.10.10.10 连接到 10.10.10.11 时 ssh 10.10.10.11 -p 4568 正常,可以正常连接
网络关系说明
服务是跑在vmware虚机(10.10.10.11) 上的,网络使用的是NAT模式,ssh配置的端口是4568,通过vmware配置了端口映射,映射到物理机(192.168.0.100)的4568端口上边,当物理机同网段的其他机器(192.168.0.101)访问到该端口时就出现了这个问题 ssh 192.168.0.100 -p 4568。而同样的虚机 10.10.10.10 就没有问题配置的端口是4567, ssh 192.168.0.100 -p 4567 就是正常的。而且 从 10.10.10.10 到 10.10.10.11 的ssh连接是正常的
问题排查
-
配置文件区别: 把10.10.10.10上的/etc/ssh/sshd_config配置文件拷贝到10.10.10.100上时问题同样存在
% sudo grep -Ev '^#|^$' /etc/ssh/sshd_config Port 4567 HostKey /etc/ssh/ssh_host_rsa_key HostKey /etc/ssh/ssh_host_ecdsa_key HostKey /etc/ssh/ssh_host_ed25519_key SyslogFacility AUTHPRIV AuthorizedKeysFile .ssh/authorized_keys PasswordAuthentication yes ChallengeResponseAuthentication no GSSAPIAuthentication yes GSSAPICleanupCredentials no UsePAM yes X11Forwarding yes UseDNS no AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE AcceptEnv XMODIFIERS Subsystem sftp /usr/libexec/openssh/sftp-server
-
软件差异:
10.10.10.10上使用的是: openssh-7.4p1-16.el7.x86_64
10.10.10.11上使用的是: openssh-8.1p1-2
到这里可以看到是版本的问题,手动安装同样的openssh-7.4p1 后问题解决,可以正常登陆,看似问题解决了,但是以后如果都不能使用新版本了吗?
-
手动配置端口映射:
取消vmware端口映射
在windows宿主机cmd上边执行:
netsh interface portproxy add v4tov4 listenport=4568 connectaddress=10.10.10.11 connectport=4568
这样子连接正常了,ssh能够正常工作
为什么会造成ssh连接问题
-
对比两个版本的sshd_config配置文件说明 一项一项的排查找到一个选项
命令: $man 5 sshd_config
参数: IPQoS
旧版本 openssh-7.4p1:
Specifies the IPv4 type-of-service or DSCP class for the connection. Accepted values are af11, af12, af13, af21, af22, af23, af31, af32, af33, af41, af42, af43, cs0, cs1, cs2, cs3, cs4, cs5, cs6, cs7, ef, lowdelay, throughput, reliability, or a numeric value. This option may take one or two arguments, separated by whitespace. If one argument is specified, it is used as the packet class unconditionally. If two values are specified, the first is automatically selected for interactive sessions and the second for non-interactive sessions. The default is lowdelay for interactive sessions and throughput for non-inter‐active sessions.
默认值是: lowdelay throughput
新版本 openssh-8.1p1-2:
Specifies the IPv4 type-of-service or DSCP class for the connection. Accepted values are af11, af12, af13, af21, af22, af23, af31, af32, af33, af41, af42, af43, cs0, cs1, cs2, cs3, cs4, cs5, cs6, cs7, ef, lowdelay, throughput, reliability, a numeric value, or none to use the operating system default. This op‐tion may take one or two arguments, separated by whitespace. If one argument is specified, it is used as the packet class unconditionally. If two values are specified, the first is automatically selected for interactive sessions and the second for non-interactive sessions. The default is af21 (Low-Latency Data) for interactive sessions and cs1 (Lower Effort) for non-interactive sessions
默认值是: af21 cs1
把新版本sshd_config文件中添加配置
IPQoS lowdelay throughput
之后同样是使用vmware 做端口映射,同样可以正常工作,看来就是这个配置项的问题了
IPQoS 配置是做什么用
翻了翻openssh的源代码
const struct toskeywords {
const char *keyword;
int val;
} *t, toskeywords[] = {
{ "af11", IPTOS_DSCP_AF11 },
{ "af12", IPTOS_DSCP_AF12 },
{ "af13", IPTOS_DSCP_AF13 },
{ "af21", IPTOS_DSCP_AF21 },
{ "af22", IPTOS_DSCP_AF22 },
{ "af23", IPTOS_DSCP_AF23 },
{ "af31", IPTOS_DSCP_AF31 },
{ "af32", IPTOS_DSCP_AF32 },
{ "af33", IPTOS_DSCP_AF33 },
{ "af41", IPTOS_DSCP_AF41 },
{ "af42", IPTOS_DSCP_AF42 },
{ "af43", IPTOS_DSCP_AF43 },
{ "critical", IPTOS_PREC_CRITIC_ECP },
{ "cs0", IPTOS_DSCP_CS0 },
{ "cs1", IPTOS_DSCP_CS1 },
{ "cs2", IPTOS_DSCP_CS2 },
{ "cs3", IPTOS_DSCP_CS3 },
{ "cs4", IPTOS_DSCP_CS4 },
{ "cs5", IPTOS_DSCP_CS5 },
{ "cs6", IPTOS_DSCP_CS6 },
{ "cs7", IPTOS_DSCP_CS7 },
{ "ef", IPTOS_DSCP_EF },
{ "inetcontrol", IPTOS_PREC_INTERNETCONTROL },
{ "lowdelay", IPTOS_LOWDELAY },
{ "netcontrol", IPTOS_PREC_NETCONTROL },
{ "reliability", IPTOS_RELIABILITY },
{ "throughput", IPTOS_THROUGHPUT },
{ NULL, -1 },
};
#ifndef IPTOS_LOWDELAY
# define IPTOS_LOWDELAY 0x10
# define IPTOS_THROUGHPUT 0x08
# define IPTOS_RELIABILITY 0x04
# define IPTOS_LOWCOST 0x02
# define IPTOS_MINCOST IPTOS_LOWCOST
#endif /* IPTOS_LOWDELAY */
/*
* Definitions for DiffServ Codepoints as per RFC2474
*/
#ifndef IPTOS_DSCP_AF11
# define IPTOS_DSCP_AF11 0x28
# define IPTOS_DSCP_AF12 0x30
# define IPTOS_DSCP_AF13 0x38
# define IPTOS_DSCP_AF21 0x48
# define IPTOS_DSCP_AF22 0x50
# define IPTOS_DSCP_AF23 0x58
# define IPTOS_DSCP_AF31 0x68
# define IPTOS_DSCP_AF32 0x70
# define IPTOS_DSCP_AF33 0x78
# define IPTOS_DSCP_AF41 0x88
# define IPTOS_DSCP_AF42 0x90
# define IPTOS_DSCP_AF43 0x98
# define IPTOS_DSCP_EF 0xb8
#endif /* IPTOS_DSCP_AF11 */
#ifndef IPTOS_DSCP_CS0
# define IPTOS_DSCP_CS0 0x00
# define IPTOS_DSCP_CS1 0x20
# define IPTOS_DSCP_CS2 0x40
# define IPTOS_DSCP_CS3 0x60
# define IPTOS_DSCP_CS4 0x80
# define IPTOS_DSCP_CS5 0xa0
# define IPTOS_DSCP_CS6 0xc0
# define IPTOS_DSCP_CS7 0xe0
#endif /* IPTOS_DSCP_CS0 */
#ifndef IPTOS_DSCP_EF
# define IPTOS_DSCP_EF 0xb8
#endif /* IPTOS_DSCP_EF */
ssh_packet_set_tos(struct ssh *ssh, int tos)
{
#ifndef IP_TOS_IS_BROKEN
if (!ssh_packet_connection_is_on_socket(ssh) || tos == INT_MAX)
return;
switch (ssh_packet_connection_af(ssh)) {
# ifdef IP_TOS
case AF_INET:
debug3("%s: set IP_TOS 0x%02x", __func__, tos);
if (setsockopt(ssh->state->connection_in,
IPPROTO_IP, IP_TOS, &tos, sizeof(tos)) < 0)
error("setsockopt IP_TOS %d: %.100s:",
tos, strerror(errno));
break;
# endif /* IP_TOS */
# ifdef IPV6_TCLASS
case AF_INET6:
debug3("%s: set IPV6_TCLASS 0x%02x", __func__, tos);
if (setsockopt(ssh->state->connection_in,
IPPROTO_IPV6, IPV6_TCLASS, &tos, sizeof(tos)) < 0)
error("setsockopt IPV6_TCLASS %d: %.100s:",
tos, strerror(errno));
break;
# endif /* IPV6_TCLASS */
}
#endif /* IP_TOS_IS_BROKEN */
}
核心就是
setsockopt(ssh->state->connection_in, IPPROTO_IP, IP_TOS, &tos, sizeof(tos))
查看文档
PROLOG
This manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ (consult the corresponding Linux manual page for
details of Linux behavior), or the interface may not be implemented on Linux.
NAME
setsockopt — set the socket options
SYNOPSIS
#include <sys/socket.h>
int setsockopt(int socket, int level, int option_name,
const void *option_value, socklen_t option_len);
DESCRIPTION
The setsockopt() function shall set the option specified by the option_name argument, at the protocol level specified by the level argument, to the value pointed to
by the option_value argument for the socket associated with the file descriptor specified by the socket argument.
The level argument specifies the protocol level at which the option resides. To set options at the socket level, specify the level argument as SOL_SOCKET. To set
options at other levels, supply the appropriate level identifier for the protocol controlling the option. For example, to indicate that an option is interpreted by
the TCP (Transport Control Protocol), set level to IPPROTO_TCP as defined in the <netinet/in.h> header.
The option_name argument specifies a single option to set. It can be one of the socket-level options defined in <sys_socket.h> and described in Section 2.10.16, Use
of Options. If option_name is equal to SO_RCVTIMEO or SO_SNDTIMEO and the implementation supports setting the option, it is unspecified whether the struct timeval
pointed to by option_value is stored as provided by this function or is rounded up to align with the resolution of the clock being used. If setsockopt() is called
with option_name equal to SO_ACCEPTCONN, SO_ERROR, or SO_TYPE, the behavior is unspecified.
该函数就是要更改ssh连接的 IP_TOS 值
在IP头中,有一Type-of-Service字段,该字段描述了IP包的优先级和QoS选项,使用IP_TOS可以来设定该字段的值,以区分不同服务的优先级:
IPTOS_LOWDELAY 用来为交互式通信最小化延迟时间, IPTOS_THROUGHPUT 用来优化吞吐量, IPTOS_RELIABILITY 用来作可靠性优化, IPTOS_MINCOST 应该被用作"填充数据",对于这些数据,低速传输是无关紧要的.至多只能声明这些 TOS 值中的一个.其它的都是无效的,应当被清除.缺省时,Linux首先发送 IPTOS_LOWDELAY 数据报, 但是确切的做法要看配置的排队规则而定. 一些高优先级的层次可能会要求一个有效的用户标识 0 或者 CAP_NET_ADMIN 能力.
这个选项在voip实践中可以用于提高rtp数据包的优先级。就是用setsockopt在某个socket上设置 ip_tos
在IPv4的报文头中,TOS字段是1字节,如下图所示。根据RFC1122的定义,IP优先级(IPPrecedence)使用最高3比特(第0~2比特)。
+++++++++++++++++++++++++++++++++
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
+++++++++++++++++++++++++++++++++3比特可以定义8个等级。
8个优先级的定义如下:
111 -Network Control 网络控制
110 -Internetwork Control 网间控制
101 -Critic 关键
100 - FlashOverride 疾速
011 -Flash 闪速
010 -Immediate 快速
001 -Priority 优先
000 -Routine 普通优先级6和7一般保留给网络控制数据使用,比如路由。
优先级5推荐给语音数据使用。
优先级4由视频会议和视频流使用。
优先级3给语音控制数据使用。
优先级1和2给数据业务使用。
优先级0为缺省标记值。
在标记数据时,既可以使用数值,也可以使用名称(英文名称)。DSCP
DSCP由RFC2474定义,它重新命名了IPv4报头中TOS使用的那1字节和IPv6报头中数据类(TrafficClass)那1字节,新的名字称为DS字段(Differentiated ServicesField)。该字段的作用没有变,仍然被QoS工具用来标记数据。不同的是IPv4使用3比特,而DSCP使用6比特,最低2比特不用。
RFC2474 定义最高3比特为级别/类别选择代码(ClassSelector Codepoints,CS),其意义和IPv4报头中IP优先级的定义是相同的,CS0 ~CS7的级别相等于IP优先级0 ~7。但它并没有定义第3到第5比特的具体含义以及使用规则。DSCP使用6比特,可以定义64个优先级(0-63)。AF
保证转发(Assured Forwarding,AF)由RFC2597对CS1~CS4进行进一步定义。它使用第3和第4比特做丢弃优先级标志。01-低丢弃优先级;10-中丢弃优先级;11-高丢弃优先级。这样,在同一类数据中,又根据被丢弃的可能性划分出3档。下表列出了AF服务等级及其对应的DSCP值:
CS1 CS2 CS3 CS4
Lowdrop AF11 AF21 AF31 AF41
001010 010010 011010 100010Mediumdrop AF12 AF22 AF32 AF42
001100 010100 011100 100100Highdrop AF13 AF23 AF33 AF43
001110 010110 011110 100110AF的定义为数据分类提供了方便,比如,运营商可以向用户提供4中服务协约(SLA):白金,金,银,铜,并为每一种服务的数据分配一定的带宽。当然,不同服务的收费标准也是不同的。
EF
无阻碍转发(Expedited Forwarding,EF)由RFC2598定义,DSCP值为46(101110)。EF服务适用于低丢包率,低延迟,低抖动及保证带宽的业务,如VOIP。
其他
DSCP = 000000 尽力转发服务等级 (EF);
CS = 6 网间控制(Internetwork Control),DSCP= 48 (110000)
CS = 7 网内控制 (Intranetwork Control),DSCP= 56 (111000)
在配置命令中,既可以使用十进制数值,也可以使用二进制数值,还可以使用名称。例如,28, 011100, AF32三个写法意义相同。
但是为什么使用vmware做端口映射时 TOS 会阻断,而直接做映射就正常,这个问题我查看了vmware的官方文档里面也没有进行说明,也找不到相关的资料只能先这样子了,估计是有可能vmware做端口映射时做了一个处理