问题
最近上线了定时发送电子邮件的功能,是基于 SpringBoot 的。但是经常出现卡死的情况,导致客户频繁咨询客服同志,使用 jstack
查看,发现线程每次都是卡在了如下的地方,很显然 SocketInputStream.socketRead0()
是在等待邮件服务器的响应,但由于某些原因一直没有响应,就会一直卡着。又因为是单线程的模型(xxl-job 的单机串行),所以一个卡着就会导致后面的都无法发送。
"Thread-10" #54 prio=10 os_prio=0 tid=0x00005627730c6800 nid=0x3c runnable [0x00007f6c02614000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
- locked <0x00000000f870d0c0> (a java.lang.Object)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:933)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
- locked <0x00000000f870d230> (a sun.security.ssl.AppInputStream)
at com.sun.mail.util.TraceInputStream.read(TraceInputStream.java:102)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
- locked <0x00000000f870d668> (a java.io.BufferedInputStream)
at com.sun.mail.util.LineInputStream.readLine(LineInputStream.java:100)
at com.sun.mail.smtp.SMTPTransport.readServerResponse(SMTPTransport.java:2455)
at com.sun.mail.smtp.SMTPTransport.issueSendCommand(SMTPTransport.java:2352)
at com.sun.mail.smtp.SMTPTransport.finishData(SMTPTransport.java:2095)
at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:1301)
- locked <0x00000000f8708b48> (a com.sun.mail.smtp.SMTPSSLTransport)
at org.springframework.mail.javamail.JavaMailSenderImpl.doSend(JavaMailSenderImpl.java:465)
at org.springframework.mail.javamail.JavaMailSenderImpl.send(JavaMailSenderImpl.java:361)
at org.springframework.mail.javamail.JavaMailSenderImpl.send(JavaMailSenderImpl.java:356)
at com.laso.report.pdf.service.MailServiceImpl.sendAttachmentsMail(MailServiceImpl.java:82)
at com.laso.report.pdf.service.MailServiceImpl.lambda$retrySendAttachmentsMail$1(MailServiceImpl.java:98)
at com.laso.report.pdf.service.MailServiceImpl$$Lambda$814/1400108637.call(Unknown Source)
at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
at com.github.rholder.retry.Retryer.call(Retryer.java:160)
at com.laso.report.pdf.service.MailServiceImpl.execute(MailServiceImpl.java:112)
at com.laso.report.pdf.service.MailServiceImpl.retrySendAttachmentsMail(MailServiceImpl.java:98)
at com.laso.report.pdf.service.MailServiceImpl.lambda$sendMailTask$2(MailServiceImpl.java:135)
at com.laso.report.pdf.service.MailServiceImpl$$Lambda$812/361139532.accept(Unknown Source)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at com.laso.report.pdf.service.MailServiceImpl.sendMailTask(MailServiceImpl.java:134)
at com.laso.report.pdf.task.PdfTask.sendEmailTask(PdfTask.java:35)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.xxl.job.core.handler.impl.MethodJobHandler.execute(MethodJobHandler.java:29)
at com.xxl.job.core.thread.JobThread.run(JobThread.java:152)
初始配置,注意此处的协议是 smtps,不是 smtp:
spring.mail.protocol=smtps
spring.mail.port=465
spring.mail.host=xx
spring.mail.username=xxx
spring.mail.password=xxx
spring.mail.default-encoding=UTF-8
...
添加超时
最直接的解决方案就是添加超时,我们使用的 SpringBoot 版本是 2.2.5.RELEASE,所以找到了对应的官方文档 boot-features-email,直接粘贴上了:
spring.mail.properties.mail.smtp.connectiontimeout=5000
spring.mail.properties.mail.smtp.timeout=10000
spring.mail.properties.mail.smtp.writetimeout=5000
运行了一段时间又发生卡死的问题了,使用 jstack
查看,还是同样的堆栈信息,还是卡在了同样的地方。第一反应就是配置没生效,为此百度了半天网上也没有其他的配置方式,期间也找了 java mail 的文档如下,问题依然没有解决。
源码中找问题
根据 jstack
的堆栈 debug,JavaMailSenderImpl
是 Spring 发送邮件的核心类,doSend()
是核心方法,在其中会调用 connectTransport()
方法,问题就出在这个方法中,重点在于 12、14 行,下面分开说明
protected Transport connectTransport() throws MessagingException {
String username = getUsername();
String password = getPassword();
if ("".equals(username)) { // probably from a placeholder
username = null;
if ("".equals(password)) { // in conjunction with "" username, this means no password to use
password = null;
}
}
// getSession() 中已经有超时相关的属性了,如下图
// transport 就是下面实例化出的 SMTPSSLTransport
Transport transport = getTransport(getSession());
// 使用协议名(smtps)作为前缀获取对应的属性
transport.connect(getHost(), getPort(), username, password);
return transport;
}
JavaMailSenderImpl .getTransport()
<init>:174, SMTPTransport (com.sun.mail.smtp)
<init>:37, SMTPSSLTransport (com.sun.mail.smtp)
newInstance0:-1, NativeConstructorAccessorImpl (sun.reflect)
newInstance:62, NativeConstructorAccessorImpl (sun.reflect)
newInstance:45, DelegatingConstructorAccessorImpl (sun.reflect)
newInstance:423, Constructor (java.lang.reflect)
getService:852, Session (javax.mail)
getTransport:772, Session (javax.mail)
getTransport:713, Session (javax.mail)
getTransport:693, Session (javax.mail)
getTransport:538, JavaMailSenderImpl (org.springframework.mail.javamail)
getTransport:713, Session (javax.mail)
根据协议(此处是 smtps)获取到了对应的 Provider
,Provider
的两个重要字段 protocol = smtps
、className = "com.sun.mail.smtp.SMTPSSLTransport"
public Transport getTransport(URLName url) throws NoSuchProviderException {
String protocol = url.getProtocol();
Provider p = getProvider(protocol);
return getTransport(p, url);
}
getService:835, Session (javax.mail)
得到了 com.sun.mail.smtp.SMTPSSLTransport
的 Class 对象:
Class<?> serviceClass = null;
serviceClass = Class.forName(provider.getClassName());
getService:852, Session (javax.mail)
获取对应参数的构造方法,实例化,com.sun.mail.smtp.SMTPSSLTransport
:
Class<?>[] c = {javax.mail.Session.class, javax.mail.URLName.class};
Constructor<?> cons = serviceClass.getConstructor(c);
Object[] o = {this, url};
service = cons.newInstance(o);
实例化 SMTPSSLTransport
在 SMTPSSLTransport
的构造方法中调用的父类 SMTPTransport
的构造方法,注意第三个参数是 smtps,然后将 this.name = "smtps"
public class SMTPSSLTransport extends SMTPTransport {
public SMTPSSLTransport(Session session, URLName urlname) {
super(session, urlname, "smtps", true);
}
}
// 省略了不重要的代码
public class SMTPSSLTransport {
protected SMTPTransport(Session session, URLName urlname, String name, boolean isSSL) {
...
this.name = name;
...
}
}
Transport.connect()
getSocket:125, SocketFetcher (com.sun.mail.util)
openServer:2160, SMTPTransport (com.sun.mail.smtp)
protocolConnect:722, SMTPTransport (com.sun.mail.smtp)
connect:342, Service (javax.mail)
openServer:2160, SMTPTransport (com.sun.mail.smtp)
// Session 中的属性,包含超时相关的
Properties props = session.getProperties();
// 上面已经说明过了 SMTPTransport.name = smtps
// 第四个参数为 "mail." + name,也就是 “mail.smtps”
serverSocket = SocketFetcher.getSocket(host, port, props, "mail." + name, isSSL);
SocketFetcher.getSocket()
mail.smtps 的形参名为 prefix
,在 getSocket()
方法中会使用 prefix + ".connectiontimeout"
、prefix + ".timeout"
获取属性,同时会调用 createSocket()
方法,其中会使用 prefix + ".writetimeout"
获取属性:
public static Socket getSocket(String host, int port, Properties props, String prefix, boolean useSSL) {
...
int cto = PropUtil.getIntProperty(props, prefix + ".connectiontimeout", -1);
Socket socket = null;
...
int to = PropUtil.getIntProperty(props, prefix + ".timeout", -1);
...
socket = createSocket(localaddr, localport, host, sfPort, cto, to, props, prefix, sf, useSSL);
...
}
private static Socket createSocket(...) {
...
int writeTimeout = PropUtil.getIntProperty(props, prefix + ".writetimeout", -1);
if (writeTimeout != -1) { // wrap original
if (logger.isLoggable(Level.FINEST))
logger.finest("set socket write timeout " + writeTimeout);
socket = new WriteTimeoutSocket(socket, writeTimeout);
}
...
}
最终解决办法
上面已经看出来 Spring 是使用 mail. + 协议名 作为前缀获取属性的,那么只需让配置的前缀与协议名一致就可以了,即将 smtp 改成 smtps:
spring.mail.properties.mail.smtps.connectiontimeout=5000
spring.mail.properties.mail.smtps.timeout=10000
spring.mail.properties.mail.smtps.writetimeout=5000