Libcurl多线程crash问题

 2014-01-15 wcdj

1 问题背景

后台系统有一个单线程的http接口，为了提高并发处理能力，开启多个线程并发在跑，修改后接口的响应确实得到提高，但是server每3分钟出现一次crash。原因是系统使用的是curl-7.21.1（August 11 2010）的库，此版本并非线程安全。遂替换了最新的curl-7.34.0（December 12 2013）库，悲催的是时隔几小时还是会偶现crash，于是再仔细阅读官方文档。

 

官方对最新版本libcurl的Multi-threading Issues解释如下^[1]：

The first basic rule is that you mustneversimultaneously share a libcurl handle (be it easy or multi or whatever) betweenmultiple threads. Only use one handle in one thread at any time. You can passthe handles around among threads, but you must never use a single handle frommore than one thread at any given time.

 

libcurl is completely thread safe, except for two issues:signals and SSL/TLS handlers. Signals are used for timingout name resolves (during DNS lookup) - when built without c-ares support andnot on Windows.

When using multiple threads youshould set the CURLOPT_NOSIGNAL option to 1 for all handles.Everything will or might work fine except that timeouts are not honored duringthe DNS lookup - which you can work around by building libcurl with c-aressupport. c-ares is a library that provides asynchronous name resolves.On some platforms, libcurl simply will not function properlymulti-threaded unless this option is set.

Also, note that CURLOPT_DNS_USE_GLOBAL_CACHE is notthread-safe.

此接口并没有使用到SSL/TLS，但会不会是用到了signals导致的crash呢？官方建议在多线程场景下应该设置CURLOPT_NOSIGNAL选项，因为在解析DNS出现超时的时候将会发生“糟糕”的情况。官方也给出了解决方法，可以使用c-ares^[2]的libcurl版本实现异步域名解析来预防这种“糟糕”的情况，但是最后一句还是告诫我们：在多线程场景下，若不设置CURLOPT_NOSIGNAL选项，可能会有“意外”的情况发生。通过官方这段描述，可以大致猜测到是没有设置这个选项造成的crash。下面是官方对此选项的说明^[3]：