WebRTC线程介绍

一. 线程模型

        WebRTC 有三类线程：信令线程，网络线程，工作线程。

        信令线程一般工作在 PeerConnection 层，它负责与应用层交互，例如 createOffer，createAnswer 等操作，并通知工作线程和网络线程相应的信号事件。

        网络线程工作在网络传输层，它负责网络收发包，从网络接收包数据并发送给工作线程，工作线程也会把要发送的包数据给网络线程。

        工作线程工作在媒体引擎层（engine），包含视频采集线程，视频渲染线程，视频编码线程，视频解码线程等。

二. 线程创建与启动

        使用 webrtc::CreatePeerConnectionFactory 创建 PeerConnectionFactoryInterface 时有 network_thread，worker_thread，signaling_thread 参数可以让我们指定使用的 Thread 对象，当传递 nullptr 时表示希望函数内部帮助我们创建并启动对应的线程。

        如果传入的 network_thread 为空则使用 rtc::Thread::CreateWithSocketServer() 创建 Thread 对象，如果传入的 worker_thread 为空则使用 rtc::Thread::Create() 创建 Thread 对象，如果传入的 signaling_thread 为空则使用 rtc::Thread::Current() 对其进行赋值。

         Thread 构造函数逻辑主要是将该 Thread 对象添加到 ThreadManager 中进行管理。

        线程创建与启动逻辑如下，对于 Windows 使用 CreateThread 创建线程，对于使用 POSIX 方式的系统则使用 pthread_create 创建线程，线程创建后都运行 PreRun 函数。

bool Thread::Start() {
  RTC_DCHECK(!IsRunning());

  if (IsRunning())
    return false;

  Restart();  // reset IsQuitting() if the thread is being restarted

  // Make sure that ThreadManager is created on the main thread before
  // we start a new thread.
  ThreadManager::Instance();

  owned_ = true;

#if defined(WEBRTC_WIN)
  thread_ = CreateThread(nullptr, 0, PreRun, this, 0, &thread_id_);
  if (!thread_) {
    return false;
  }
#elif defined(WEBRTC_POSIX)
  pthread_attr_t attr;
  pthread_attr_init(&attr);

  int error_code = pthread_create(&thread_, &attr, PreRun, this);
  if (0 != error_code) {
    RTC_LOG(LS_ERROR) << "Unable to create pthread, error " << error_code;
    thread_ = 0;
    return false;
  }
  RTC_DCHECK(thread_);
#endif
  return true;
}

         Thread::PreRun 主要的逻辑为 thread->Run()，该函数调用 ProcessMessages 从 Thread 对象里的 messages_ 取出消息并进行处理，不同的线程接收不同的消息处理，消息处理逻辑由 MessageHandler::OnMessage 定义。

三. 线程管理

         在线程创建与启动我们可以看到，WebRTC 是通过 Thread 对象来管理某一个具体线程，通过 ThreadManager 对象管理多个线程。

        Thread 类重要的方法和成员如下。

class RTC_LOCKABLE RTC_EXPORT Thread : public webrtc::TaskQueueBase {

  // Get() will process I/O until:
  //  1) A message is available (returns true)
  //  2) cmsWait seconds have elapsed (returns false)
  //  3) Stop() is called (returns false)
  virtual bool Get(Message* pmsg,
                   int cmsWait = kForever,
                   bool process_io = true);
  virtual bool Peek(Message* pmsg, int cmsWait = 0);
  // |time_sensitive| is deprecated and should always be false.
  virtual void Post(const Location& posted_from,
                    MessageHandler* phandler,
                    uint32_t id = 0,
                    MessageData* pdata = nullptr,
                    bool time_sensitive = false);
  virtual void PostDelayed(const Location& posted_from,
                           int delay_ms,
                           MessageHandler* phandler,
                           uint32_t id = 0,
                           MessageData* pdata = nullptr);
  virtual void PostAt(const Location& posted_from,
                      int64_t run_at_ms,
                      MessageHandler* phandler,
                      uint32_t id = 0,
                      MessageData* pdata = nullptr);
  virtual void Clear(MessageHandler* phandler,
                     uint32_t id = MQID_ANY,
                     MessageList* removed = nullptr);
  virtual void Dispatch(Message* pmsg);

  // Amount of time until the next message can be retrieved
  virtual int GetDelay();

  bool fPeekKeep_;
  Message msgPeek_;
  MessageList messages_ RTC_GUARDED_BY(crit_);
  PriorityQueue delayed_messages_ RTC_GUARDED_BY(crit_);
  uint32_t delayed_next_num_ RTC_GUARDED_BY(crit_);
  CriticalSection crit_;
  bool fInitialized_;
  bool fDestroyed_;

  volatile int stop_;

  // The SocketServer might not be owned by Thread.
  SocketServer* const ss_;
  // Used if SocketServer ownership lies with |this|.
  std::unique_ptr<SocketServer> own_ss_;

  std::string name_;
}

        ThreadManager 重要的方法与成员如下。ThreadManager 实现为单例模式，通过 Instance() 获取唯一实例。

class RTC_EXPORT ThreadManager {
 public:
  static const int kForever = -1;

  // Singleton, constructor and destructor are private.
  static ThreadManager* Instance();

  static void Add(Thread* message_queue);
  static void Remove(Thread* message_queue);
  static void Clear(MessageHandler* handler);

  // TODO(nisse): Delete alias, as soon as downstream code is updated.
  static void ProcessAllMessageQueues() { ProcessAllMessageQueuesForTesting(); }

  // For testing purposes, for use with a simulated clock.
  // Ensures that all message queues have processed delayed messages
  // up until the current point in time.
  static void ProcessAllMessageQueuesForTesting();

  Thread* CurrentThread();
  void SetCurrentThread(Thread* thread);
  // Allows changing the current thread, this is intended for tests where we
  // want to simulate multiple threads running on a single physical thread.
  void ChangeCurrentThreadForTest(Thread* thread);

  // Returns a thread object with its thread_ ivar set
  // to whatever the OS uses to represent the thread.
  // If there already *is* a Thread object corresponding to this thread,
  // this method will return that.  Otherwise it creates a new Thread
  // object whose wrapped() method will return true, and whose
  // handle will, on Win32, be opened with only synchronization privileges -
  // if you need more privilegs, rather than changing this method, please
  // write additional code to adjust the privileges, or call a different
  // factory method of your own devising, because this one gets used in
  // unexpected contexts (like inside browser plugins) and it would be a
  // shame to break it.  It is also conceivable on Win32 that we won't even
  // be able to get synchronization privileges, in which case the result
  // will have a null handle.
  Thread* WrapCurrentThread();
  void UnwrapCurrentThread();

  bool IsMainThread();

#if RTC_DCHECK_IS_ON
  // Registers that a Send operation is to be performed between |source| and
  // |target|, while checking that this does not cause a send cycle that could
  // potentially cause a deadlock.
  void RegisterSendAndCheckForCycles(Thread* source, Thread* target);
#endif

 private:
  ThreadManager();
  ~ThreadManager();

  void SetCurrentThreadInternal(Thread* thread);
  void AddInternal(Thread* message_queue);
  void RemoveInternal(Thread* message_queue);
  void ClearInternal(MessageHandler* handler);
  void ProcessAllMessageQueuesInternal();
#if RTC_DCHECK_IS_ON
  void RemoveFromSendGraph(Thread* thread) RTC_EXCLUSIVE_LOCKS_REQUIRED(crit_);
#endif

  // This list contains all live Threads.
  std::vector<Thread*> message_queues_ RTC_GUARDED_BY(crit_);

  // Methods that don't modify the list of message queues may be called in a
  // re-entrant fashion. "processing_" keeps track of the depth of re-entrant
  // calls.
  CriticalSection crit_;
  size_t processing_ RTC_GUARDED_BY(crit_) = 0;
#if RTC_DCHECK_IS_ON
  // Represents all thread seand actions by storing all send targets per thread.
  // This is used by RegisterSendAndCheckForCycles. This graph has no cycles
  // since we will trigger a CHECK failure if a cycle is introduced.
  std::map<Thread*, std::set<Thread*>> send_graph_ RTC_GUARDED_BY(crit_);
#endif

#if defined(WEBRTC_POSIX)
  pthread_key_t key_;
#endif

#if defined(WEBRTC_WIN)
  const DWORD key_;
#endif

  // The thread to potentially autowrap.
  const PlatformThreadRef main_thread_ref_;

  RTC_DISALLOW_COPY_AND_ASSIGN(ThreadManager);
};

四. 线程事件处理

         以信令线程与工作线程/网络线程的事件通知处理逻辑为例，信令线程触发事件通知公共对象，工作线程/网络线程等待该公共对象事件然后执行相应的处理逻辑。

        WebRTC 有两个事件处理类，NullSocketServer 用于处理不包含 socket 事件的事件，PhysicalSocketServer 用于处理包含了 socket 事件的事件，网络线程创建时使用了 PhysicalSocketServer，信令线程和工作线程则使用 NullSocketServer。

owned_network_thread_ = rtc::Thread::CreateWithSocketServer();

owned_worker_thread_ = rtc::Thread::Create();

signaling_thread_ = rtc::Thread::Current();

NullSocketServer

        WebRTC 处理普通事件的类是 NullSocketServer，该类包含一个 Event。

class RTC_EXPORT NullSocketServer : public SocketServer {
 public:
  NullSocketServer();
  ~NullSocketServer() override;

  bool Wait(int cms, bool process_io) override;
  void WakeUp() override;

  Socket* CreateSocket(int family, int type) override;
  AsyncSocket* CreateAsyncSocket(int family, int type) override;

 private:
  Event event_;
};

        Windows 平台下事件涉及的 API：CreateEvent 创建事件句柄，WaitForSingleObject 等待事件，SetEvent 触发事件，ResetEvent 重置事件。

Event::Event(bool manual_reset, bool initially_signaled) {
  event_handle_ = ::CreateEvent(nullptr,  // Security attributes.
                                manual_reset, initially_signaled,
                                nullptr);  // Name.
  RTC_CHECK(event_handle_);
}

Event::~Event() {
  CloseHandle(event_handle_);
}

void Event::Set() {
  SetEvent(event_handle_);
}

void Event::Reset() {
  ResetEvent(event_handle_);
}

bool Event::Wait(const int give_up_after_ms, int /*warn_after_ms*/) {
  ScopedYieldPolicy::YieldExecution();
  const DWORD ms = give_up_after_ms == kForever ? INFINITE : give_up_after_ms;
  return (WaitForSingleObject(event_handle_, ms) == WAIT_OBJECT_0);
}

PhysicalSocketServer

        WebRTC 处理包含 socket 事件的事件的类是 PhysicalSocketServer，该类包含一个 WSAEvent，这种事件可以与 socket 进行绑定。

class RTC_EXPORT PhysicalSocketServer : public SocketServer {
 public:
  PhysicalSocketServer();
  ~PhysicalSocketServer() override;

  // SocketFactory:
  Socket* CreateSocket(int family, int type) override;
  AsyncSocket* CreateAsyncSocket(int family, int type) override;

  // Internal Factory for Accept (virtual so it can be overwritten in tests).
  virtual AsyncSocket* WrapSocket(SOCKET s);

  // SocketServer:
  bool Wait(int cms, bool process_io) override;
  void WakeUp() override;

  void Add(Dispatcher* dispatcher);
  void Remove(Dispatcher* dispatcher);
  void Update(Dispatcher* dispatcher);

 private:
  typedef std::set<Dispatcher*> DispatcherSet;

  void AddRemovePendingDispatchers();
  // 省略...
  DispatcherSet dispatchers_;
  DispatcherSet pending_add_dispatchers_;
  DispatcherSet pending_remove_dispatchers_;
  bool processing_dispatchers_ = false;
  Signaler* signal_wakeup_;
  CriticalSection crit_;
#if defined(WEBRTC_WIN)
  const WSAEVENT socket_ev_;
#endif
  bool fWait_;
};

        Windows 平台下事件涉及的 API：WSACreateEvent 创建事件句柄，WSAWaitForMultipleEvent 等待事件，WSASetEvent 触发事件，WSAResetEvent 重置事件，WSAEventSelect 将 socket 与事件绑定，WSAEnumNetworkEvents 枚举发生事件的 socket。

事件的触发

        等待事件的线程调用相应 SocketServer 的 Wait 函数后开始等待事件，而其他线程需要调用通知该线程事件并唤醒，举例说明线程间事件的通知与触发。

        如下调用 worker_thread_->Invoke 通知 worker 线程执行匿名函数的函数体。

if (media_engine_) {
  initialized_ = worker_thread_->Invoke<bool>(
      RTC_FROM_HERE, [&] { return media_engine_->Init(); });
  RTC_DCHECK(initialized_);
} else {
  initialized_ = true;
}

template <
    class ReturnT,
    typename = typename std::enable_if<!std::is_void<ReturnT>::value>::type>
ReturnT Invoke(const Location& posted_from, FunctionView<ReturnT()> functor) {
  ReturnT result;
  InvokeInternal(posted_from, [functor, &result] { result = functor(); });
  return result;
}

         Invoke 函数会调用线程的 Send 函数构造一个 Message 对象，其中 Message.phandler 为消息对应的执行函数，Message.pdata 为函数参数，再调用 PostTask 将消息存放到该线程的 messages_ 中，最后调用 WakeUpSocketServer 唤醒该线程，如果此时该线程处于 Wait 状态则会被唤醒，然后拿到 message 并执行相应的函数体。

void Thread::InvokeInternal(const Location& posted_from,
                            rtc::FunctionView<void()> functor) {
  TRACE_EVENT2("webrtc", "Thread::Invoke", "src_file", posted_from.file_name(),
               "src_func", posted_from.function_name());

  class FunctorMessageHandler : public MessageHandler {
   public:
    explicit FunctorMessageHandler(rtc::FunctionView<void()> functor)
        : functor_(functor) {}
    void OnMessage(Message* msg) override { functor_(); }

   private:
    rtc::FunctionView<void()> functor_;
  } handler(functor);

  Send(posted_from, &handler);
}