WebRTC 實(shí)戰(zhàn): QT for Windows 多人音視頻通話(huà)

簡(jiǎn)介

在經(jīng)過(guò)前面幾篇文章對(duì) WebRTC ?的描述，相信已經(jīng)不需再過(guò)多對(duì)它介紹了。前面幾篇文章我們實(shí)現(xiàn)了 Web 、Android 端的音視頻通話(huà)項(xiàng)目，該篇我們使用 QT UI 框架搭建 Windows 端的多 P2P 音視頻通話(huà)實(shí)戰(zhàn)項(xiàng)目。

項(xiàng)目地址:https://github.com/yangkun19921001/OpenRTCClient/tree/develop/examples/p2ps/p2ps

最終與 Android、Web 端運(yùn)行后的效果如下:

環(huán)境搭建

1. QT 環(huán)境準(zhǔn)備

首選我們?nèi)ハ螺d QT 6.6.0 最新版本，下載地址為: https://www.qt.io/download ,安裝好后選擇 cmake 進(jìn)行構(gòu)建項(xiàng)目

2. 信令服務(wù)器準(zhǔn)備

2.1 clone 信令服務(wù)器代碼

      
      git?clone?https://github.com/yangkun19921001/OpenRTCProject.git

    

2.2 啟動(dòng)信令服務(wù)器

      
      cd?p2ps/server

//修改你自己的證書(shū)
const?server?=?process.env.HTTPS?===?'true'???http.createServer({
????key?:?fs.readFileSync('./cert/rtcmedia.top.key'),
????cert:?fs.readFileSync('./cert/rtcmedia.top_bundle.pem')
??},?app):?http.createServer(app);

//執(zhí)行啟動(dòng)命令
./server_run.js
./proxy_server_run.js

//查看是否啟動(dòng)成功
lsof?-i:8880?和?443


    

3. webrtc 靜態(tài)庫(kù)準(zhǔn)備

1. clone webrtc(m98) develop 代碼

      
      git?clone?https://github.com/yangkun19921001/OpenRTCClient.git

    

1. 按照 README 進(jìn)行編譯，編譯完成后拿到靜態(tài)庫(kù) build/win/debug/obj/webrtc.lib

4. socketio-client-cpp 2.0.0 靜態(tài)庫(kù)準(zhǔn)備

按照官方文檔進(jìn)行編譯，可參考:https://github.com/socketio/socket.io-client-cpp/blob/2.0.0/INSTALL.md#with-cmake

5. QT cmake 編寫(xiě)

上序 4 步如果都準(zhǔn)備好，那么就可以通過(guò) cmake 進(jìn)行將其依賴(lài)進(jìn)來(lái)，如下所示:

      
      cmake_minimum_required(VERSION 3.5)
project(p2ps VERSION 0.1 LANGUAGES CXX)
set(CMAKE_AUTOUIC ON)
set(CMAKE_AUTOMOC ON)
set(CMAKE_AUTORCC ON)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(BUILD_TYPE debug)
if(MSVC)
    if(CMAKE_BUILD_TYPE MATCHES Debug)
        set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /MTd")
        set(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /MTd")
    elseif(CMAKE_BUILD_TYPE MATCHES Release)
        set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /MT")
        set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} /MT")
        #set(BUILD_TYPE release)
    endif()
endif()
set(WEBRTC_THIRD_PARTY_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../../../webrtc/third_party)
set(LIBWEBRTC_INCLUDE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../../../webrtc)
set(LIBWEBRTC_BINARY_PATH ${CMAKE_CURRENT_SOURCE_DIR}/../../../build_system/build/win/x64/${BUILD_TYPE}/obj)

set(DEPS_ROOT_PATH ${CMAKE_CURRENT_SOURCE_DIR}/deps)
set(SOCKET_IO_BINARY_PATH  ${DEPS_ROOT_PATH}/socketio/win/x64/${BUILD_TYPE})
set(SOCKET_IO_INCLUDE_PATH  ${DEPS_ROOT_PATH}/socketio/include)

set(JSONCPP_SOURCE
    "${WEBRTC_THIRD_PARTY_PATH}/jsoncpp/source/src/lib_json/json_reader.cpp"
    "${WEBRTC_THIRD_PARTY_PATH}/jsoncpp/source/src/lib_json/json_tool.h"
    "${WEBRTC_THIRD_PARTY_PATH}/jsoncpp/source/src/lib_json/json_value.cpp"
    "${WEBRTC_THIRD_PARTY_PATH}/jsoncpp/source/src/lib_json/json_writer.cpp"
)


target_include_directories(${PROJECT_NAME} PUBLIC
        "${LIBWEBRTC_INCLUDE_PATH}"
        "${LIBWEBRTC_INCLUDE_PATH}/third_party/abseil-cpp"
        "${LIBWEBRTC_INCLUDE_PATH}/third_party/jsoncpp/source/include"
        "${LIBWEBRTC_INCLUDE_PATH}/third_party/jsoncpp/generated"
        "${LIBWEBRTC_INCLUDE_PATH}/third_party/libyuv/include"
        "${SOCKET_IO_INCLUDE_PATH}"
)

add_definitions(
        #webrtc & qt 沖突
        -DQT_DEPRECATED_WARNINGS
        -DQT_NO_KEYWORDS

        #jsoncpp
        -DJSON_USE_EXCEPTION=0
        -DJSON_USE_NULLREF=0

        #socketio
        #-DSIO_TLS

        -DUSE_AURA=1
        -D_HAS_EXCEPTIONS=0
        -D__STD_C
        -D_CRT_RAND_S
        -D_CRT_SECURE_NO_DEPRECATE
        -D_SCL_SECURE_NO_DEPRECATE
        -D_ATL_NO_OPENGL
        -D_WINDOWS
        -DCERT_CHAIN_PARA_HAS_EXTRA_FIELDS
        -DPSAPI_VERSION=2
        -DWIN32
        -D_SECURE_ATL
        -DWINUWP
        -D__WRL_NO_DEFAULT_LIB__
 #       -DWINAPI_FAMILY=WINAPI_FAMILY_PC_APP
        -DWIN10=_WIN32_WINNT_WIN10
        -DWIN32_LEAN_AND_MEAN
        -DNOMINMAX
        -D_UNICODE
        -DUNICODE
        -DNTDDI_VERSION=NTDDI_WIN10_RS2
        -D_WIN32_WINNT=0x0A00
        -DWINVER=0x0A00
        -DDEBUG
        -DNVALGRIND
        -DDYNAMIC_ANNOTATIONS_ENABLED=0
        -DWEBRTC_ENABLE_PROTOBUF=0
        -DWEBRTC_INCLUDE_INTERNAL_AUDIO_DEVICE
        -DRTC_ENABLE_VP9
        -DHAVE_SCTP
        -DWEBRTC_LIBRARY_IMPL
        -DWEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS=0
        -DWEBRTC_WIN
        -DABSL_ALLOCATOR_NOTHROW=1
        -DHAVE_SCTP
        -DWEBRTC_VIDEO_CAPTURE_WINRT)

        target_link_libraries(p2ps PRIVATE
        ...
        ${SOCKET_IO_BINARY_PATH}/sioclient.lib ${SOCKET_IO_BINARY_PATH}/sioclient_tls.lib

        ${LIBWEBRTC_BINARY_PATH}/webrtc.lib
        winmm.lib iphlpapi.lib wbemuuid.lib secur32.lib advapi32.lib Mmdevapi.lib
        Mfuuid.lib msdmo.lib dmoguids.lib wmcodecdspuuid.lib  comdlg32.lib dbghelp.lib
        dnsapi.lib gdi32.lib msimg32.lib odbc32.lib odbccp32.lib oleaut32.lib shell32.lib
        shlwapi.lib user32.lib usp10.lib uuid.lib version.lib wininet.lib winmm.lib   
        winspool.lib ws2_32.lib delayimp.lib kernel32.lib ole32.lib crypt32.lib
        amstrmid.lib strmiids.lib
)

    

到此，在 QT 項(xiàng)目中的 WebRTC 和 SocketIO 環(huán)境已經(jīng)搭建完成了，這里為什么要搭建 SocketIO 呢？因?yàn)樵谥暗??web 、server、Android 都使用的是 socketio 開(kāi)源庫(kù)來(lái)做的信令通信，正好它又是跨平臺(tái)的，所以就不再更換了。

如何構(gòu)建多人音視頻通話(huà)?

要在 QT 中構(gòu)建一個(gè)多人的音視頻通話(huà)項(xiàng)目，必定要經(jīng)過(guò)如下幾個(gè)步驟

1. 信令交互協(xié)議

server 端的信令其實(shí)設(shè)計(jì)的很簡(jiǎn)單，就三組信令，如下所示:

join > joined ：加入房間和加入成功的通知 leave > leaved：離開(kāi)房間和離開(kāi)成功的通知 message：交換 offer ,candidate

更加詳細(xì)的流程可以參考下面的流程圖

2. 本地信令交互封裝

首先定義 ISinnalClient.h 抽象接口

      
      namespace?PCS{
class?ISignalClient?{
public:
????enum?class?SignalEvent?{
????????JOINED?=?0,
????????JOIN,
????????LEAVED,
????????LEAVE,
????????MESSAGE
????};

????static?std::string?SignalEventToString(SignalEvent?event)?{
????????switch(event)?{
????????case?SignalEvent::JOINED:??return?"joined";
????????case?SignalEvent::LEAVED:??return?"leaved";
????????case?SignalEvent::JOIN:????return?"join";
????????case?SignalEvent::LEAVE:???return?"leave";
????????case?SignalEvent::MESSAGE:?return?"message";
????????default:?return?"unknown";
????????}
????}

????/*連接服務(wù)端*/
????virtual?bool?connect(const?std::string?url,OnSignalEventListener*?listener)?=?0;
????/*加入會(huì)話(huà)*/
????virtual?void?join(const?std::string?roomId)?=?0;
????/*離開(kāi)會(huì)話(huà)*/
????virtual?void?leave(const?std::string?roomId)?=?0;
????/*銷(xiāo)毀?client?*/
????virtual?void?release()?=?0;
????/*發(fā)送消息*/
????virtual?void?sendMessage(const?std::string?roomId,?const?std::string?remoteId,?const?std::string?message)?=?0;

????virtual?std::string?getSocketId()?=?0;
????virtual?~ISignalClient()?=?default;
};

}

    

然后 socketio 根據(jù)對(duì)應(yīng)的 api 去封裝實(shí)現(xiàn)即可，詳細(xì)的使用，可以參考 https://github.com/yangkun19921001/OpenRTCClient/blob/develop/examples/p2ps/p2ps/src/common/SocketIoSignalClientImpl.cpp

3. 多 PeerConnection 管理

多 PeerConnection 管理其實(shí)就是把每次加入房間的 Peer 添加到一個(gè)容器中，管理起來(lái)。這里我們可以定義一個(gè) map 結(jié)構(gòu)進(jìn)行管理，核心 api 如下

3.1 定義一個(gè) PeerConnection 管理結(jié)構(gòu)體

      
      struct?Peer
{
????Peer()?{}
????rtc::scoped_refptr<webrtc::PeerConnectionInterface>?peer_conn_inter_;
????std::unique_ptr<PeerConnectionObserverImpl>?peer_conn_obser_impl_;
????std::unique_ptr<CreateSessionDescriptionObserImpl>?create_offer_sess_des_impl_;
????std::unique_ptr<CreateSessionDescriptionObserImpl>?create_answer_sess_des_impl_;
};

    

內(nèi)部主要包含每個(gè) PeerConnection 所需要的成員，然后通過(guò) std::map 進(jìn)行管理，如下所示

      
      std::map<std::string?,std::unique_ptr<Peer>>?peers_;

    

map 中的key 就是連接到房間中的 id

3.2 創(chuàng)建 PeerConnectionFactory

通過(guò)如下核心代碼即可創(chuàng)建出 PeerConnectionFactory

      
      ???????peer_connection_factory_?=?webrtc::CreatePeerConnectionFactory(
???????????this->network_thread_.get()?/*?network_thread?*/,
???????????this->worker_thread_.get()?/*?worker_thread?*/,
???????????this->signaling_thread_.get(),?/*?signaling_thread?*/
???????????nullptr?/*?default_adm?*/,
???????????webrtc::CreateBuiltinAudioEncoderFactory(),
???????????webrtc::CreateBuiltinAudioDecoderFactory(),
???????????webrtc::CreateBuiltinVideoEncoderFactory(),
???????????webrtc::CreateBuiltinVideoDecoderFactory(),?nullptr?/*?audio_mixer?*/,
???????????nullptr?/*?audio_processing?*/);

    

當(dāng)創(chuàng)建成功后，再創(chuàng)建本地音視頻軌道，后續(xù)會(huì)將本地音視頻軌道添加到 PeerConnection 中

      
      //音頻軌道
?audio_track_??=?peer_connection_factory_->CreateAudioTrack(
????????kAudioLabel,?peer_connection_factory_->CreateAudioSource(
????????????cricket::AudioOptions()));
?//視頻軌道
??rtc::scoped_refptr<CameraCapturerTrackSource>?video_device?=
??????CameraCapturerTrackSource::Create(1280,720,30);
??video_track_?=
??????peer_connection_factory_->CreateVideoTrack(kVideoLabel,?video_device);

    

3.3 創(chuàng)建 PeerConnection

      
      bool?PeerManager::createPeerConnection(const?std::string?&peerId,?const?webrtc::PeerConnectionInterface::RTCConfiguration?&config
???????????????????????????????????????,??OnPeerManagerEvents*?ets)
{
??RTC_LOG(LS_INFO)?<<__FUNCTION__?<<"?peerId:"<<peerId;
??RTC_DCHECK(peer_connection_factory_);
??????std::unique_ptr<PeerConnectionObserverImpl>?peer_conn_obimpl?=
??????????std::make_unique<PeerConnectionObserverImpl>(peerId,ets);
??????auto?peerConnection?=?peer_connection_factory_->CreatePeerConnection(
??????????config,
??????????nullptr,
??????????nullptr,
??????????peer_conn_obimpl.get());
??????if?(peerConnection)?{
??????????auto?peer_ptr?=?std::make_unique<Peer>();
??????????peer_ptr->peer_conn_inter_?=?peerConnection;
??????????peer_ptr->peer_conn_obser_impl_?=std::move(peer_conn_obimpl);
??????????peers_[peerId]?=?std::move(peer_ptr);
??????????if(video_track_)
???????????peerConnection->AddTrack(video_track_,?{?kVideoLabel?});
???????????if(audio_track_)
??????????peerConnection->AddTrack(audio_track_,?{?kAudioLabel?});
??????????return?true;
??????}
??????return?false;
}

    

由上面的代碼得知，我們通過(guò) peer_connection_factory_->CreatePeerConnection 就可以構(gòu)建一個(gè) PeerConnection ，并把之前創(chuàng)建出來(lái)的本地音視頻軌道添加到 PeerConnection 中，最后我們將構(gòu)建出來(lái)的 PeConn 緩存到 map 中，便于后續(xù)的處理。

3.4 創(chuàng)建 offer

      
      void?PeerManager::createOffer(const?std::string?&peerId,?OnPeerManagerEvents*?ets)
{
??RTC_LOG(LS_INFO)?<<__FUNCTION__?<<"?peerId:"<<peerId;
??auto?it?=?peers_.find(peerId);
??if?(it?!=?peers_.end()?&&?it->second->peer_conn_inter_?!=?nullptr)?{
??????auto?obs?=?std::make_unique<CreateSessionDescriptionObserImpl>(true,peerId,ets);
??????it->second->peer_conn_inter_->CreateOffer(obs.get(),?webrtc::PeerConnectionInterface::RTCOfferAnswerOptions());
??????it->second->create_offer_sess_des_impl_?=?std::move(obs);
??}else?{
??????RTC_LOG(LS_ERROR)?<<?__FUNCTION__?<<??"?peers_?not?found?id:"<<peerId;

??}
}

    

上面的代碼，首先根據(jù) peerId 從緩存中拿到對(duì)應(yīng)的 PeerConnection ,然后再調(diào)用它內(nèi)部的 api 來(lái)進(jìn)行 CreateOffer

3.5 設(shè)置本地/遠(yuǎn)端 SDP

      
      void?PeerManager::setLocalDescription(const?std::string?&peerId,webrtc::SessionDescriptionInterface?*desc_ptr)
{
??RTC_LOG(LS_INFO)?<<__FUNCTION__?<<"?peerId:"<<peerId;
??auto?pt?=?peers_.find(peerId);
??if?(pt?!=?peers_.end())?{
??????pt->second->peer_conn_inter_->SetLocalDescription(SetSessionDescriptionObserverImpl::Create(),desc_ptr);

??}else?{
??????RTC_LOG(LS_ERROR)?<<?__FUNCTION__?<<??"?peers_?not?found?id:"<<peerId;

??}
}

void?PeerManager::setRemoteDescription(const?std::string?&peerId,webrtc::SessionDescriptionInterface?*desc_ptr)
{
??RTC_LOG(LS_INFO)?<<__FUNCTION__?<<"?peerId:"<<peerId;
??auto?pt?=?peers_.find(peerId);
??if?(pt?!=?peers_.end())?{
??pt->second->peer_conn_inter_->SetRemoteDescription(SetSessionDescriptionObserverImpl::Create(),desc_ptr);
??}else?{
??RTC_LOG(LS_ERROR)?<<?__FUNCTION__?<<??"?peers_?not?found?id:"<<peerId;

??}
}

    

從緩存中拿到對(duì)應(yīng)的 PeerConnection ,然后設(shè)置本地或遠(yuǎn)端的 sdp 描述信息

3.6 創(chuàng)建 answer

      
      void?PeerManager::createAnswer(const?std::string?&peerId,?OnPeerManagerEvents*?ets)
{
?RTC_LOG(LS_INFO)?<<__FUNCTION__?<<"?peerId:"<<peerId;
??auto?it?=?peers_.find(peerId);
??if?(it?!=?peers_.end()?&&?it->second->peer_conn_inter_?!=?nullptr)?{
??????auto?obs?=?std::make_unique<CreateSessionDescriptionObserImpl>(false,peerId,ets);
??????it->second->peer_conn_inter_->CreateAnswer(obs.get(),?webrtc::PeerConnectionInterface::RTCOfferAnswerOptions());
??????it->second->create_answer_sess_des_impl_?=?std::move(obs);
??}else?{
??????RTC_LOG(LS_ERROR)?<<?__FUNCTION__?<<??"?peers_?not?found?id:"<<peerId;

??}
}

    

此處與上一步邏輯一樣

3.7 處理 ice

      
      void?PeerManager::handleCandidate(std::string?peerId,?std::unique_ptr<webrtc::IceCandidateInterface>?candidate)
{
??????RTC_LOG(LS_INFO)?<<?__FUNCTION__?<<"?peerId:"<<peerId;
??????auto?pt?=?peers_.find(peerId);
??????if?(pt?!=?peers_.end())?{
??????????pt->second->peer_conn_inter_->AddIceCandidate(
??????????????std::move(candidate),
??????????????[peerId](webrtc::RTCError?error){
??????????????????if?(error.ok())?{
??????????????????????RTC_LOG(LS_INFO)?<<"?peerId:"<<?peerId?<<?"?AddIceCandidate?success.";
??????????????????}?else?{
??????????????????????RTC_LOG(LS_INFO)?<<"?peerId:"<<?peerId?<<?"AddIceCandidate?failed,?error:?"?<<?error.message();
??????????????????}
???????});
????}else?{
??????????RTC_LOG(LS_ERROR)?<<?__FUNCTION__?<<??"?peers_?not?found?id:"<<peerId;

????}
}

    

也是從緩存中拿到對(duì)應(yīng)的 PeerConnection ，然后將對(duì)方的候選者地址添加進(jìn)去。

4. 房間管理

定義 RTCRoomManager ，實(shí)現(xiàn)信令回調(diào)和PeerManager 回調(diào)，定義的核心 API 如下

      
      namespace?PCS{

class?RTCRoomManager?:?public?OnSignalEventListener,public?OnPeerManagerEvents?{
public:
????RTCRoomManager();
????virtual?~RTCRoomManager();

????//連接服務(wù)器
????void?connect(const?std::string?url,OnRoomStateChangeCallback*?callback);
????//設(shè)置本地軌道的回調(diào)監(jiān)聽(tīng)
????void?setLocalTrackCallback(std::function<void(std::string,int,int,rtc::scoped_refptr<webrtc::VideoTrackInterface>)>?localVideoTrack);
????//加入房間
????void?join(const?std::string?roomId);
????//離開(kāi)房間
????void?leave(const?std::string?roomId);
????//銷(xiāo)毀
????void?release();
????//處理?ui?傳遞過(guò)來(lái)的?消息
????void?onUIMessage(Message?msg);

private:
????//實(shí)現(xiàn)連接成功的處理代碼
????void?onConnectSuccessful()?override;
????//實(shí)現(xiàn)正在連接的處理代碼
????void?onConnecting()?override?;
????//實(shí)現(xiàn)連接錯(cuò)誤的處理代碼
????void?onConnectError(const?std::string&?error)?override?;
????//實(shí)現(xiàn)已加入房間的處理代碼
????void?onJoined(const?std::string&?room,?const?std::string&?id,?const?std::vector<std::string>&?otherClientIds)?override;
????//實(shí)現(xiàn)離開(kāi)房間的處理代碼
????void?onLeaved(const?std::string&?room,?const?std::string&?id)?override?;
????//實(shí)現(xiàn)接收到消息的處理代碼
????void?onMessage(const?std::string&?from,?const?std::string&?to,?const?std::string&?message)?override?;

????//當(dāng)需要添加遠(yuǎn)端的軌道
?????void?OnAddTrack(std::string?peerid,
????????????????????????????rtc::scoped_refptr<webrtc::RtpReceiverInterface>?receiver,
????????????????????????????const?std::vector<rtc::scoped_refptr<webrtc::MediaStreamInterface>>&
????????????????????????????????streams)override;
????//當(dāng)刪除遠(yuǎn)端的軌道
?????void?OnRemoveTrack(std::string?peerid,
???????????????????????????????rtc::scoped_refptr<webrtc::RtpReceiverInterface>?receiver)?override;
?????//?datachannel?消息
?????void?OnDataChannel(std::string?peerid,
???????????????????????????????rtc::scoped_refptr<webrtc::DataChannelInterface>?channel)?override;
?????//ice?消息??????????????????????????
?????void?OnIceCandidate(std::string?peerid,const?webrtc::IceCandidateInterface*?candidate)?override;
?????//offer?or?answer?create?成功
?????void?OnCreateSuccess(bool?offer,std::string?peerid,webrtc::SessionDescriptionInterface*?desc)?override;
?????//offer?or?answer?create?失敗
?????void?OnCreateFailure(bool?offer,std::string?peerid,webrtc::RTCError?error)?override;




private:
????std::unique_ptr<ISignalClient>?socket_signal_client_imp_;
????std::unique_ptr<PeerManager>?peer_manager_;
????OnRoomStateChangeCallback?*?room_state_change_callback_;
????std::function<void(std::string,int,int,rtc::scoped_refptr<webrtc::VideoTrackInterface>)>?local_track_callback_;
????std::string?room_id_;

};

}?//?end?namespace?PCS

    

這就是核心 API, 它持有 peer_manager_、socket_signal_client_imp_ ，分別是對(duì) PeerConnection 和信令的交互。

比如現(xiàn)在 A 用戶(hù)先進(jìn)入房間，B 后進(jìn)入房間，然后對(duì)它們的管理流程是這樣的

1. A 用戶(hù)連接服務(wù)器 -> 連接成功 ->發(fā)起 join 信令->收到 joined 信令->createPeerConnectionFactory->攝像頭開(kāi)始采集->等待預(yù)覽
2. B 用戶(hù)連接服務(wù)器 -> 連接成功 ->發(fā)起 join 信令 ->收到j(luò)oined 信令(并帶上了 A 用戶(hù)在房間中的信息) ->createPeerConnectionFactory->等待本地預(yù)覽->createPeerConnection(A用戶(hù))
3. A 用戶(hù)收到 B 用戶(hù) joined 加入房間的信令 -> createOffer -> 設(shè)置本地 SDP -> 發(fā)送 本地 SDP offer 信令到服務(wù)器
4. B 用戶(hù)收到 服務(wù)器轉(zhuǎn)發(fā)過(guò)來(lái)的 A 用戶(hù)的 offer 信令 -> 設(shè)置遠(yuǎn)端的 SDP 信息 ->CreateAnswer(A用戶(hù))->設(shè)置本地 SDP 信息 -> 發(fā)送 answer 消息給 A
5. A 收到 B 發(fā)送的 answer 信令消息，調(diào)用 setRemoteDescription ?函數(shù)將 B 的 SDP 設(shè)置進(jìn)去
6. A,B 交換 candidate 消息，并將對(duì)方的 candidate 調(diào)用 AddIceCandidate 添加進(jìn)去
7. 到這一步后，就等待 onAddTrack 回調(diào)了，下一步就是如何將對(duì)方和本地的畫(huà)面進(jìn)行顯示了

5. 如何顯示

在 webrtc 架構(gòu)中，是通過(guò) rtc::VideoSinkInterface<webrtc::VideoFrame> 的 onFrame 虛函數(shù)進(jìn)行通知需要新視頻數(shù)據(jù)的渲染。

我們定義一個(gè) VideoRendererWidget 然后實(shí)現(xiàn) rtc::VideoSinkInterface<webrtc::VideoFrame> 的 onFrame 函數(shù)，如下所示:

      
      namespace?PCS?{
class?VideoRendererWidget?:?public?QOpenGLWidget,?protected?QOpenGLFunctions,
????????????????????????????public?rtc::VideoSinkInterface<webrtc::VideoFrame>
{
????Q_OBJECT

public:
????VideoRendererWidget(std::string?peerId?="",QWidget*?parent?=?nullptr,webrtc::VideoTrackInterface?*track?=nullptr);
????~VideoRendererWidget();

public?Q_SLOTS:
????void?PlayOneFrame();

protected:
????void?initializeGL()?Q_DECL_OVERRIDE;
????void?resizeGL(int?w,?int?h)?Q_DECL_OVERRIDE;
????void?paintGL()?Q_DECL_OVERRIDE;



public:
????//?VideoSinkInterface?implementation
????void?OnFrame(const?webrtc::VideoFrame&?frame)?override;


private:
...

public:
???webrtc::VideoTrackInterface*??video_track_;
};

}

void?VideoRendererWidget::OnFrame(const?webrtc::VideoFrame?&video_frame)
{
????std::lock_guard<std::mutex>?guard(renderer_mutex_);
????rtc::scoped_refptr<webrtc::I420BufferInterface>?buffer(
????????video_frame.video_frame_buffer()->ToI420());
????if?(video_frame.rotation()?!=?webrtc::kVideoRotation_0)?{
????????buffer?=?webrtc::I420Buffer::Rotate(*buffer,?video_frame.rotation());
????}

????int?width?=?buffer->width();
????int?height?=?buffer->height();

????int?ySize?=?width?*?height;
????int?uvSize?=?ySize?/?4;??//?For?each?U?and?V?component

????if(m_nVideoW?==?0?&&?m_nVideoH?==0)
????{
???????if(!peer_id_.empty())??RTC_LOG(LS_INFO)?<<?__FUNCTION__?<<?"?init?w:"<<width<<"?height:"<<height?<<?"?peerId:"?<<peer_id_;
????}

????if(width?!=?m_nVideoW?&&?height?!=?m_nVideoH?&&?video_data_?!=?nullptr)
????{
?????????video_data_?.reset();

????????if(!peer_id_.empty())?RTC_LOG(LS_INFO)?<<?__FUNCTION__?<<?"?change?w:"<<width<<"?height:"<<height?<<?"?peerId:"?<<peer_id_;

????}

????if(video_data_?==?nullptr){
??????video_data_?=?std::make_unique<uint8_t[]>(width?*?height?*?1.5);?//?Use?make_unique?to?allocate?array
??????if(!peer_id_.empty())
??????RTC_LOG(LS_INFO)?<<?__FUNCTION__?<<?"?malloc?Id:"<<peer_id_<<"?width:"?<<?width?<<"?height:"<<height;
????}
????memcpy(video_data_.get(),?buffer->DataY(),?ySize);
????memcpy(video_data_.get()?+?ySize,?buffer->DataU(),?uvSize);
????memcpy(video_data_.get()?+?ySize?+?uvSize,?buffer->DataV(),?uvSize);
????m_nVideoW?=?width;
????m_nVideoH?=?height;


????//?刷新界面,觸發(fā)paintGL接口
????Q_EMIT?PlayOneFrame();

}


    

當(dāng)我們調(diào)用 PlayOneFrame 時(shí)，就可以通過(guò) QOpenGL 來(lái)進(jìn)行渲染 I420 YUV 數(shù)據(jù)了。

6. 如何管理多個(gè)窗口的創(chuàng)建和銷(xiāo)毀的

可以通過(guò)管理 PeerConnection 那樣管理 VideoRendererWidget ,還是定義一個(gè) map

      
      std::map<std::string,?std::unique_ptr<PCS::VideoRendererWidget>>?video_renderer_widgets_;

    

根據(jù)對(duì)方的 peerid 來(lái)進(jìn)行緩存窗口，

當(dāng)需要添加窗口時(shí):

      
      void?MainWindow::addVideoRendererWidgetToMainWindow(std::string?id,?std::unique_ptr<PCS::VideoRendererWidget>?renderer)
{

?????const?int?itemsPerRow?=?3;
?????std::unique_ptr<PCS::VideoRendererWidget>?videoRenderer;
?????if(renderer?==?nullptr)
????????videoRenderer?=??std::make_unique<PCS::VideoRendererWidget>();
?????else?{
????????videoRenderer?=?std::move(renderer);
?????}
?????videoRenderer->setSizePolicy(QSizePolicy::Expanding,?QSizePolicy::Expanding);
?????//?計(jì)算新的位置
?????int?count?=?ui->gridLayout->count();
?????int?row?=?count?/?itemsPerRow;
?????int?column?=?count?%?itemsPerRow;

?????//?添加到?gridLayout?中
?????ui->gridLayout->addWidget(videoRenderer.get(),row,column);
?????//videoRenderer->setFixedSize(1280/3,720/3);
?????//?將?widget?添加到?map?中
?????video_renderer_widgets_.insert({id,?std::move(videoRenderer)});

}

    

當(dāng)需要?jiǎng)h除窗口時(shí):

      
      void?MainWindow::removeVideoRendererWidgetFromMainWindow(std::string?id)
????{
?????//?從?layout?中移除?widget，并刪除它
?????//?查找?widget
?????auto?it?=?video_renderer_widgets_.find(id);
?????if?(it?!=?video_renderer_widgets_.end())
?????{
????????//?從?layout?中移除?widget
????????ui->gridLayout->removeWidget(it->second.get());

????????//?從?map?中移除并刪除?widget
????????video_renderer_widgets_.erase(it);
?????}
????}

    

總結(jié)

我們通過(guò)簡(jiǎn)短的描述和一些基礎(chǔ)的 api 來(lái)介紹了如何通過(guò) QT webrtc 來(lái)構(gòu)建一個(gè)多人的音視頻通話(huà)的項(xiàng)目。由于本人對(duì) QT 不是太熟悉，所以 UI 上還有少許 Bug 。但不影響核心 API 調(diào)用。

到此，通過(guò)本篇文章和之前的幾篇文章我們已經(jīng)實(shí)現(xiàn)了 Web 、Android 、Windows 之前的互通，后續(xù)會(huì)繼續(xù)介紹 WebRTC 源碼分析和實(shí)戰(zhàn)項(xiàng)目的開(kāi)發(fā)。