C++ 在 Unreal 中為游戲增加實時音視頻互動的教程詳解
更新時間：2020年05月29日 09:31:25 作者：聲網(wǎng)Agora
這篇文章主要介紹了C++ 在 Unreal 中為游戲增加實時音視頻互動的教程，本文通過圖文并茂的形式給大家介紹的非常詳細(xì)，對大家的學(xué)習(xí)或工作具有一定的參考借鑒價值,需要的朋友可以參考下
我們已經(jīng)上線了 Agora Unreal SDK，提供了支持 Blueprint 和 C++ 的兩個版本 SDK。我們分享了如何基于 Blueprint 在游戲中創(chuàng)建實時音視頻功能 。在本文中，我們來分享如何基于聲網(wǎng) Agora Unreal SDK C++版本，在游戲中實現(xiàn)實時音視頻功能。
本篇教程較長，建議在 Web 瀏覽器端瀏覽，體驗更好。
準(zhǔn)備工作
需要的開發(fā)環(huán)境和需要準(zhǔn)備的與 Blueprint 一樣：
- Unreal 4.34 以上版本
- Visual Studio 或 Xcode（版本根據(jù) Unreal 配置要求而定）
- 運行 Windows 7 以上系統(tǒng)的 PC 或 一臺 Mac
- Agora 注冊賬號一枚（免費注冊，見官網(wǎng) Agora.io）
- 如果你的企業(yè)網(wǎng)絡(luò)存在防火墻，請在聲網(wǎng)文檔中心搜索「應(yīng)用企業(yè)防火墻限制」，進(jìn)行配置。
新建項目
如果你已經(jīng)有 Unreal 項目了，可以跳過這一步。在 Unreal 中創(chuàng)建一個 C++類型的項目。
確保在 [your_project]/Source/[project_name]/[project_name].Build.cs文件的 PrivateDependencyModuleNames一行，去掉注釋。Unreal 默認(rèn)是將它注釋掉的，這會導(dǎo)致在編譯的時候報錯。
// Uncomment if you are using Slate UI
PrivateDependencyModuleNames.AddRange(new string[] { "UMG", "Slate", "SlateCore" });
接下來我們在項目中集成 Agora SDK
1.將 SDK 復(fù)制到這個路徑下 [your_project]/Plugins
2.把插件依賴添加到[your_project]/Source/[project_name]/[project_name].Build.cs文件的私有依賴（Private Dependencies）部分 PrivateDependencyModuleNames.AddRange(new string[] { "AgoraPlugin", "AgoraBlueprintable" });
3.重啟 Unreal
4.點擊 Edit->Plugin，在分類中找到 Project->Other，確定插件已經(jīng)生效
創(chuàng)建新的 Level
接下來我們將創(chuàng)建一個新的 Level，在那里建立我們的游戲環(huán)境。有幾種不同的方法可以創(chuàng)建一個新的 Level，我們將使用文件菜單的方法，其中列出了關(guān)卡選擇選項。
在虛幻編輯器里面，點擊文件菜單選項，然后選擇新建 Level......
然后會打開一個新的對話框 。
選擇Empty Level ，然后指定一個存儲的路徑。
創(chuàng)建核心類
在這里我們要創(chuàng)建兩個類：VideoFrameObserver 和VideoCall C++ Class。他們會負(fù)責(zé)與 Agora SDK 進(jìn)行通信。
首先是 VideoFrameObserver。VideoFrameObserver 執(zhí)行的是 agora::media::IVideoFrameObserver。這個方法在 VideoFrameObserver 類中負(fù)責(zé)管理視頻幀的回調(diào)。它是用 registerVideoFrameObserver 在 agora::media::IMediaEngine 中注冊的。
在 Unreal 編輯器中，選擇 File->Add New C++ Class。
父類誰定為 None，然后點擊下一步。
為 VideoFrameObserver明明，然后選擇 Create Class。
創(chuàng)建 VideoFrameObserver 類接口。
打開 VideoFrameObserver.h 文件然后添加如下代碼：
//VideoFrameObserver.h
#include "CoreMinimal.h"
#include <functional>
#include "AgoraMediaEngine.h"
class AGORAVIDEOCALL_API VideoFrameObserver : public agora::media::IVideoFrameObserver
{
public:
virtual ~VideoFrameObserver() = default;
public:
bool onCaptureVideoFrame(VideoFrame& videoFrame) override;
bool onRenderVideoFrame(unsigned int uid, VideoFrame& videoFrame) override;
void setOnCaptureVideoFrameCallback(
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> callback);
void setOnRenderVideoFrameCallback(
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> callback);
virtual VIDEO_FRAME_TYPE getVideoFormatPreference() override { return FRAME_TYPE_RGBA; }
private:
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> OnCaptureVideoFrame;
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> OnRenderVideoFrame;
};
AGORAVIDEOCALL_API 是項目依賴的定義，而不是由Unreal 生成的你自己的定義。
重寫onCaptureVideoFrame/onRenderVideoFrame方法
onCaptureVideoFrame 會獲取到攝像頭捕獲的畫面，轉(zhuǎn)換為 ARGB 格式并觸發(fā) OnCaptureVideoFrame 回調(diào)。
onRenderVideoFrame 講收到的特定用戶畫面轉(zhuǎn)換為 ARGB 格式，然后觸發(fā) onRenderVideoFrame 回調(diào)。
//VideoFrameObserver.cpp
bool VideoFrameObserver::onCaptureVideoFrame(VideoFrame& Frame)
{
const auto BufferSize = Frame.yStride*Frame.height;
if (OnCaptureVideoFrame)
{
OnCaptureVideoFrame( static_cast< uint8_t* >( Frame.yBuffer ), Frame.width, Frame.height, BufferSize );
}
return true;
}
bool VideoFrameObserver::onRenderVideoFrame(unsigned int uid, VideoFrame& Frame)
{
const auto BufferSize = Frame.yStride*Frame.height;
if (OnRenderVideoFrame)
{
OnRenderVideoFrame( static_cast<uint8_t*>(Frame.yBuffer), Frame.width, Frame.height, BufferSize );
}
return true;
}
增加setOnCaptureVideoFrameCallback/setOnRenderVideoFrameCallback方法。
設(shè)定回調(diào)，用來獲取攝像頭獲取到的本地畫面和遠(yuǎn)端的畫面。
//VideoFrameObserver.cpp
void VideoFrameObserver::setOnCaptureVideoFrameCallback(
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> Callback)
{
OnCaptureVideoFrame = Callback;
}
void VideoFrameObserver::setOnRenderVideoFrameCallback(
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> Callback)
{
OnRenderVideoFrame = Callback;
}
創(chuàng)建視頻通話C++類
VideoCall 類管理與 Agora SDK 的通信。需要創(chuàng)建多個方法和接口。
創(chuàng)建類接口
回到 Unreal 編輯器，再創(chuàng)建一個新的 C++類，命名為 VideoCall.h。然后進(jìn)入VideoCall.h文件，添加一下接口：
//VideoCall.h
#pragma once
#include "CoreMinimal.h"
#include <functional>
#include <vector>
#include "AgoraRtcEngine.h"
#include "AgoraMediaEngine.h"
class VideoFrameObserver;
class AGORAVIDEOCALL_API VideoCall
{
public:
VideoCall();
~VideoCall();
FString GetVersion() const;
void RegisterOnLocalFrameCallback(
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> OnLocalFrameCallback);
void RegisterOnRemoteFrameCallback(
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> OnRemoteFrameCallback);
void StartCall(
const FString& ChannelName,
const FString& EncryptionKey,
const FString& EncryptionType);
void StopCall();
bool MuteLocalAudio(bool bMuted = true);
bool IsLocalAudioMuted();
bool MuteLocalVideo(bool bMuted = true);
bool IsLocalVideoMuted();
bool EnableVideo(bool bEnable = true);
private:
void InitAgora();
private:
TSharedPtr<agora::rtc::ue4::AgoraRtcEngine> RtcEnginePtr;
TSharedPtr<agora::media::ue4::AgoraMediaEngine> MediaEnginePtr;
TUniquePtr<VideoFrameObserver> VideoFrameObserverPtr;
//callback
//data, w, h, size
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> OnLocalFrameCallback;
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> OnRemoteFrameCallback;
bool bLocalAudioMuted = false;
bool bLocalVideoMuted = false;
};
創(chuàng)建初始化方法
進(jìn)入 VideoCall.cpp 文件，添加以下代碼：
//VideoCall.cpp #include "AgoraVideoDeviceManager.h" #include "AgoraAudioDeviceManager.h" #include "MediaShaders.h" #include "VideoFrameObserver.h"
用agora::rtc::ue4::AgoraRtcEngine::createAgoraRtcEngine()創(chuàng)建引擎，初始化 RtcEnginePtr 變量。創(chuàng)建一個RtcEngineContext對象，然后在ctx.eventHandler 和ctx.appId中設(shè)定 event handler 和 App ID 。初始化引擎，并創(chuàng)建AgoraMediaEngine對象，初始化 MediaEnginePtr。
//VideoCall.cpp
VideoCall::VideoCall()
{
InitAgora();
}
VideoCall::~VideoCall()
{
StopCall();
}
void VideoCall::InitAgora()
{
RtcEnginePtr = TSharedPtr<agora::rtc::ue4::AgoraRtcEngine>(agora::rtc::ue4::AgoraRtcEngine::createAgoraRtcEngine());
static agora::rtc::RtcEngineContext ctx;
ctx.appId = "aab8b8f5a8cd4469a63042fcfafe7063";
ctx.eventHandler = new agora::rtc::IRtcEngineEventHandler();
int ret = RtcEnginePtr->initialize(ctx);
if (ret < 0)
{
UE_LOG(LogTemp, Warning, TEXT("RtcEngine initialize ret: %d"), ret);
}
MediaEnginePtr = TSharedPtr<agora::media::ue4::AgoraMediaEngine>(agora::media::ue4::AgoraMediaEngine::Create(RtcEnginePtr.Get()));
}
FString VideoCall::GetVersion() const
{
if (!RtcEnginePtr)
{
return "";
}
int build = 0;
const char* version = RtcEnginePtr->getVersion(&build);
return FString(ANSI_TO_TCHAR(version));
}
創(chuàng)建回調(diào)方法
接下來創(chuàng)建回調(diào)方法，返回本地和遠(yuǎn)端的視頻幀
//VideoCall.cpp
void VideoCall::RegisterOnLocalFrameCallback(
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> OnFrameCallback)
{
OnLocalFrameCallback = std::move(OnFrameCallback);
}
void VideoCall::RegisterOnRemoteFrameCallback(
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> OnFrameCallback)
{
OnRemoteFrameCallback = std::move(OnFrameCallback);
}
創(chuàng)建呼叫方法
我們需要利用這個方法來實現(xiàn)“加入頻道”和“離開頻道”。
增加 StartCall
首先創(chuàng)建 VideoFrameObserver 對象，然后根據(jù)你的場景來設(shè)置以下回調(diào)。
- OnLocalFrameCallback：用于 SDK 獲取本地攝像頭采集到的視頻幀。
- OnRemoteFrameCallback：用于 SDK 獲取遠(yuǎn)端攝像頭采集到的視頻幀。
在 InitAgora 的 MediaEngine 對象中通過 registerVideoFrameObserver 方法注冊 VideoFrameObserver。為了保證 EncryptionType 和 EncryptionKey 不為空，需要先設(shè)置 EncryptionMode 和 EncryptionSecret。然后按照你的需要來設(shè)置頻道參數(shù)，并調(diào)用 joinChannel。
//VideoCall.cpp
void VideoCall::StartCall(
const FString& ChannelName,
const FString& EncryptionKey,
const FString& EncryptionType)
{
if (!RtcEnginePtr)
{
return;
}
if (MediaEnginePtr)
{
if (!VideoFrameObserverPtr)
{
VideoFrameObserverPtr = MakeUnique<VideoFrameObserver>();
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> OnCaptureVideoFrameCallback
= [this](std::uint8_t* buffer, std::uint32_t width, std::uint32_t height, std::uint32_t size)
{
if (OnLocalFrameCallback)
{
OnLocalFrameCallback(buffer, width, height, size);
}
else { UE_LOG(LogTemp, Warning, TEXT("VideoCall OnLocalFrameCallback isn't set")); }
};
VideoFrameObserverPtr->setOnCaptureVideoFrameCallback(std::move(OnCaptureVideoFrameCallback));
std::function<void(std::uint8_t*, std::uint32_t, std::uint32_t, std::uint32_t)> OnRenderVideoFrameCallback
= [this](std::uint8_t* buffer, std::uint32_t width, std::uint32_t height, std::uint32_t size)
{
if (OnRemoteFrameCallback)
{
OnRemoteFrameCallback(buffer, width, height, size);
}
else { UE_LOG(LogTemp, Warning, TEXT("VideoCall OnRemoteFrameCallback isn't set")); }
};
VideoFrameObserverPtr->setOnRenderVideoFrameCallback(std::move(OnRenderVideoFrameCallback));
}
MediaEnginePtr->registerVideoFrameObserver(VideoFrameObserverPtr.Get());
}
int nRet = RtcEnginePtr->enableVideo();
if (nRet < 0)
{
UE_LOG(LogTemp, Warning, TEXT("enableVideo : %d"), nRet)
}
if (!EncryptionType.IsEmpty() && !EncryptionKey.IsEmpty())
{
if (EncryptionType == "aes-256")
{
RtcEnginePtr->setEncryptionMode("aes-256-xts");
}
else
{
RtcEnginePtr->setEncryptionMode("aes-128-xts");
}
nRet = RtcEnginePtr->setEncryptionSecret(TCHAR_TO_ANSI(*EncryptionKey));
if (nRet < 0)
{
UE_LOG(LogTemp, Warning, TEXT("setEncryptionSecret : %d"), nRet)
}
}
nRet = RtcEnginePtr->setChannelProfile(agora::rtc::CHANNEL_PROFILE_COMMUNICATION);
if (nRet < 0)
{
UE_LOG(LogTemp, Warning, TEXT("setChannelProfile : %d"), nRet)
}
//"demoChannel1";
std::uint32_t nUID = 0;
nRet = RtcEnginePtr->joinChannel(NULL, TCHAR_TO_ANSI(*ChannelName), NULL, nUID);
if (nRet < 0)
{
UE_LOG(LogTemp, Warning, TEXT("joinChannel ret: %d"), nRet);
}
}
增加 StopCall 功能
根據(jù)你的場景需要，通過調(diào)用 leaveChannel 方法來結(jié)束通話，比如當(dāng)要結(jié)束通話的時候，當(dāng)你需要關(guān)閉應(yīng)用的時候，或是當(dāng)你的應(yīng)用運行于后臺的時候。調(diào)用 nullptr 作為實參的 registerVideoFrameObserver，用來取消 VideoFrameObserver的注冊。
//VideoCall.cpp
void VideoCall::StopCall()
{
if (!RtcEnginePtr)
{
return;
}
auto ConnectionState = RtcEnginePtr->getConnectionState();
if (agora::rtc::CONNECTION_STATE_DISCONNECTED != ConnectionState)
{
int nRet = RtcEnginePtr->leaveChannel();
if (nRet < 0)
{
UE_LOG(LogTemp, Warning, TEXT("leaveChannel ret: %d"), nRet);
}
if (MediaEnginePtr)
{
MediaEnginePtr->registerVideoFrameObserver(nullptr);
}
}
}
創(chuàng)建 Video 方法
這些方法是用來管理視頻的。
加 EnableVideo() 方法
EnableVideo() 會啟用本示例中的視頻。初始化 nRet，值為 0。如果 bEnable 為 true，則通過 RtcEnginePtr->enableVideo() 啟用視頻。否則，通過 RtcEnginePtr->disableVideo() 關(guān)閉視頻。
//VideoCall.cpp
bool VideoCall::EnableVideo(bool bEnable)
{
if (!RtcEnginePtr)
{
return false;
}
int nRet = 0;
if (bEnable)
nRet = RtcEnginePtr->enableVideo();
else
nRet = RtcEnginePtr->disableVideo();
return nRet == 0 ? true : false;
}
增加 MuteLocalVideo() 方法
MuteLocalVideo() 方法負(fù)責(zé)開啟或關(guān)閉本地視頻。在其余方法完成運行之前，需要保證 RtcEnginePtr 不為 nullptr。如果可以成功mute 或 unmute 本地視頻，那么把 bLocalVideoMuted 設(shè)置為 bMuted。
//VideoCall.cpp
bool VideoCall::MuteLocalVideo(bool bMuted)
{
if (!RtcEnginePtr)
{
return false;
}
int ret = RtcEnginePtr->muteLocalVideoStream(bMuted);
if (ret == 0)
bLocalVideoMuted = bMuted;
return ret == 0 ? true : false;
}
增加 IsLocalVideoMuted() 方法
IsLocalVideoMuted() 方法的作用是，當(dāng)本地視頻開啟或關(guān)閉的時候，返回 bLocalVideoMuted。
//VideoCall.cpp
bool VideoCall::IsLocalVideoMuted()
{
return bLocalVideoMuted;
}
創(chuàng)建音頻相關(guān)的方法
這些方法是用來管理音頻的。
添加 MuteLocalAudio() 方法
MuteLocalAudio()用于 mute 或 unmute 本地音頻：
//VideoCall.cpp
bool VideoCall::MuteLocalAudio(bool bMuted)
{
if (!RtcEnginePtr)
{
return false;
}
int ret = RtcEnginePtr->muteLocalAudioStream(bMuted);
if (ret == 0)
bLocalAudioMuted = bMuted;
return ret == 0 ? true : false;
}
增加 IsLocalAudioMuted() 方法
IsLocalAudioMuted()方法的作用是，當(dāng) mute 或 unmute 本地音頻的時候，返回 bLocalAudioMuted。
//VideoCall.cpp
bool VideoCall::IsLocalAudioMuted()
{
return bLocalAudioMuted;
}
創(chuàng)建 GUI
接下來就是要為一對一對話創(chuàng)建用戶交互界面了，包括：
- 創(chuàng)建 VideoCallPlayerController
- 創(chuàng)建 EnterChannelWidget C++ Class
- 創(chuàng)建 VideoViewWidget C++ Class
- 創(chuàng)建 VideoCallViewWidget C++ Class
- 創(chuàng)建 VideoCallWidget C++ Class
- 創(chuàng)建 BP_EnterChannelWidget blueprint asset
- 創(chuàng)建 BP_VideoViewWidget Asset
- 創(chuàng)建 BP_VideoCallViewWidget Asset
- 創(chuàng)建 BP_VideoCallWidget Asset
- 創(chuàng)建 BP_VideoCallPlayerController blueprint asset
- 創(chuàng)建 BP_AgoraVideoCallGameModeBase Asset
- 修改 Game Mode
創(chuàng)建 VideoCallPlayerController
為了能夠?qū)⑽覀兊腤idget Blueprints添加到Viewport中，我們創(chuàng)建我們的自定義播放器控制器類。
在 "內(nèi)容瀏覽器 "中，按 "Add New "按鈕，選擇 "新建C++類"。在 "添加C++類 "窗口中，勾選 "顯示所有類 "按鈕，并輸入PlayerController。按 "下一步 "按鈕，給類命名為 VideoCallPlayerController。按Create Class按鈕。
//VideoCallPlayerController.h
#include "CoreMinimal.h"
#include "GameFramework/PlayerController.h"
#include "VideoCallPlayerController.generated.h"
UCLASS()
class AGORAVIDEOCALL_API AVideoCallPlayerController : public APlayerController
{
GENERATED_BODY()
public:
};
這個類是 BP_VideoCallPlayerController 的 Blueprint Asset 的基類，我們將在最后創(chuàng)建。
增加需要的 Include
在 VideoCallPlayerController.h 文件的頭部包括了所需的頭文件。
//VideoCallPlayerController.h #include "CoreMinimal.h" #include "GameFramework/PlayerController.h" #include "Templates/UniquePtr.h" #include "VideoCall.h" #include "VideoCallPlayerController.generated.h" //VideoCallPlayerController.cpp #include "Blueprint/UserWidget.h" #include "EnterChannelWidget.h" #include "VideoCallWidget.h"
類聲明
為下一個類添加轉(zhuǎn)發(fā)聲明：
//VideoCallPlayerController.h class UEnterChannelWidget; class UVideoCallWidget;
稍后我們將跟進(jìn)其中的兩個創(chuàng)建，即 UEnterChannelWidget 和 UVideoCallWidget。
添加成員變量
現(xiàn)在，在編輯器中添加成員引用到 UMG Asset 中。
//VideoCallPlayerController.h
...
UCLASS()
class AGORAVIDEOCALL_API AVideoCallPlayerController : public APlayerController
{
GENERATED_BODY()
public:
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Widgets")
TSubclassOf<class UUserWidget> wEnterChannelWidget;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "Widgets")
TSubclassOf<class UUserWidget> wVideoCallWidget;
...
};
變量來保持創(chuàng)建后的小部件，以及一個指向VideoCall的指針。
//VideoCallPlayerController.h
...
UCLASS()
class AGORAVIDEOCALL_API AVideoCallPlayerController : public APlayerController
{
GENERATED_BODY()
public:
...
UEnterChannelWidget* EnterChannelWidget = nullptr;
UVideoCallWidget* VideoCallWidget = nullptr;
TUniquePtr<VideoCall> VideoCallPtr;
...
};
覆蓋 BeginPlay/EndPlay
//VideoCallPlayerController.h
...
UCLASS()
class AGORAVIDEOCALL_API AVideoCallPlayerController : public APlayerController
{
GENERATED_BODY()
public:
...
void BeginPlay() override;
void EndPlay(const EEndPlayReason::Type EndPlayReason) override;
...
};
//VideoCallPlayerController.cpp
void AVideoCallPlayerController::BeginPlay()
{
Super::BeginPlay();
//initialize wigets
if (wEnterChannelWidget) // Check if the Asset is assigned in the blueprint.
{
// Create the widget and store it.
if (!EnterChannelWidget)
{
EnterChannelWidget = CreateWidget<UEnterChannelWidget>(this, wEnterChannelWidget);
EnterChannelWidget->SetVideoCallPlayerController(this);
}
// now you can use the widget directly since you have a referance for it.
// Extra check to make sure the pointer holds the widget.
if (EnterChannelWidget)
{
//let add it to the view port
EnterChannelWidget->AddToViewport();
}
//Show the Cursor.
bShowMouseCursor = true;
}
if (wVideoCallWidget)
{
if (!VideoCallWidget)
{
VideoCallWidget = CreateWidget<UVideoCallWidget>(this, wVideoCallWidget);
VideoCallWidget->SetVideoCallPlayerController(this);
}
if (VideoCallWidget)
{
VideoCallWidget->AddToViewport();
}
VideoCallWidget->SetVisibility(ESlateVisibility::Collapsed);
}
//create video call and switch on the EnterChannelWidget
VideoCallPtr = MakeUnique<VideoCall>();
FString Version = VideoCallPtr->GetVersion();
Version = "Agora version: " + Version;
EnterChannelWidget->UpdateVersionText(Version);
SwitchOnEnterChannelWidget(std::move(VideoCallPtr));
}
void AVideoCallPlayerController::EndPlay(const EEndPlayReason::Type EndPlayReason)
{
Super::EndPlay(EndPlayReason);
}
這時你可能注意到EnterChannelWidget和VideoCallWidget方法被標(biāo)記為錯誤，那是因為它們還沒有實現(xiàn)。我們將在接下來的步驟中實現(xiàn)它們。
增加 StartCall/EndCall
//VideoCallPlayerController.h
...
UCLASS()
class AGORAVIDEOCALL_API AVideoCallPlayerController : public APlayerController
{
GENERATED_BODY()
public:
...
void StartCall(
TUniquePtr<VideoCall> PassedVideoCallPtr,
const FString& ChannelName,
const FString& EncryptionKey,
const FString& EncryptionType
);
void EndCall(TUniquePtr<VideoCall> PassedVideoCallPtr);
...
};
//VideoCallPlayerController.cpp
void AVideoCallPlayerController::StartCall(
TUniquePtr<VideoCall> PassedVideoCallPtr,
const FString& ChannelName,
const FString& EncryptionKey,
const FString& EncryptionType)
{
SwitchOnVideoCallWidget(std::move(PassedVideoCallPtr));
VideoCallWidget->OnStartCall(
ChannelName,
EncryptionKey,
EncryptionType);
}
void AVideoCallPlayerController::EndCall(TUniquePtr<VideoCall> PassedVideoCallPtr)
{
SwitchOnEnterChannelWidget(std::move(PassedVideoCallPtr));
}
增加打開另一個小工具的方法
//VideoCallPlayerController.h
...
UCLASS()
class AGORAVIDEOCALL_API AVideoCallPlayerController : public APlayerController
{
GENERATED_BODY()
public:
...
void SwitchOnEnterChannelWidget(TUniquePtr<VideoCall> PassedVideoCallPtr);
void SwitchOnVideoCallWidget(TUniquePtr<VideoCall> PassedVideoCallPtr);
...
};
//VideoCallPlayerController.cpp
void AVideoCallPlayerController::SwitchOnEnterChannelWidget(TUniquePtr<VideoCall> PassedVideoCallPtr)
{
if (!EnterChannelWidget)
{
return;
}
EnterChannelWidget->SetVideoCall(std::move(PassedVideoCallPtr));
EnterChannelWidget->SetVisibility(ESlateVisibility::Visible);
}
void AVideoCallPlayerController::SwitchOnVideoCallWidget(TUniquePtr<VideoCall> PassedVideoCallPtr)
{
if (!VideoCallWidget)
{
return;
}
VideoCallWidget->SetVideoCall(std::move(PassedVideoCallPtr));
VideoCallWidget->SetVisibility(ESlateVisibility::Visible);
}
創(chuàng)建 EnterChannelWidget C++類
EnterChannelWidget是負(fù)責(zé)管理 UI 元素交互的。我們要創(chuàng)建一個新的 UserWidget 類型的類。在內(nèi)容瀏覽器中，按Add New按鈕，選擇New C++類，然后勾選Show All Classes按鈕，輸入UserWidget。按下 "下一步 "按鈕，為類設(shè)置一個名稱，EnterChannelWidget。
我們會得到如下代碼：
//EnterChannelWidget.h
#include "CoreMinimal.h"
#include "Blueprint/UserWidget.h"
#include "EnterChannelWidget.generated.h"
UCLASS()
class AGORAVIDEOCALL_API UEnterChannelWidget : public UUserWidget
{
GENERATED_BODY()
};
在EnterChannelWidget.h文件中增加一些必要的 include：
//EnterCahnnelWidget.h #include "CoreMinimal.h" #include "Blueprint/UserWidget.h" #include "Components/TextBlock.h" #include "Components/RichTextBlock.h" #include "Components/EditableTextBox.h" #include "Components/ComboBoxString.h" #include "Components/Button.h" #include "Components/Image.h" #include "VideoCall.h" #include "EnterChannelWidget.generated.h"
class AVideoCallPlayerController; //EnterCahnnelWidget.cpp #include "Blueprint/WidgetTree.h" #include "VideoCallPlayerController.h"
然后我們需要增加如下變量：
//EnterChannelWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UEnterChannelWidget : public UUserWidget
{
GENERATED_BODY()
public:
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UTextBlock* HeaderTextBlock = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UTextBlock* DescriptionTextBlock = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UEditableTextBox* ChannelNameTextBox = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UEditableTextBox* EncriptionKeyTextBox = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UTextBlock* EncriptionTypeTextBlock = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UComboBoxString* EncriptionTypeComboBox = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UButton* JoinButton = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UButton* TestButton = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UButton* VideoSettingsButton = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UTextBlock* ContactsTextBlock = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadOnly, meta = (BindWidget))
UTextBlock* BuildInfoTextBlock = nullptr;
...
};
這些變量用來公職 blueprint asset 中相關(guān)的 UI 元素。這里最重要的是 BindWidget 元屬性。通過將指向小部件的指針標(biāo)記為 BindWidget，你可以在你的 C++類的 Blueprint 子類中創(chuàng)建一個同名的小部件，并在運行時從 C++中訪問它。
同時，還要添加如下成員：
//EnterChannelWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UEnterChannelWidget : public UUserWidget
{
GENERATED_BODY()
...
public:
AVideoCallPlayerController* PlayerController = nullptr;
TUniquePtr<VideoCall> VideoCallPtr;
...
};
添加 Constructor 和 Construct/Destruct 方法
//EnterChannelWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UEnterChannelWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
UEnterChannelWidget(const FObjectInitializer& objectInitializer);
void NativeConstruct() override;
...
};
//EnterChannelWidget.cpp
UEnterChannelWidget::UEnterChannelWidget(const FObjectInitializer& objectInitializer)
: Super(objectInitializer)
{
}
void UEnterChannelWidget::NativeConstruct()
{
Super::NativeConstruct();
if (HeaderTextBlock)
HeaderTextBlock->SetText(FText::FromString("Enter a conference room name"));
if (DescriptionTextBlock)
DescriptionTextBlock->SetText(FText::FromString("If you are the first person to specify this name, \
the room will be created and you will\nbe placed in it. \
If it has already been created you will join the conference in progress"));
if (ChannelNameTextBox)
ChannelNameTextBox->SetHintText(FText::FromString("Channel Name"));
if (EncriptionKeyTextBox)
EncriptionKeyTextBox->SetHintText(FText::FromString("Encription Key"));
if (EncriptionTypeTextBlock)
EncriptionTypeTextBlock->SetText(FText::FromString("Enc Type:"));
if (EncriptionTypeComboBox)
{
EncriptionTypeComboBox->AddOption("aes-128");
EncriptionTypeComboBox->AddOption("aes-256");
EncriptionTypeComboBox->SetSelectedIndex(0);
}
if (JoinButton)
{
UTextBlock* JoinTextBlock = WidgetTree->ConstructWidget<UTextBlock>(UTextBlock::StaticClass());
JoinTextBlock->SetText(FText::FromString("Join"));
JoinButton->AddChild(JoinTextBlock);
JoinButton->OnClicked.AddDynamic(this, &UEnterChannelWidget::OnJoin);
}
if (ContactsTextBlock)
ContactsTextBlock->SetText(FText::FromString("agora.io Contact support: 400 632 6626"));
if (BuildInfoTextBlock)
BuildInfoTextBlock->SetText(FText::FromString(" "));
}
增加 Setter 方法
初始化 PlayerController 和 VideoCallPtr 變量
//EnterChannelWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UEnterChannelWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
void SetVideoCallPlayerController(AVideoCallPlayerController* VideoCallPlayerController);
void SetVideoCall(TUniquePtr<VideoCall> PassedVideoCallPtr);
...
};
//EnterChannelWidget.cpp
void UEnterChannelWidget::SetVideoCallPlayerController(AVideoCallPlayerController* VideoCallPlayerController)
{
PlayerController = VideoCallPlayerController;
}
void UEnterChannelWidget::SetVideoCall(TUniquePtr<VideoCall> PassedVideoCallPtr)
{
VideoCallPtr = std::move(PassedVideoCallPtr);
}
增加 BlueprintCallable方法
要對相應(yīng)的按鈕 "onButtonClick "事件做出反應(yīng)。
//EnterChannelWidget.h
..
UCLASS()
class AGORAVIDEOCALL_API UEnterChannelWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
UFUNCTION(BlueprintCallable)
void OnJoin();
...
};
//EnterChannelWidget.cpp
void UEnterChannelWidget::OnJoin()
{
if (!PlayerController || !VideoCallPtr)
{
return;
}
FString ChannelName = ChannelNameTextBox->GetText().ToString();
FString EncryptionKey = EncriptionKeyTextBox->GetText().ToString();
FString EncryptionType = EncriptionTypeComboBox->GetSelectedOption();
SetVisibility(ESlateVisibility::Collapsed);
PlayerController->StartCall(
std::move(VideoCallPtr),
ChannelName,
EncryptionKey,
EncryptionType);
}
增加 update 方法
//EnterChannelWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UEnterChannelWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
void UpdateVersionText(FString newValue);
...
};
//EnterChannelWidget.cpp
void UEnterChannelWidget::UpdateVersionText(FString newValue)
{
if (BuildInfoTextBlock)
BuildInfoTextBlock->SetText(FText::FromString(newValue));
}
創(chuàng)建 VideoViewWidget C++ 類
VideoViewWidget是一個存儲動態(tài)紋理并使用RGBA buffer 更新動態(tài)紋理的類，該類是從VideoCall OnLocalFrameCallback/OnRemoteFrameCallback函數(shù)中接收到的。
創(chuàng)建類和添加所需的 include
//VideoViewWidget.h #include "CoreMinimal.h" #include "Blueprint/UserWidget.h" #include "Components/Image.h" #include "VideoViewWidget.generated.h" //VideoViewWidget.cpp #include "EngineUtils.h" #include "Engine/Texture2D.h" #include <algorithm>
添加成員變量
- Buffer：用于存儲RGBA緩沖區(qū)、Width、Height和BufferSize的變量 - 視頻幀的參數(shù)。
- RenderTargetImage：允許你在UI中顯示Slate Brush或紋理或材質(zhì)的圖像小部件。
- RenderTargetTexture：動態(tài)紋理，我們將使用Buffer變量更新。
FUpdateTextureRegion2D：指定一個紋理的更新區(qū)域 刷子 - 一個包含如何繪制Slate元素的筆刷。我們將用它來繪制RenderTargetImage上的RenderTargetTexture。
//VideoViewWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoViewWidget : public UUserWidget
{
GENERATED_BODY()
public:
UPROPERTY(BlueprintReadOnly, meta = (BindWidget))
UImage* RenderTargetImage = nullptr;
UPROPERTY(EditDefaultsOnly)
UTexture2D* RenderTargetTexture = nullptr;
UTexture2D* CameraoffTexture = nullptr;
uint8* Buffer = nullptr;
uint32_t Width = 0;
uint32_t Height = 0;
uint32 BufferSize = 0;
FUpdateTextureRegion2D* UpdateTextureRegion = nullptr;
FSlateBrush Brush;
FCriticalSection Mutex;
...
};
覆蓋 NativeConstruct() 方法
在NativeConstruct中，我們將用默認(rèn)顏色初始化我們的圖像。為了初始化我們的RenderTargetTexture，我們需要使用CreateTransient調(diào)用創(chuàng)建動態(tài)紋理（Texture2D）。然后分配BufferSize為Width * Height * 4的BufferSize（用于存儲RGBA格式，每個像素可以用4個字節(jié)表示）。為了更新我們的紋理，我們可以使用UpdateTextureRegions函數(shù)。這個函數(shù)的輸入?yún)?shù)之一是我們的像素數(shù)據(jù)緩沖區(qū)。這樣，每當(dāng)我們修改像素數(shù)據(jù)緩沖區(qū)時，我們就需要調(diào)用這個函數(shù)來使變化在紋理中可見?，F(xiàn)在用我們的RenderTargetTexture初始化Brush變量，然后在RenderTargetImage widget中設(shè)置這個Brush。
//VideoViewWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoViewWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
void NativeConstruct() override;
...
};
//VideoViewWidget.cpp
void UVideoViewWidget::NativeConstruct()
{
Super::NativeConstruct();
Width = 640;
Height = 360;
RenderTargetTexture = UTexture2D::CreateTransient(Width, Height, PF_R8G8B8A8);
RenderTargetTexture->UpdateResource();
BufferSize = Width * Height * 4;
Buffer = new uint8[BufferSize];
for (uint32 i = 0; i < Width * Height; ++i)
{
Buffer[i * 4 + 0] = 0x32;
Buffer[i * 4 + 1] = 0x32;
Buffer[i * 4 + 2] = 0x32;
Buffer[i * 4 + 3] = 0xFF;
}
UpdateTextureRegion = new FUpdateTextureRegion2D(0, 0, 0, 0, Width, Height);
RenderTargetTexture->UpdateTextureRegions(0, 1, UpdateTextureRegion, Width * 4, (uint32)4, Buffer);
Brush.SetResourceObject(RenderTargetTexture);
RenderTargetImage->SetBrush(Brush);
}
覆蓋 NativeDestruct() 方法
//VideoViewWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoViewWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
void NativeDestruct() override;
...
};
//VideoViewWidget.cpp
void UVideoViewWidget::NativeDestruct()
{
Super::NativeDestruct();
delete[] Buffer;
delete UpdateTextureRegion;
}
覆蓋 NativeTick() 方法
如果UpdateTextureRegion Width或Height不等于memember的Width Height值，我們需要重新創(chuàng)建RenderTargetTexture以支持更新的值，并像Native Construct成員一樣重復(fù)初始化。否則只需用Buffer調(diào)用UpdateTextureRegions。
//VideoViewWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoViewWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
void NativeTick(const FGeometry& MyGeometry, float DeltaTime) override;
...
};
//VideoViewWidget.cpp
void UVideoViewWidget::NativeTick(const FGeometry& MyGeometry, float DeltaTime)
{
Super::NativeTick(MyGeometry, DeltaTime);
FScopeLock lock(&Mutex);
if (UpdateTextureRegion->Width != Width ||
UpdateTextureRegion->Height != Height)
{
auto NewUpdateTextureRegion = new FUpdateTextureRegion2D(0, 0, 0, 0, Width, Height);
auto NewRenderTargetTexture = UTexture2D::CreateTransient(Width, Height, PF_R8G8B8A8);
NewRenderTargetTexture->UpdateResource();
NewRenderTargetTexture->UpdateTextureRegions(0, 1, NewUpdateTextureRegion, Width * 4, (uint32)4, Buffer);
Brush.SetResourceObject(NewRenderTargetTexture);
RenderTargetImage->SetBrush(Brush);
//UClass's such as UTexture2D are automatically garbage collected when there is no hard pointer references made to that object.
//So if you just leave it and don't reference it elsewhere then it will be destroyed automatically.
FUpdateTextureRegion2D* TmpUpdateTextureRegion = UpdateTextureRegion;
RenderTargetTexture = NewRenderTargetTexture;
UpdateTextureRegion = NewUpdateTextureRegion;
delete TmpUpdateTextureRegion;
return;
}
RenderTargetTexture->UpdateTextureRegions(0, 1, UpdateTextureRegion, Width * 4, (uint32)4, Buffer);
}
增加 UpdateBuffer() 方法
通過調(diào)用來更新 Buffer 值。我們希望從 Agora SDK 線程接收到新的值。由于 UE4 的限制，我們將值保存到變量 Buffer 中，并在 NativeTick 方法中更新紋理，所以這里不調(diào)用UpdateTextureRegions。
//VideoViewWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoViewWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
void UpdateBuffer( uint8* RGBBuffer, uint32_t Width, uint32_t Height, uint32_t Size );
void ResetBuffer();
...
};
//VideoViewWidget.cpp
void UVideoViewWidget::UpdateBuffer(
uint8* RGBBuffer,
uint32_t NewWidth,
uint32_t NewHeight,
uint32_t NewSize)
{
FScopeLock lock(&Mutex);
if (!RGBBuffer)
{
return;
}
if (BufferSize == NewSize)
{
std::copy(RGBBuffer, RGBBuffer + NewSize, Buffer);
}
else
{
delete[] Buffer;
BufferSize = NewSize;
Width = NewWidth;
Height = NewHeight;
Buffer = new uint8[BufferSize];
std::copy(RGBBuffer, RGBBuffer + NewSize, Buffer);
}
}
void UVideoViewWidget::ResetBuffer()
{
for (uint32 i = 0; i < Width * Height; ++i)
{
Buffer[i * 4 + 0] = 0x32;
Buffer[i * 4 + 1] = 0x32;
Buffer[i * 4 + 2] = 0x32;
Buffer[i * 4 + 3] = 0xFF;
}
}
創(chuàng)建 VideoCallViewWidget C++類
VideoCallViewWidget 類的作用是顯示本地和遠(yuǎn)程用戶的視頻。我們需要兩個 VideoViewWidget 小部件，一個用來顯示來自本地攝像頭的視頻，另一個用來顯示從遠(yuǎn)程用戶收到的視頻（假設(shè)我們只支持一個遠(yuǎn)程用戶）。
創(chuàng)建類和添加所需的 include
像之前那樣創(chuàng)建Widget C++類，添加所需的include。
//VideoCallViewWidget.h #include "CoreMinimal.h" #include "Blueprint/UserWidget.h" #include "Components/SizeBox.h" #include "VideoViewWidget.h" #include "VideoCallViewWidget.generated.h" //VideoCallViewWidget.cpp #include "Components/CanvasPanelSlot.h"
添加成員變量
//VideoCallViewWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallViewWidget : public UUserWidget
{
GENERATED_BODY()
public:
UPROPERTY(BlueprintReadOnly, meta = (BindWidget))
UVideoViewWidget* MainVideoViewWidget = nullptr;
UPROPERTY(BlueprintReadOnly, meta = (BindWidget))
USizeBox* MainVideoSizeBox = nullptr;
UPROPERTY(BlueprintReadOnly, meta = (BindWidget))
UVideoViewWidget* AdditionalVideoViewWidget = nullptr;
UPROPERTY(BlueprintReadOnly, meta = (BindWidget))
USizeBox* AdditionalVideoSizeBox = nullptr;
public:
int32 MainVideoWidth = 0;
int32 MainVideoHeight = 0;
...
};
覆蓋 NativeTick() 方法
``
//VideoCallViewWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallViewWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
void NativeTick(const FGeometry& MyGeometry, float DeltaTime) override;
...
};
//VideoCallViewWidget.cpp
void UVideoCallViewWidget::NativeTick(const FGeometry& MyGeometry, float DeltaTime)
{
Super::NativeTick(MyGeometry, DeltaTime);
auto ScreenSize = MyGeometry.GetLocalSize();
if (MainVideoHeight != 0)
{
float AspectRatio = 0;
AspectRatio = MainVideoWidth / (float)MainVideoHeight;
auto MainVideoGeometry = MainVideoViewWidget->GetCachedGeometry();
auto MainVideoScreenSize = MainVideoGeometry.GetLocalSize();
if (MainVideoScreenSize.X == 0)
{
return;
}
auto NewMainVideoHeight = MainVideoScreenSize.Y;
auto NewMainVideoWidth = AspectRatio * NewMainVideoHeight;
MainVideoSizeBox->SetMinDesiredWidth(NewMainVideoWidth);
MainVideoSizeBox->SetMinDesiredHeight(NewMainVideoHeight);
UCanvasPanelSlot* CanvasSlot = Cast<UCanvasPanelSlot>(MainVideoSizeBox->Slot);
CanvasSlot->SetAutoSize(true);
FVector2D NewPosition;
NewPosition.X = -NewMainVideoWidth / 2;
NewPosition.Y = -NewMainVideoHeight / 2;
CanvasSlot->SetPosition(NewPosition);
}
}
更新 UpdateMainVideoBuffer/UpdateAdditionalVideoBuffe
//VideoCallViewWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallViewWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
void UpdateMainVideoBuffer( uint8* RGBBuffer, uint32_t Width, uint32_t Height, uint32_t Size);
void UpdateAdditionalVideoBuffer( uint8* RGBBuffer, uint32_t Width, uint32_t Height, uint32_t Size);
void ResetBuffers();
...
};
//VideoCallViewWidget.cpp
void UVideoCallViewWidget::UpdateMainVideoBuffer(
uint8* RGBBuffer,
uint32_t Width,
uint32_t Height,
uint32_t Size)
{
if (!MainVideoViewWidget)
{
return;
}
MainVideoWidth = Width;
MainVideoHeight = Height;
MainVideoViewWidget->UpdateBuffer(RGBBuffer, Width, Height, Size);
}
void UVideoCallViewWidget::UpdateAdditionalVideoBuffer(
uint8* RGBBuffer,
uint32_t Width,
uint32_t Height,
uint32_t Size)
{
if (!AdditionalVideoViewWidget)
{
return;
}
AdditionalVideoViewWidget->UpdateBuffer(RGBBuffer, Width, Height, Size);
}
void UVideoCallViewWidget::ResetBuffers()
{
if (!MainVideoViewWidget || !AdditionalVideoViewWidget)
{
return;
}
MainVideoViewWidget->ResetBuffer();
AdditionalVideoViewWidget->ResetBuffer();
}
創(chuàng)建 VideoCallWidget C++ 類
VideoCallWidget 類作為示例應(yīng)用程序的音頻/視頻調(diào)用小部件。它包含以下控件，與藍(lán)圖資產(chǎn)中的UI元素綁定。
創(chuàng)建類和添加所需的include
像之前那樣創(chuàng)建Widget C++類，添加必要的include和轉(zhuǎn)發(fā)聲明。
//VideoCallWidget.h #include "CoreMinimal.h" #include "Blueprint/UserWidget.h" #include "Templates/UniquePtr.h" #include "Components/Image.h" #include "Components/Button.h" #include "Engine/Texture2D.h" #include "VideoCall.h" #include "VideoCallViewWidget.h" #include "VideoCallWidget.generated.h" class AVideoCallPlayerController; class UVideoViewWidget; //VideoCallWidget.cpp #include "Kismet/GameplayStatics.h" #include "UObject/ConstructorHelpers.h" #include "Components/CanvasPanelSlot.h" #include "VideoViewWidget.h" #include "VideoCallPlayerController.h"
增加成員變量
//VideoCallWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallWidget : public UUserWidget
{
GENERATED_BODY()
public:
AVideoCallPlayerController* PlayerController = nullptr;
public:
UPROPERTY(BlueprintReadOnly, meta = (BindWidget))
UVideoCallViewWidget* VideoCallViewWidget = nullptr;
//Buttons
UPROPERTY(BlueprintReadOnly, meta = (BindWidget))
UButton* EndCallButton = nullptr;
UPROPERTY(BlueprintReadOnly, meta = (BindWidget))
UButton* MuteLocalAudioButton = nullptr;
UPROPERTY(BlueprintReadOnly, meta = (BindWidget))
UButton* VideoModeButton = nullptr;
//Button textures
int32 ButtonSizeX = 96;
int32 ButtonSizeY = 96;
UTexture2D* EndCallButtonTexture = nullptr;
UTexture2D* AudioButtonMuteTexture = nullptr;
UTexture2D* AudioButtonUnmuteTexture = nullptr;
UTexture2D* VideomodeButtonCameraoffTexture = nullptr;
UTexture2D* VideomodeButtonCameraonTexture = nullptr;
TUniquePtr<VideoCall> VideoCallPtr;
...
};
初始化VideoCallWidget
為每個按鈕找到asset圖像，并將其分配到相應(yīng)的紋理。然后用紋理初始化每個按鈕。
//VideoCallWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallWidget : public UUserWidget
{
GENERATED_BODY()
public:
..
UVideoCallWidget(const FObjectInitializer& ObjectInitializer);
void NativeConstruct() override;
void NativeDestruct() override;
private:
void InitButtons();
...
};
//VideoCallWidget.cpp
void UVideoCallWidget::NativeConstruct()
{
Super::NativeConstruct();
InitButtons();
}
void UVideoCallWidget::NativeDestruct()
{
Super::NativeDestruct();
if (VideoCallPtr)
{
VideoCallPtr->StopCall();
}
}
UVideoCallWidget::UVideoCallWidget(const FObjectInitializer& ObjectInitializer)
: Super(ObjectInitializer)
{
static ConstructorHelpers::FObjectFinder<UTexture2D>
EndCallButtonTextureFinder(TEXT("Texture'/Game/ButtonTextures/hangup.hangup'"));
if (EndCallButtonTextureFinder.Succeeded())
{
EndCallButtonTexture = EndCallButtonTextureFinder.Object;
}
static ConstructorHelpers::FObjectFinder<UTexture2D>
AudioButtonMuteTextureFinder(TEXT("Texture'/Game/ButtonTextures/mute.mute'"));
if (AudioButtonMuteTextureFinder.Succeeded())
{
AudioButtonMuteTexture = AudioButtonMuteTextureFinder.Object;
}
static ConstructorHelpers::FObjectFinder<UTexture2D>
AudioButtonUnmuteTextureFinder(TEXT("Texture'/Game/ButtonTextures/unmute.unmute'"));
if (AudioButtonUnmuteTextureFinder.Succeeded())
{
AudioButtonUnmuteTexture = AudioButtonUnmuteTextureFinder.Object;
}
static ConstructorHelpers::FObjectFinder<UTexture2D>
VideomodeButtonCameraonTextureFinder(TEXT("Texture'/Game/ButtonTextures/cameraon.cameraon'"));
if (VideomodeButtonCameraonTextureFinder.Succeeded())
{
VideomodeButtonCameraonTexture = VideomodeButtonCameraonTextureFinder.Object;
}
static ConstructorHelpers::FObjectFinder<UTexture2D>
VideomodeButtonCameraoffTextureFinder(TEXT("Texture'/Game/ButtonTextures/cameraoff.cameraoff'"));
if (VideomodeButtonCameraoffTextureFinder.Succeeded())
{
VideomodeButtonCameraoffTexture = VideomodeButtonCameraoffTextureFinder.Object;
}
}
void UVideoCallWidget::InitButtons()
{
if (EndCallButtonTexture)
{
EndCallButton->WidgetStyle.Normal.SetResourceObject(EndCallButtonTexture);
EndCallButton->WidgetStyle.Normal.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
EndCallButton->WidgetStyle.Normal.DrawAs = ESlateBrushDrawType::Type::Image;
EndCallButton->WidgetStyle.Hovered.SetResourceObject(EndCallButtonTexture);
EndCallButton->WidgetStyle.Hovered.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
EndCallButton->WidgetStyle.Hovered.DrawAs = ESlateBrushDrawType::Type::Image;
EndCallButton->WidgetStyle.Pressed.SetResourceObject(EndCallButtonTexture);
EndCallButton->WidgetStyle.Pressed.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
EndCallButton->WidgetStyle.Pressed.DrawAs = ESlateBrushDrawType::Type::Image;
}
EndCallButton->OnClicked.AddDynamic(this, &UVideoCallWidget::OnEndCall);
SetAudioButtonToMute();
MuteLocalAudioButton->OnClicked.AddDynamic(this, &UVideoCallWidget::OnMuteLocalAudio);
SetVideoModeButtonToCameraOff();
VideoModeButton->OnClicked.AddDynamic(this, &UVideoCallWidget::OnChangeVideoMode);
}
添加按鈕紋理
在演示程序中找到目錄Content/ButtonTextures（你不必打開項目，只需在文件系統(tǒng)中找到這個文件夾即可）。所有的按鈕紋理都存儲在那里。在你的項目內(nèi)容中創(chuàng)建一個名為ButtonTextures的新目錄，將所有的按鈕圖片拖放到那里，讓它們在你的項目中可用。
添加Setters
//VideoCallWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallWidget : public UUserWidget
{
GENERATED_BODY()
...
public:
void SetVideoCallPlayerController(AVideoCallPlayerController* VideoCallPlayerController);
void SetVideoCall(TUniquePtr<VideoCall> PassedVideoCallPtr);
...
};
//VideoCallWidget.cpp
void UVideoCallWidget::SetVideoCallPlayerController(AVideoCallPlayerController* VideoCallPlayerController)
{
PlayerController = VideoCallPlayerController;
}
void UVideoCallWidget::SetVideoCall(TUniquePtr<VideoCall> PassedVideoCallPtr)
{
VideoCallPtr = std::move(PassedVideoCallPtr);
}
增加用來更新 view 的方法
//VideoCallWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallWidget : public UUserWidget
{
GENERATED_BODY()
...
private:
void SetVideoModeButtonToCameraOff();
void SetVideoModeButtonToCameraOn();
void SetAudioButtonToMute();
void SetAudioButtonToUnMute();
...
};
//VideoCallWidget.cpp
void UVideoCallWidget::SetVideoModeButtonToCameraOff()
{
if (VideomodeButtonCameraoffTexture)
{
VideoModeButton->WidgetStyle.Normal.SetResourceObject(VideomodeButtonCameraoffTexture);
VideoModeButton->WidgetStyle.Normal.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
VideoModeButton->WidgetStyle.Normal.DrawAs = ESlateBrushDrawType::Type::Image;
VideoModeButton->WidgetStyle.Hovered.SetResourceObject(VideomodeButtonCameraoffTexture);
VideoModeButton->WidgetStyle.Hovered.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
VideoModeButton->WidgetStyle.Hovered.DrawAs = ESlateBrushDrawType::Type::Image;
VideoModeButton->WidgetStyle.Pressed.SetResourceObject(VideomodeButtonCameraoffTexture);
VideoModeButton->WidgetStyle.Pressed.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
VideoModeButton->WidgetStyle.Pressed.DrawAs = ESlateBrushDrawType::Type::Image;
}
}
void UVideoCallWidget::SetVideoModeButtonToCameraOn()
{
if (VideomodeButtonCameraonTexture)
{
VideoModeButton->WidgetStyle.Normal.SetResourceObject(VideomodeButtonCameraonTexture);
VideoModeButton->WidgetStyle.Normal.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
VideoModeButton->WidgetStyle.Normal.DrawAs = ESlateBrushDrawType::Type::Image;
VideoModeButton->WidgetStyle.Hovered.SetResourceObject(VideomodeButtonCameraonTexture);
VideoModeButton->WidgetStyle.Hovered.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
VideoModeButton->WidgetStyle.Hovered.DrawAs = ESlateBrushDrawType::Type::Image;
VideoModeButton->WidgetStyle.Pressed.SetResourceObject(VideomodeButtonCameraonTexture);
VideoModeButton->WidgetStyle.Pressed.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
VideoModeButton->WidgetStyle.Pressed.DrawAs = ESlateBrushDrawType::Type::Image;
}
}
void UVideoCallWidget::SetAudioButtonToMute()
{
if (AudioButtonMuteTexture)
{
MuteLocalAudioButton->WidgetStyle.Normal.SetResourceObject(AudioButtonMuteTexture);
MuteLocalAudioButton->WidgetStyle.Normal.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
MuteLocalAudioButton->WidgetStyle.Normal.DrawAs = ESlateBrushDrawType::Type::Image;
MuteLocalAudioButton->WidgetStyle.Hovered.SetResourceObject(AudioButtonMuteTexture);
MuteLocalAudioButton->WidgetStyle.Hovered.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
MuteLocalAudioButton->WidgetStyle.Hovered.DrawAs = ESlateBrushDrawType::Type::Image;
MuteLocalAudioButton->WidgetStyle.Pressed.SetResourceObject(AudioButtonMuteTexture);
MuteLocalAudioButton->WidgetStyle.Pressed.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
MuteLocalAudioButton->WidgetStyle.Pressed.DrawAs = ESlateBrushDrawType::Type::Image;
}
}
void UVideoCallWidget::SetAudioButtonToUnMute()
{
if (AudioButtonUnmuteTexture)
{
MuteLocalAudioButton->WidgetStyle.Normal.SetResourceObject(AudioButtonUnmuteTexture);
MuteLocalAudioButton->WidgetStyle.Normal.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
MuteLocalAudioButton->WidgetStyle.Normal.DrawAs = ESlateBrushDrawType::Type::Image;
MuteLocalAudioButton->WidgetStyle.Hovered.SetResourceObject(AudioButtonUnmuteTexture);
MuteLocalAudioButton->WidgetStyle.Hovered.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
MuteLocalAudioButton->WidgetStyle.Hovered.DrawAs = ESlateBrushDrawType::Type::Image;
MuteLocalAudioButton->WidgetStyle.Pressed.SetResourceObject(AudioButtonUnmuteTexture);
MuteLocalAudioButton->WidgetStyle.Pressed.ImageSize = FVector2D(ButtonSizeX, ButtonSizeY);
MuteLocalAudioButton->WidgetStyle.Pressed.DrawAs = ESlateBrushDrawType::Type::Image;
}
}
增加 OnStartCall 方法
//VideoCallWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
void OnStartCall( const FString& ChannelName, const FString& EncryptionKey, const FString& EncryptionType );
...
};
//VideoCallWidget.cpp
void UVideoCallWidget::OnStartCall(
const FString& ChannelName,
const FString& EncryptionKey,
const FString& EncryptionType)
{
if (!VideoCallPtr)
{
return;
}
auto OnLocalFrameCallback = [this](
std::uint8_t* Buffer,
std::uint32_t Width,
std::uint32_t Height,
std::uint32_t Size)
{
VideoCallViewWidget->UpdateAdditionalVideoBuffer(Buffer, Width, Height, Size);
};
VideoCallPtr->RegisterOnLocalFrameCallback(OnLocalFrameCallback);
auto OnRemoteFrameCallback = [this](
std::uint8_t* Buffer,
std::uint32_t Width,
std::uint32_t Height,
std::uint32_t Size)
{
VideoCallViewWidget->UpdateMainVideoBuffer(Buffer, Width, Height, Size);
};
VideoCallPtr->RegisterOnRemoteFrameCallback(OnRemoteFrameCallback);
VideoCallPtr->StartCall(ChannelName, EncryptionKey, EncryptionType);
}
增加 OnEndCall方法
//VideoCallWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
UFUNCTION(BlueprintCallable)
void OnEndCall();
...
};
//VideoCallWidget.cpp
void UVideoCallWidget::OnEndCall()
{
if (VideoCallPtr)
{
VideoCallPtr->StopCall();
}
if (VideoCallViewWidget)
{
VideoCallViewWidget->ResetBuffers();
}
if (PlayerController)
{
SetVisibility(ESlateVisibility::Collapsed);
PlayerController->EndCall(std::move(VideoCallPtr));
}
}
增加 OnMuteLocalAudio 方法
//VideoCallWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
UFUNCTION(BlueprintCallable)
void OnMuteLocalAudio();
...
};
//VideoCallWidget.cpp
void UVideoCallWidget::OnMuteLocalAudio()
{
if (!VideoCallPtr)
{
return;
}
if (VideoCallPtr->IsLocalAudioMuted())
{
VideoCallPtr->MuteLocalAudio(false);
SetAudioButtonToMute();
}
else
{
VideoCallPtr->MuteLocalAudio(true);
SetAudioButtonToUnMute();
}
}
增加 OnChangeVideoMode方法
//VideoCallWidget.h
...
UCLASS()
class AGORAVIDEOCALL_API UVideoCallWidget : public UUserWidget
{
GENERATED_BODY()
public:
...
UFUNCTION(BlueprintCallable)
void OnChangeVideoMode();
...
};
//VideoCallWidget.cpp
void UVideoCallWidget::OnChangeVideoMode()
{
if (!VideoCallPtr)
{
return;
}
if (!VideoCallPtr->IsLocalVideoMuted())
{
VideoCallPtr->MuteLocalVideo(true);
SetVideoModeButtonToCameraOn();
}
else
{
VideoCallPtr->EnableVideo(true);
VideoCallPtr->MuteLocalVideo(false);
SetVideoModeButtonToCameraOff();
}
}
增加 Blueprint 類
確保C++代碼正確編譯。沒有成功編譯的項目，你將無法進(jìn)行下一步的操作。如果你已經(jīng)成功地編譯了C++代碼，但在虛幻編輯器中仍然沒有看到所需的類，請嘗試重新打開項目。
創(chuàng)建 BP_EnterChannelWidget Blueprint Asset。
創(chuàng)建一個 UEnterChannelWidget 的 Blueprint，右鍵點擊內(nèi)容，選擇用戶界面菜單并選擇 Widget Blueprint。
更改這個新的用戶小工具的類的父類。打開 Blueprint，會出現(xiàn) UMG 編輯器界面，默認(rèn)情況下 Designer 選項卡是打開的。點擊圖形按鈕（右上角按鈕），選擇 "類設(shè)置"。在面板 "Details "中，點擊下拉列表 "父類"，選擇之前創(chuàng)建的C++ 類 UEnterChannelWidget?，F(xiàn)在返回到設(shè)計頁面。調(diào)色板窗口包含幾種不同類型的小部件，你可以用它們來構(gòu)造你的 UI 元素。找到 Text、Editable Text、Button 和 ComboBox（String）元素，然后將它們拖到工作區(qū)，如圖中所示。然后進(jìn)入 "EnterChannelWidget.h "文件中的 UEnterChannelWidget 的定義，查看成員變量的名稱和對應(yīng)的類型（UTextBlock、EditableTextBox、UButton和UComboBoxString）。返回到 BP_VideoCallWiewVidget 編輯器中，給你拖動的UI元素設(shè)置相同的名稱。你可以通過點擊元素并在 "詳細(xì)信息 "面板中更改名稱來完成。嘗試編譯藍(lán)圖。如果你忘了添加什么東西，或者在你的UserWidget類中出現(xiàn)了widget名稱/類型不匹配的情況，你會出現(xiàn)一個錯誤。
保存到文件夾中，例如 /Content/Widgets/BP_EnterChannelWidget.uasset。
創(chuàng)建 BP_VideoViewWidget Asset。
設(shè)定圖片的錨點
創(chuàng)建 BP_VideoCallViewWidget Asset
創(chuàng)建 BP VideoCallViewWidget Asset ，將父類設(shè)置為 UVideoCallViewWidget，并添加 BP VideoViewWidget 類型的 UI 元素MainVideoViewWidget 和ExtendedVideoViewWidget。同時添加 SizeBox 類型的 MainVideoSizeBox 和 AdditionalVideoSizeBox UI 元素。
創(chuàng)建 BP_VideoCallWidget Asset
創(chuàng)建BPVideoCallWidget Asset，將父類設(shè)置為UVideoCallWidget，在 Palette UI 元素BPVideoCallViewWidget 中找到并添加名稱為VideoCallViewWidget，添加 EndCallButton、MuteLocalAudioButton 和 VideoModeButton 按鈕。
創(chuàng)建 BP_VideoCallPlayerController blueprint asset
現(xiàn)在是創(chuàng)建 BPVideoCallPlayerPlayerController blueprint asset 的時候了，基于我們前面描述的 AVideoCallPlayerPlayerController 類，創(chuàng)建 BPVideoCallPlayerController 藍(lán)圖資產(chǎn)。
創(chuàng)建一個AVideoCallPlayerPlayerController的bluepringt。右鍵點擊內(nèi)容，按Add New按鈕，選擇Blueprint類，在窗口中選擇父類，在Pick parent類進(jìn)入All classes部分，找到VideoCallPlayerController類。
現(xiàn)在將我們之前創(chuàng)建的小部件分配給PlayerController，如下圖所示。
將其保存到文件夾，例如 /Content/Widgets/BP_VideoCallPlayerController.uasset。
創(chuàng)建 BP_AgoraVideoCallGameModeBase Asset
創(chuàng)建一個 AVideoCallPlayerController 的 Blueprint，右鍵點擊內(nèi)容，按 Add New 按鈕，選擇 Blueprint 類，在 Pick parent class 窗口中選擇 Game Mode Base Class。這是所有游戲模式的父類。
修改 GameMode
現(xiàn)在你需要設(shè)置你的自定義 GameMode 類和玩家控制器。到世界設(shè)置中，設(shè)置指定的變量：
指定項目的設(shè)置
進(jìn)入 Edit->Project settings，打開 Maps & Modes。設(shè)定默認(rèn)參數(shù)：
總結(jié)
到此這篇關(guān)于C++ 在 Unreal 中為游戲增加實時音視頻互動的文章就介紹到這了,更多相關(guān)C++ 游戲增加音視頻互動內(nèi)容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！
相關(guān)文章
C語言數(shù)據(jù)結(jié)構(gòu) 雙向鏈表的建立與基本操作
這篇文章主要介紹了C語言數(shù)據(jù)結(jié)構(gòu) 雙向鏈表的建立與基本操作的相關(guān)資料,需要的朋友可以參考下2017-03-03
MySQL系列教程之使用C語言來連接數(shù)據(jù)庫
c語言操作Mysql數(shù)據(jù)庫,主要就是為了實現(xiàn)對數(shù)據(jù)庫的增、刪、改、查等操作,下面這篇文章主要給大家介紹了關(guān)于MySQL系列教程之使用C語言來連接數(shù)據(jù)庫的相關(guān)資料,文中通過實例代碼介紹的非常詳細(xì),需要的朋友可以參考下2022-09-09
C++關(guān)于構(gòu)造函數(shù)可向父類或者本類傳參的講解
今天小編就為大家分享一篇關(guān)于C++關(guān)于構(gòu)造函數(shù)可向父類或者本類傳參的講解，小編覺得內(nèi)容挺不錯的，現(xiàn)在分享給大家，具有很好的參考價值，需要的朋友一起跟隨小編來看看吧2018-12-12
這篇文章主要為大家詳細(xì)介紹了Linux下用C語言實現(xiàn)小老鼠推箱子的游戲，文中示例代碼介紹的非常詳細(xì)，具有一定的參考價值，感興趣的小伙伴們可以參考一下2020-07-07
QT使用QComBox和QLineEdit實現(xiàn)模糊查詢功能
模糊查詢是指根據(jù)用戶輸入的文本,在下拉框的選項中進(jìn)行模糊匹配,并動態(tài)地顯示匹配的選項,本文將使用QComBox和QLineEdit實現(xiàn)模糊查詢功能,需要的可以參考下2023-11-11
Qt?QGraphicsItem?移動時出現(xiàn)殘影問題記錄
自定義QGraphicsItem時,繪制rect,對象移動時出現(xiàn)殘影的問題記錄,本文給大家介紹Qt?QGraphicsItem?移動時出現(xiàn)殘影問題記錄,感興趣的朋友跟隨小編一起看看吧2024-06-06
昨天在使用Visual Studio 2022編寫C語言程序時遇到了scanf問題,對于vs的編譯器來說scanf是不安全的,編譯器通過不了scanf,本文就來介紹一下解決方法,感興趣的可以了解一下2023-12-12
VC6.0實現(xiàn)讀取Excel數(shù)據(jù)的方法
這篇文章主要介紹了VC6.0實現(xiàn)讀取Excel數(shù)據(jù)的方法,非常實用的功能,需要的朋友可以參考下2014-07-07
當(dāng)年學(xué)習(xí)C語言的第一門課就提到過標(biāo)記(Token)的概念，不過，相信在多年之后你再次聽到這個術(shù)語時會一臉懵逼，比如我。因此特地翻了翻資料，整理下來這些筆記，希望對大家有所幫助2022-11-11
C語言數(shù)據(jù)結(jié)構(gòu)與算法之時間空間復(fù)雜度入門
這篇文章主要為大家介紹了C語言數(shù)據(jù)結(jié)構(gòu)與算法之時間空間復(fù)雜度的入門教程示例，有需要的朋友可以借鑒參考下，希望能夠有所幫助，祝大家多多進(jìn)步2022-02-02