wpf實現(xiàn)超低延遲的RTMP或RTSP播放

更新時間：2024年04月11日 08:24:17 作者：音視頻牛哥

這篇文章主要為大家詳細介紹了wpf如何實現(xiàn)超低延遲的RTMP或RTSP播放,文中的示例代碼講解詳細,感興趣的小伙伴可以跟隨小編一起學習一下

?技術背景

我們在做Windows平臺RTMP和RTSP播放模塊對接的時候，有開發(fā)者需要在wpf下調(diào)用，如果要在wpf下使用，只需要參考C#的對接demo即可，唯一不同的是，視頻流數(shù)據(jù)顯示的話，要么通過控件模式，要么可以讓RTMP、RTSP播放模塊回調(diào)rgb數(shù)據(jù)上來，在wpf直接繪制即可。

技術實現(xiàn)

本文以大牛直播SDK的Windows平臺SmartPlayer為例，回調(diào)數(shù)據(jù)的模式，其他不再說明，只要處理好上來的數(shù)據(jù)就好：

播放之前，設置回調(diào)，選擇NT_SP_E_VIDEO_FRAME_FORMAT_RGB32：

video_frame_call_back_ = new SP_SDKVideoFrameCallBack(SetVideoFrameCallBack);

NTSmartPlayerSDK.NT_SP_SetVideoFrameCallBack(player_handle_, (Int32)NT.NTSmartPlayerDefine.NT_SP_E_VIDEO_FRAME_FORMAT.NT_SP_E_VIDEO_FRAME_FORMAT_RGB32, IntPtr.Zero, video_frame_call_back_);

處理rgb數(shù)據(jù)回調(diào)：

       /*
        * nt_player_wrapper.cs
        * Author: daniusdk.com
        */
        public void SetVideoFrameCallBack(IntPtr handle, IntPtr userData, UInt32 status, IntPtr frame)
        {
            if (frame == IntPtr.Zero)
            {
                return;
            }

            //如需直接處理RGB數(shù)據(jù)，請參考以下流程
            NT_SP_VideoFrame video_frame = (NT_SP_VideoFrame)Marshal.PtrToStructure(frame, typeof(NT_SP_VideoFrame));

            if (video_frame.format_ != (Int32)NT.NTSmartPlayerDefine.NT_SP_E_VIDEO_FRAME_FORMAT.NT_SP_E_VIDEO_FRAME_FORMAT_RGB32)
                return;

            NT_SP_VideoFrame pVideoFrame = new NT_SP_VideoFrame();

            pVideoFrame.format_ = video_frame.format_;
            pVideoFrame.width_ = video_frame.width_;
            pVideoFrame.height_ = video_frame.height_;

            pVideoFrame.timestamp_ = video_frame.timestamp_;
            pVideoFrame.stride0_ = video_frame.stride0_;
            pVideoFrame.stride1_ = video_frame.stride1_;
            pVideoFrame.stride2_ = video_frame.stride2_;
            pVideoFrame.stride3_ = video_frame.stride3_;


            Int32 argb_size = video_frame.stride0_ * video_frame.height_;

            pVideoFrame.plane0_ = Marshal.AllocHGlobal(argb_size);
            CopyMemory(pVideoFrame.plane0_, video_frame.plane0_, (UInt32)argb_size);
        }

另外一種，可以用picturebox，在MainWindow.xaml 做以下設置：

        <WindowsFormsHost HorizontalAlignment="Left" Height="338" Margin="10,10,0,0" VerticalAlignment="Top" Width="480" Background="Black">
            <wf:PictureBox x:Name="RealPlayWnd"></wf:PictureBox>
        </WindowsFormsHost>

為了便于多實例集成參考，以播放2路為例（一路2560*1440，一路1920*1080）：

具體實現(xiàn)如下：

       /*
        * MainWindow.xaml.cs
        * Author: daniusdk.com
        */
        public MainWindow()
        {
            InitializeComponent();

            if (!InitSDK())
                return;

            UIDispatcher = Dispatcher.CurrentDispatcher;

            player1_ = new nt_player_wrapper(RealPlayWnd, UIDispatcher);
            player1_.EventGetPlayerEventMsg += new DelGetPlayerEventMsg(GetPlayerEventMsgInfo);
            player1_.EventGetVideoSize += new DelGetVideoSize(GetVideoSize);

            player2_ = new nt_player_wrapper(RealPlayWnd1, UIDispatcher);
            player2_.EventGetPlayerEventMsg += new DelGetPlayerEventMsg(GetPlayerEventMsgInfo);
            player2_.EventGetVideoSize += new DelGetVideoSize(GetVideoSize);
        }

        private void GetPlayerEventMsgInfo(IntPtr handle, String msg)
        {
            this.Dispatcher.Invoke((Action)delegate()
            {
                event_label.Content = msg;
            });
        }

        private void GetVideoSize(IntPtr handle, String size)
        {
            this.Dispatcher.Invoke((Action)delegate()
            {
                video_size.Content = size;
            });
        }

        private bool InitSDK()
        {
            if (!is_player_sdk_init_)
            {
                UInt32 isPlayerInited = NT.NTSmartPlayerSDK.NT_SP_Init(0, IntPtr.Zero);
                if (isPlayerInited != 0)
                {
                    MessageBox.Show("調(diào)用NT_SP_Init失敗..");
                    return false;
                }

                is_player_sdk_init_ = true;
            }

            return true;
        }

        private void Button_Click_1(object sender, RoutedEventArgs e)
        {
            if (!player1_.IsPlaying())
            {
                player1_.SetBuffer(0);
                bool is_mute = true;

                if (!player1_.StartPlay("rtsp://admin:daniulive12345@192.168.0.120:554/h264/ch1/main/av_stream", false, is_mute))
                    return;

                btn_playback1.Content = "停止播放";
            }
            else
            {
                player1_.StopPlay();
                btn_playback1.Content = "開始播放";
            }
        }

        private void Button_Click_2(object sender, RoutedEventArgs e)
        {
            if (!player2_.IsPlaying())
            {
                player2_.SetBuffer(0);
                bool is_mute = true;

                if (!player2_.StartPlay("rtsp://admin:admin123456@192.168.0.121:554/cam/realmonitor?channel=1&subtype=0", false, is_mute))
                    return;

                btn_playback2.Content = "停止播放";
            }
            else
            {
                player2_.StopPlay();
                btn_playback2.Content = "開始播放";
            }
        }

        protected override void OnClosing(System.ComponentModel.CancelEventArgs e)
        {
            if (MessageBox.Show("確定要關閉窗口嗎？", "確認", MessageBoxButton.YesNo) != MessageBoxResult.Yes)
            {
                // 如果用戶選擇“否”，取消關閉
                e.Cancel = true;
            }

            if (player1_.IsPlaying())
            {
                player1_.StopPlay();
            }

            player1_.Dispose();

            if (player2_.IsPlaying())
            {
                player2_.StopPlay();
            }

            player2_.Dispose();

            if (is_player_sdk_init_)
            {
                NTSmartPlayerSDK.NT_SP_UnInit();
                is_player_sdk_init_ = false;
            }    

            base.OnClosing(e);
        }

延遲依舊毫秒級，CPU占用如下，如果用硬解碼，體驗會更好：