zoukankan      html  css  js  c++  java
  • MPSoC VCU调试方法和调试流程

    Author

    Hank FU hankf@xilinx.com 2021-09-29

    概要介绍

    MPSoC VCU在很多产品中得到了应用。在不同产品的调试过程中,有一些共同的办法。

    首先看看Video子系统的简单框图,其中包含视频输入、编解码、视频输出等模块。视频输入模块,硬件可能是HDMI-RX和FrmBufWrite; 软件一般是Linux V4L2。编解码模块是VCU,配套的软件是GStreamer,或者Ctrl-SW。视频输出模块,硬件可能是HDMI-TX和FrmBufRead; 软件一般是Linux DRM。

    Video_Subsystem_Simple_Diagram

    调试时,根据上述框图,逐个检查其中的模块的输入输出。

    视频输入模块

    检查视频输入源

    调试VCU时,首先检查视频输入模块的输出是否正常。如果Linux V4L2管理视频输入模块的硬件,则会创建Linux的media/video设备,可以使用工具media-ctl查看它们的信息。对于HDMI-Rx,可以看到有没有连接源设备,视频输入源的分辨率、格式等。

    #media-ctl -p -d /dev/media0 
     
    Media controller API version 0.1.0
      
    Media device information
    ------------------------
    driver          xilinx-video
    model           Xilinx Video Composite Device
    serial
    bus info
    hw revision     0x0
    driver version  0.0.0
      
    Device topology
    - entity 1: vcap_hdmi output 0 (1 pad, 1 link)
                type Node subtype V4L flags 0
                device node name /dev/video0
            pad0: Sink
                    <- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
      
    - entity 5: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
                type V4L2 subdev subtype Unknown flags 0
                device node name /dev/v4l-subdev0
            pad0: Source
                    [fmt:RBG24/1920x1080 field:none]
                    [dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
                    [dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
                    -> "vcap_hdmi output 0":0 [ENABLED]
    

    抓取原始图像

    在确保视频输入源的工作正常、分辨率和格式正确后,可以再继续使用yavta来抓取原始图像,从而检查图像内容。

    #yavta -n 3 -c10 -f UYVY -f UYVY -s 1920x1080 --skip 7 -F /dev/video0
    Device /dev/video0 opened.
    Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes)[63898.159475] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
     device.
    Video format set: UYVY (59565955) 1920x1080 field none[63898.173868] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
    , 1 planes:
     * Stride 3840, buffer size 4147200
    Video format:[63898.188189] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
     UYVY (59565955) 1920x1080 field none, 1 planes:
     * Stride 3840, buffer size 4147200
    3 buffers requested.
    length: 1 offset: 3444933696 timestamp type/source: mono/EoF
    Buffer 0/0 mapped at address 0x7fa84b0000.
    length: 1 offset: 3444933696 timestamp type/source: mono/EoF
    Buffer 1/0 mapped at address 0x7fa80bb000.
    length: 1 offset: 3444933696 timestamp type/source: mono/EoF
    Buffer 2/0 mapped at address 0x7fa7cc6000.
    Unable to start streaming: Invalid argument (22).
    3 buffers released.
    

    检查原始图像

    接下来再使用YUV Player 或者 YUV Player github 分支 检查原始图像是否正确。需要根据图像源,正确设置分辨率和YUV格式。如果图像轮廓就时错误的,这表明Y数据就是错误的,或者分辨率的行宽是错误的,如果轮廓正确、颜色错误,这表明Y数据是对的,分辨率的高不对,或色彩格式不对。大块的连续色彩错误,有可能是把Y数据,当成了UV数据。

    Encoder

    得到了正确的YUV数据后,继续使用Encoder编码。如果有问题,保存Encoder输出的码流,传送到PC机上,使用FFMpeg检查。首先可以使用ffplay播放,用肉眼观察图像。

    Microsoft Windows [Version 10.0.18363.1801]
    (c) 2019 Microsoft Corporation. All rights reserved.
    C:prj>ffplay vcu_example.h265
    ffplay version N-89369-g5a93a85fd0 Copyright (c) 2003-2017 the FFmpeg developers
      built with gcc 7.2.0 (GCC)
      configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
      libavutil      56.  4.100 / 56.  4.100
      libavcodec     58.  6.102 / 58.  6.102
      libavformat    58.  2.103 / 58.  2.103
      libavdevice    58.  0.100 / 58.  0.100
      libavfilter     7.  6.100 /  7.  6.100
      libswscale      5.  0.101 /  5.  0.101
      libswresample   3.  0.101 /  3.  0.101
      libpostproc    55.  0.100 / 55.  0.100
    [hevc @ 000001b016710180] Stream #0: not enough frames to estimate rate; consider increasing probesize
    Input #0, hevc, from 'vcu_example.h265':=    0KB sq=    0B f=0/0
      Duration: N/A, bitrate: N/A
        Stream #0:0: Video: hevc (Main), yuv420p(tv, unknown/bt709/unknown), 3840x2160, 45 fps, 45 tbr, 1200k tbn, 45 tbc
        nan M-V:    nan fd=   2 aq=    0KB vq=  844KB sq=    0B f=0/0
    

    其次可以使用ffprobe导出码流信息,查看各帧的类型和大小。

    C:prj>ffprobe vcu_example.h265 > vcu_example.h265.txt
    ffprobe version N-89369-g5a93a85fd0 Copyright (c) 2007-2017 the FFmpeg developers
      built with gcc 7.2.0 (GCC)
      configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
      libavutil      56.  4.100 / 56.  4.100
      libavcodec     58.  6.102 / 58.  6.102
      libavformat    58.  2.103 / 58.  2.103
      libavdevice    58.  0.100 / 58.  0.100
      libavfilter     7.  6.100 /  7.  6.100
      libswscale      5.  0.101 /  5.  0.101
      libswresample   3.  0.101 /  3.  0.101
      libpostproc    55.  0.100 / 55.  0.100
    [hevc @ 0000019dfe94cd80] Stream #0: not enough frames to estimate rate; consider increasing probesize
    Input #0, hevc, from 'vcu_example.h265':
      Duration: N/A, bitrate: N/A
        Stream #0:0: Video: hevc (Main), yuv420p(tv, unknown/bt709/unknown), 3840x2160, 45 fps, 45 tbr, 1200k tbn, 45 tbc
    

    还可以使用ffmpeg把每一帧转换成bmp图片,仔细查看。

    C:prj>ffmpeg  -i vcu_example.h265  -frames 10 vcu_example.%04d.bmp
    ffmpeg version N-89369-g5a93a85fd0 Copyright (c) 2000-2017 the FFmpeg developers
      built with gcc 7.2.0 (GCC)
      configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
      libavutil      56.  4.100 / 56.  4.100
      libavcodec     58.  6.102 / 58.  6.102
      libavformat    58.  2.103 / 58.  2.103
      libavdevice    58.  0.100 / 58.  0.100
      libavfilter     7.  6.100 /  7.  6.100
      libswscale      5.  0.101 /  5.  0.101
      libswresample   3.  0.101 /  3.  0.101
      libpostproc    55.  0.100 / 55.  0.100
    [hevc @ 000002bda8bd9880] Stream #0: not enough frames to estimate rate; consider increasing probesize
    Input #0, hevc, from 'vcu_example.h265':
      Duration: N/A, bitrate: N/A
        Stream #0:0: Video: hevc (Main), yuv420p(tv, unknown/bt709/unknown), 3840x2160, 45 fps, 45 tbr, 1200k tbn, 45 tbc
    Stream mapping:
      Stream #0:0 -> #0:0 (hevc (native) -> bmp (native))
    Press [q] to stop, [?] for help
    Output #0, image2, to 'vcu_example.%04d.bmp':
      Metadata:
        encoder         : Lavf58.2.103
        Stream #0:0: Video: bmp, bgr24, 3840x2160, q=2-31, 200 kb/s, 45 fps, 45 tbn, 45 tbc
        Metadata:
          encoder         : Lavc58.6.102 bmp
    frame=   10 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.22 bitrate=N/A speed=0.568x
    video:243001kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    

    有问题时,再使用保存的YUV文件,使用xilinx发布的ctrlsw_encoder在ZCU106单板上编码,对比效果。也可以试试Xilinx提供的配置文件。如果有区别,再对比实际代码和xilinx发布的ctrlsw_encoder、配置文件的区别。

    Decoder

    检查完encoder的输出,再检查decoder的输出。ctrlsw_decoder默认输出I420(YUV420)/I422(YUV422)格式的YUV输出。如果需要,可以参考VCU Ctrl-SW 2020.2输出 NV12 的 YUV 文件,使ctrlsw_decoder输出NV12/NV16的文件。输出后,也可以使用YUV Player 或者 YUV Player github 分支 检查解码后的YUV图像是否正确。

    图像质量

    如果图像质量有问题,建议使用大的显示器或电视机查看。显示器或电视机要真实支持检查的分辨率,在显示中不能有缩放,避免缩放引起新的问题。也可以使用ffmpeg的下面命令,将码流转换为bmp图片,一帧一帧查看。

    C:prj>ffmpeg  -i vcu_example.h265  -frames 10 vcu_example.%04d.bmp
    ffmpeg version N-89369-g5a93a85fd0 Copyright (c) 2000-2017 the FFmpeg developers
      built with gcc 7.2.0 (GCC)
      configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
      libavutil      56.  4.100 / 56.  4.100
      libavcodec     58.  6.102 / 58.  6.102
      libavformat    58.  2.103 / 58.  2.103
      libavdevice    58.  0.100 / 58.  0.100
      libavfilter     7.  6.100 /  7.  6.100
      libswscale      5.  0.101 /  5.  0.101
      libswresample   3.  0.101 /  3.  0.101
      libpostproc    55.  0.100 / 55.  0.100
    [hevc @ 000001bb953e9880] Stream #0: not enough frames to estimate rate; consider increasing probesize
    Input #0, hevc, from 'vcu_example.h265':
      Duration: N/A, bitrate: N/A
        Stream #0:0: Video: hevc (Main), yuv420p(tv, unknown/bt709/unknown), 3840x2160, 45 fps, 45 tbr, 1200k tbn, 45 tbc
    Stream mapping:
      Stream #0:0 -> #0:0 (hevc (native) -> bmp (native))
    Press [q] to stop, [?] for help
    Output #0, image2, to 'vcu_example.%04d.bmp':
      Metadata:
        encoder         : Lavf58.2.103
        Stream #0:0: Video: bmp, bgr24, 3840x2160, q=2-31, 200 kb/s, 45 fps, 45 tbn, 45 tbc
        Metadata:yo
          encoder         : Lavc58.6.102 bmp
    frame=   10 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.22 bitrate=N/A speed=0.553x
    video:243001kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    

    通常,增大码率、减小QP值、使用VBR、使用跳帧等,可以优化图像质量。

    VCU性能

    如果检查VCU性能,建议使用实际视频输入输出,因为文件操作有耗时内存拷贝。如果用文件,也一定要把文件放在Linux的ramdisk上,不能使用sd等存放。

    测试性能时,千万记住,不能使用软件复制Video数据,包括Gstreamer的tee插件。

    案例

    使用VBR优化图像质量

    有项目反应图像质量与参考的视频有差距。拿到VCU编码的视频码流和参考视频码流后,使用ffprobe分析,得到每一帧的大小。发现VCU的视频码流的I帧小于150KByte,而参考视频码流的视频码流的I帧有400KByte。建议使用VBR。应用VBR后,图像质量明显改善。

    解决I帧小导致的图像质量问题

    有项目反应图像质量与参考的视频有差距。拿到VCU编码的视频码流和参考视频码流后,使用ffprobe分析,得到每一帧的大小。发现VCU的视频码流的I帧大约时是180KByte,而参考视频码流的视频码流的I帧有1000KByte。该项目使用VBR后,I帧图像也没有增大。检查VCU的配置,发现配置文件了有“MaxPictureSize = 1500”,相当于180KByte。建议客户设置“MaxPictureSize = 20000”,或者移除这一条后,图像质量明显改善。MaxPictureSize用于限制帧编码后的最大大小,目的是减少码率波动,会限制I帧的质量。

    建议

    分析VCU的问题时,请提供VCU的视频码流、参考视频码流、原始码流、配置文件。

  • 相关阅读:
    Educational Codeforces Round 88 (Rated for Div. 2) D. Yet Another Yet Another Task(枚举/最大连续子序列)
    Educational Codeforces Round 88 (Rated for Div. 2) A. Berland Poker(数学)
    Educational Codeforces Round 88 (Rated for Div. 2) E. Modular Stability(数论)
    Educational Codeforces Round 88 (Rated for Div. 2) C. Mixing Water(数学/二分)
    Codeforces Round #644 (Div. 3)
    Educational Codeforces Round 76 (Rated for Div. 2)
    Educational Codeforces Round 77 (Rated for Div. 2)
    Educational Codeforces Round 87 (Rated for Div. 2)
    AtCoder Beginner Contest 168
    Codeforces Round #643 (Div. 2)
  • 原文地址:https://www.cnblogs.com/hankfu/p/15354059.html
Copyright © 2011-2022 走看看