【分享】以H.265 1080p 运行 Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display
运行 Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display，可以测试HDMI输入输出，和VCU的低延时编码。Xilinx wiki的文章MPSoC VCU TRD 2019.2 - Xilinx Low Latency PL DDR XV20 HDMI Video Capture and Display以H.264和4K分辨率为例。
下面记录H.265和1080p分辨率的运行命令。
1. 设置HDMI输入分辨率
首先设置HDMI输入，保证输入分辨率是1080p。我使用电脑Dell 5580作为ZCU106单板的HDMI输入源。电脑Dell 5580上看到两个显示器，一个是真实显示器，一个是ZCU106单板的HDMI输入。虽然我在电脑Dell 5580设置ZCU106单板的HDMI输入为1080p，但是在ZCU106单板使用命令“xmedia-ctl -p -d /dev/media1”查看，HDMI输入的分辨率一直是4K。后来把电脑Dell 5580的输出模式从扩展(Extend)改为复制(Duplicate)后，在ZCU106单板才看到HDMI输入的分辨率是1080p。这一点很莫名其妙。
错误的分辨率信息
root@vcu_llp2_xv20:~# xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:VYYUYY10_4X20/1280x720 field:none colorspace:srgb]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:VYYUYY10_4X20/1920x1080 field:none colorspace:srgb]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/3840x2160 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps: progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 3840x2160p30 (4400x2250) stds:CEA-861 flags:CE-video]
-> "a0040000.v_proc_ss":0 [ENABLED]
正确的分辨率信息
root@vcu_llp2_xv20:~# xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:VYYUYY10_4X20/1280x720 field:none colorspace:srgb]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:VYYUYY10_4X20/1920x1080 field:none colorspace:srgb]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps: progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
-> "a0040000.v_proc_ss":0 [ENABLED]
2. 设置VPSS分辨率和格式
接下来设置ZCU106的VPSS的分辨率和格式。设置后，再使用命令“xmedia-ctl -p -d /dev/media1”查看，所有pad的分辨率信息都是1080p了。
root@vcu_llp2_xv20:~# xmedia-ctl -d /dev/media1 -V ""a0040000.v_proc_ss":0 [fmt:RBG888_1X24/1920x1080 field:none]"
root@vcu_llp2_xv20:~# xmedia-ctl -d /dev/media1 -V ""a0040000.v_proc_ss":1 [fmt:UYVY10_1X20/1920x1080 field:none]"
root@vcu_llp2_xv20:~# xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:RBG888_1X24/1920x1080 field:none]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:UYVY10_1X20/1920x1080 field:none]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
-> "a0040000.v_proc_ss":0 [ENABLED]
3. 设置HDMI输出分辨率
使用命令“modetest -D a00c0000.v_mix -s 35:1920x1080-30@BG24”，设置设置HDMI输出分辨率。设置时，会产生异常，可以暂时忽略。设置后，HDMI显示器会显示蓝屏。
root@vcu_llp2_xv20:~# modetest -D a00c0000.v_mix -s 35:1920x1080-30@BG24
setting mode 1920x1080-30Hz@BG24 on connectors 35, crtc 33
[ 254.364908] ------------[ cut here ]------------
[ 254.369527] driver forgot to call drm_crtc_vblank_off()
[ 254.374783] WARNING: CPU: 2 PID: 2878 at drivers/gpu/drm/drm_atomic_helper.c:1000 drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[ 254.386941] Modules linked in: dmaproxy(O) al5d(O) al5e(O) allegro(O) xlnx_vcu xlnx_vcu_clk xilinx_hdmi_tx(O) xilinx_hdmi_rx(O) xlnx_vcu_core xilinx_vphy(O) dp159(O) mali(O) uio_pdrv_genirq
[ 254.403812] CPU: 2 PID: 2878 Comm: modetest Tainted: G O 4.19.0-xilinx-v2019.2 #1
[ 254.412584] Hardware name: ZynqMP ZCU106 RevA (DT)
[ 254.417361] pstate: 60000005 (nZCv daif -PAN -UAO)
[ 254.422135] pc : drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[ 254.428646] lr : drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[ 254.435154] sp : ffffff8014feba00
[ 254.438453] x29: ffffff8014feba00 x28: ffffffc87b2d2400
[ 254.443757] x27: ffffffc85fc14800 x26: 0000000000000000
[ 254.449061] x25: 0000000000000038 x24: ffffff80091c725b
[ 254.454365] x23: ffffff8008d74e98 x22: 0000000000000000
[ 254.459669] x21: ffffffc85f29c800 x20: ffffffc85fc53890
[ 254.464973] x19: ffffffc87ae49980 x18: 0000000000000010
[ 254.470277] x17: 0000000000000000 x16: 0000000000000000
[ 254.475581] x15: ffffffffffffffff x14: ffffff8009148648
[ 254.480885] x13: ffffff80891e7f0f x12: ffffff80091e7f17
[ 254.486189] x11: ffffff800915a000 x10: ffffff8014feb6e0
[ 254.491493] x9 : 00000000ffffffd0 x8 : ffffff8008543da8
[ 254.496797] x7 : 6c6163206f742074 x6 : 00000000000001d7
[ 254.502101] x5 : 0000000000000003 x4 : 0000000000000000
[ 254.507396] x3 : 0000000000000000 x2 : ffffffffffffffff
[ 254.512700] x1 : 6b059e08b5ce2000 x0 : 0000000000000000
[ 254.518004] Call trace:
[ 254.520438] drm_atomic_helper_commit_modeset_disables+0x3b0/0x3b8
[ 254.526609] drm_atomic_helper_commit_tail+0x20/0x78
[ 254.531556] commit_tail+0x74/0x78
[ 254.534942] drm_atomic_helper_commit+0xc8/0x140
[ 254.539544] drm_atomic_commit+0x48/0x58
[ 254.543458] drm_atomic_helper_set_config+0xa0/0xb0
[ 254.548321] drm_mode_setcrtc+0x144/0x5d8
[ 254.552322] drm_ioctl_kernel+0xb4/0x100
[ 254.556228] drm_ioctl+0x204/0x3a8
[ 254.559616] do_vfs_ioctl+0xb8/0x8a0
[ 254.563182] ksys_ioctl+0x44/0x90
[ 254.566481] __arm64_sys_ioctl+0x1c/0x28
[ 254.570388] el0_svc_common+0x84/0xd8
[ 254.574041] el0_svc_handler+0x68/0x80
[ 254.577774] el0_svc+0x8/0xc
[ 254.580637] ---[ end trace 44a2c3f75cacf5a7 ]---
4. 本板运行
复制文件/media/card/config/input.cfg为/media/card/config/input-1080p.cfg，修改其中的分辨率为1080p。再以命令"vcu_gst_app /media/card/config/input-1080p.cfg"运行，可以看到本板的HDMI输入显示到了HDMI输出上。
root@vcu_llp2_xv20:/media/card/config# cat input-1080p.cfg
Common Configuration : START
Num Of Input : 1
Output : HDMI
Out Type : Display
Frame Rate : 60
Exit
Input Configuration : START
Input Num : 1
Input Type : hdmi_1
Raw : FALSE
Width : 1920
Height : 1080
Format : XV20
Enable LLP2 : FALSE
Exit
Encoder Configuration : START
Encoder Num : 1
Encoder Name : AVC
Profile : High
Rate Control : Low_Latency
Filler Data : False
QP : Auto
L2 Cache : TRUE
Latency Mode : Sub_Frame
Low Bandwidth : FALSE
Gop Mode : Basic
Bitrate : 25000
B Frames : 0
Slice : 8
GoP Length : 60
Preset : Custom
Exit
Streaming Configuration : START
Streaming Num : 1
Host IP : 192.168.25.89
Port : 5004
Exit
Trace Configuration : START
FPS Info : TRUE
APM Info : TRUE
Pipeline Info : TRUE
Exit
root@vcu_llp2_xv20:/media/card/config# vcu_gst_app /media/card/config/input-1080p.cfg
/*************************Pipeline Information Start*************************/
Pipeline Info : On
Fps Info : On
APM Info : On
Output goes on : HDMI
Frame rate : 60
Number of Source is : 1
Use case is to : Display
B Frames : 0
Bitrate : 25000
Enable L2Cache : True
Enc Name : AVC
Gop Len : 60
Profile : High
Qp Mode : Auto
Rc Mode : Low_Latency
Num Slice : 8
GoP Mode : Basic
Filler Data : False
Low Bandwidth : False
Latency Mode : Sub_Frame
Device Type : HDMI
Format : XV20
Width : 1920
Height : 1080
Raw : False
Accelerator flag : False
Enable_scd flag : False
Enable_roi flag : False
Enable_llp2 flag : False
Src Type : Live Src
/*************************Pipeline Information End*************************/
!! The specified Level is too low and will be adjusted !!
Encoder Memory Bandwidth (1.04 Gbps)
Decoder Memory Bandwidth (0.00 Gbps)
Pipeline [1] Fps[0]
Encoder Memory Bandwidth (8.72 Gbps)
Decoder Memory Bandwidth (7.99 Gbps)
Pipeline [1] Fps[20]
Encoder Memory Bandwidth (8.78 Gbps)
Decoder Memory Bandwidth (7.12 Gbps)
Pipeline [1] Fps[22]
Encoder Memory Bandwidth (8.76 Gbps)
Decoder Memory Bandwidth (7.18 Gbps)
Pipeline [1] Fps[22]
Encoder Memory Bandwidth (8.72 Gbps)
Decoder Memory Bandwidth (7.17 Gbps)
Pipeline [1] Fps[22]
^CHit Ctrl-C
Quitting the app now
Quitting the loop
playback count 1
pipeline stopped successfully
root@vcu_llp2_xv20:/media/card/config# ^C
5. 网络运行
5.1. 设置网络
在HDMI输入单板上执行命令“ifconfig eth0 192.168.25.10 up”，配置HDMI输入单板的IP为192.168.25.10。
在HDMI输出单板上执行命令“ifconfig eth0 192.168.25.89 up”，配置HDMI输出单板的IP为192.168.25.89。再执行命令，“ping 192.168.25.10”，检查网络是否通畅。
5.2. 发送视频流
root@vcu_llp2_xv20:~# ifconfig eth0 192.168.25.10 up
root@vcu_llp2_xv20:/media/card/config# gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw(memory:XLNXLL), width=1920, height=1080, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=low-delay-p gop-length=60 b-frames=0 target-bitrate=20000 num-slices=8 control-rate=low-latency prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! rtph265pay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
!! Warning : Adapting profile to support bitdepth and chroma mode
!! The specified Level is too low and will be adjusted !!
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:21.804704005
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
root@vcu_llp2_xv20:/media/card/config# gst-launch-1.0 v4l2src io-mode=4 device=/dev/video0 ! video/x-raw(memory:XLNXLL), width=1920, height=1080, format=NV16_10LE32, framerate=60/1 ! omxh265enc qp-mode=auto gop-mode=low-delay-p gop-length=60 b-frames=0 target-bitrate=20000 num-slices=8 control-rate=low-latency prefetch-buffer=TRUE low-bandwidth=false filler-data=0 cpb-size=1000 initial-delay=500 ! video/x-h265, alignment=nal ! queue max-size-buffers=0 ! rtph265pay ! udpsink host=192.168.25.89 port=5004 buffer-size=60000000 max-bitrate=120000000 max-lateness=-1 qos-dscp=60 async=false
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
!! Warning : Adapting profile to support bitdepth and chroma mode
!! The specified Level is too low and will be adjusted !!
5.3. 接收视频流
root@vcu_llp2_xv20:~# ifconfig eth0 192.168.25.89 up
root@vcu_llp2_xv20:~#
root@vcu_llp2_xv20:~# ping 192.168.25.10
PING 192.168.25.10 (192.168.25.10): 56 data bytes
64 bytes from 192.168.25.10: seq=0 ttl=64 time=0.229 ms
64 bytes from 192.168.25.10: seq=1 ttl=64 time=0.071 ms
^C
--- 192.168.25.10 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.071/0.150/0.229 ms
root@vcu_llp2_xv20:~# gst-launch-1.0 udpsrc port=5004 buffer-size=60000000 caps="application/x-rtp, media=video, clock-rate=90000, payload=96, encoding-name=H265" ! rtpjitterbuffer latency=5 ! rtph265depay ! h265parse ! video/x-h265, alignment=nal ! omxh265dec low-latency=1 ! video/x-raw(memory:XLNXLL) ! queue max-size-bytes=0 ! fpsdisplaysink name=fpssink text-overlay=false video-sink="kmssink bus-id=a00c0000.v_mix plane-id=30" sync=true -v
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: display-width = 1920
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: display-height = 1080
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: sync = true
/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
/GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstRtpJitterBuffer:rtpjitterbuffer0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
/GstPipeline:pipeline0/GstRtpH265Depay:rtph265depay0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal
/GstPipeline:pipeline0/GstRtpH265Depay:rtph265depay0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, payload=(int)96, encoding-name=(string)H265
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
Redistribute latency...
/GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)nal, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, chroma-format=(string)4:2:2, bit-depth-luma=(uint)10, bit-depth-chroma=(uint)10, colorimetry=(string)2:0:0:1, parsed=(boolean)true, pixel-aspect-ratio=(fraction)1/1, profile=(string)main-422-10, tier=(string)main, level=(string)4.1
Redistribute latency...
/GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0.GstPad:src: caps = video/x-raw(memory:GLMemory), format=(string)RGBA, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:1:7:1, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstOMXH265Dec-omxh265dec:omxh265dec-omxh265dec0.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink.GstGhostPad:sink: caps = video/x-raw(memory:XLNXLL), format=(string)NV16_10LE32, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstKMSSink:kmssink0: sync = true
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 14, dropped: 15, fps: 27.85, drop rate: 29.84
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 29, dropped: 23, fps: 30.00, drop rate: 16.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 45, dropped: 31, fps: 30.00, drop rate: 15.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 60, dropped: 39, fps: 30.00, drop rate: 16.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 75, dropped: 44, fps: 29.99, drop rate: 10.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 90, dropped: 49, fps: 29.97, drop rate: 9.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 105, dropped: 54, fps: 29.98, drop rate: 9.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 120, dropped: 59, fps: 29.97, drop rate: 9.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 136, dropped: 64, fps: 30.09, drop rate: 9.40
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 150, dropped: 70, fps: 26.25, drop rate: 11.25
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 162, dropped: 76, fps: 23.85, drop rate: 11.92
6. 技巧
Xilinx wiki的文章的命令中，大约每秒发送一个I帧，造成视频闪烁。如果去掉“periodicity-idr=60”，就不会有视频闪烁了。
以网络运行demo时，先执行接收端Gstreamer命令，再执行发送端Gstreamer命令。如果顺序反了，接收端收不到包，不能显示图像。
7. 已知问题
使用Yavta不能成功捕获视频帧。
root@vcu_llp2_xv20:~# xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:RBG888_1X24/1920x1080 field:none]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:UYVY10_1X20/1920x1080 field:none]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
-> "a0040000.v_proc_ss":0 [ENABLED]
root@vcu_llp2_xv20:~# yavta -n 3 -c10 -f UYVY -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes)[63898.159475] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
device.
Video format set: UYVY (59565955) 1920x1080 field none[63898.173868] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
, 1 planes:
* Stride 3840, buffer size 4147200
Video format:[63898.188189] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
UYVY (59565955) 1920x1080 field none, 1 planes:
* Stride 3840, buffer size 4147200
3 buffers requested.
length: 1 offset: 3444933696 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7fa84b0000.
length: 1 offset: 3444933696 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7fa80bb000.
length: 1 offset: 3444933696 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7fa7cc6000.
Unable to start streaming: Invalid argument (22).
3 buffers released.
root@vcu_llp2_xv20:~# yavta -n 3 -c10 -f YUYV -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes) device.
Video format set: YUYV (56595559) 1920x1080 field none, 1 planes:
* Stride 3840, buffer size 4147200
Video format: YUYV (56595559) 1920x1080 field none, 1 planes:
* Stride 3840, buffer size 4147200
3 buffers requested.
length: 1 offset: 4217307040 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7f8ad60000.
length: 1 offset: 4217307040 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7f8a96b000.
length: 1 offset: 4217307040 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7f8a576000.
Unable to start streaming: Invalid argument (22).
3 buffers released.
root@vcu_llp2_xv20:~# yavta -n 3 -c10 -f Y10 -s 1920x1080 --skip 7 -F /dev/video0
Device /dev/video0 opened.
Device `vcap_hdmi output 0' on `platform:vcap_hdmi:0' is a video output (without mplanes)[63965.043523] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x20303159
device.
Video format set: Y10 (20303159) 1920x1080 field none,[63965.057886] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x20303159
1 planes:
* Stride 3840, buffer size 4147200
Video format: [63965.072212] xilinx-frmbuf a0010000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x20303159
Y10 (20303159) 1920x1080 field none, 1 planes:
* Stride 3840, buffer size 4147200
3 buffers requested.
length: 1 offset: 4160493712 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0x7fbdb14000.
length: 1 offset: 4160493712 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0x7fbd71f000.
length: 1 offset: 4160493712 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0x7fbd32a000.
Unable to start streaming: Invalid argument (22).
3 buffers released.
root@vcu_llp2_xv20:~# v4l2-ctl --list-devices
vcap_hdmi output 0 (platform:vcap_hdmi:0):
/dev/video0
vcap_hdmi_2 output 0 (platform:vcap_hdmi_2:0):
/dev/video1
root@vcu_llp2_xv20:~# ls /sys/class/video4linux/video0/device/ | grep -i media
media1
root@vcu_llp2_xv20:~# xmedia-ctl -p -d /dev/media1
Media controller API version 4.19.0
Media device information
------------------------
driver xilinx-video
model Xilinx Video Composite Device
serial
bus info
hw revision 0x0
driver version 4.19.0
Device topology
- entity 1: vcap_hdmi output 0 (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "a0040000.v_proc_ss":1 [ENABLED]
- entity 5: a0040000.v_proc_ss (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:RBG888_1X24/1920x1080 field:none]
<- "a0000000.v_hdmi_rx_ss":0 [ENABLED]
pad1: Source
[fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
-> "vcap_hdmi output 0":0 [ENABLED]
- entity 8: a0000000.v_hdmi_rx_ss (1 pad, 1 link)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:RBG888_1X24/1920x1080 field:none colorspace:srgb]
[dv.caps:BT.656/1120 min:0x0@25000000 max:4096x2160@297000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 1920x1080p60 (2200x1125) stds:CEA-861 flags:CE-video]
-> "a0040000.v_proc_ss":0 [ENABLED]
root@vcu_llp2_xv20:~# media-ctl -d /dev/media1 -V '"a0040000.v_proc_ss":1 [fmt:VYYUYY8_1X24/1920x10e]'field:none
Unable to setup formats: Invalid argument (22)
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842522/Xilinx+V4L2+MIPI+CSI+driver
root@vcu_llp2_xv20:~# media-ctl -d /dev/media1 -V '"a0040000.v_proc_ss":1 [fmt:UYVY/1920x1080 colorspace:srgb field:none]'
root@vcu_llp2_xv20:~# yavta -n 3 -c10 -f UYVY -s 1920x1080 --skip 7 -F /dev/video1
Device /dev/video1 opened.
Device `vcap_hdmi_2 output 0' on `platform:vcap_hdmi_2:0' is a video output (without mplan[64761.083514] xilinx-frmbuf a0080000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
es) device.
Video format set: UYVY (59565955) 1920x1080 field n[64761.097896] xilinx-frmbuf a0080000.v_frmbuf_wr: Invalid dma template or missing dma video fmt config
one, 1 planes:
* Stride 3840, buffer size 4147200
Video form[64761.112561] xilinx-video amba_pl@0:vcap_hdmi_2: Failed to prepare DMA transfer
at: UYVY (59565955) 1920x1080 field none, 1 planes:
* Stride [64761.125320] xilinx-frmbuf a0080000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
3840, buffer size 4147200
3 buffers requested.
length: 1 offse[64761.139647] xilinx-frmbuf a0080000.v_frmbuf_wr: Invalid dma template or missing dma video fmt config
t: 4059898224 timestamp type/source: mono/EoF
Buffer 0/0 mapped[64761.154311] xilinx-video amba_pl@0:vcap_hdmi_2: Failed to prepare DMA transfer
at address 0x7fb4223000.
length: 1 offset: 4059898224 timestam[64761.167077] xilinx-frmbuf a0080000.v_frmbuf_wr: Framebuffer not configured for fourcc 0x59565955
p type/source: mono/EoF
Buffer 1/0 mapped at address 0x7fb3e2e0[64761.181400] xilinx-frmbuf a0080000.v_frmbuf_wr: Invalid dma template or missing dma video fmt config
00.
length: 1 offset: 4059898224 timestamp type/source: mono/Eo[64761.196071] xilinx-video amba_pl@0:vcap_hdmi_2: Failed to prepare DMA transfer
F
Buffer 2/0 mapped at address 0x7fb3a39000.
[64761.265216] WARNING: CPU: 2 PID: 18896 at drivers/media/common/videobuf2/videobuf2-core.c:1371 vb2_start_streaming+0xe4/0x140
[64761.276502] Modules linked in: dmaproxy(O) al5e(O) al5d(O) allegro(O) xlnx_vcu_clk xlnx_vcu xilinx_hdmi_rx(O) xilinx_hdmi_tx(O) dp159(O) xlnx_vcu_core xilinx_vphy(O) mali(O) uio_pdrv_genirq
[64761.293373] CPU: 2 PID: 18896 Comm: yavta Tainted: G W O 4.19.0-xilinx-v2019.2 #1
[64761.301972] Hardware name: ZynqMP ZCU106 RevA (DT)
[64761.306748] pstate: 20000005 (nzCv daif -PAN -UAO)
[64761.311523] pc : vb2_start_streaming+0xe4/0x140
[64761.316036] lr : vb2_start_streaming+0x64/0x140
[64761.320549] sp : ffffff8009663b20
[64761.323847] x29: ffffff8009663b20 x28: ffffffc87924bc00
[64761.329151] x27: ffffff800919e5c8 x26: ffffff8008c02d60
[64761.334455] x25: 0000000000000000 x24: ffffffc87bb757a8
[64761.339759] x23: 0000000000000000 x22: ffffff8009148648
[64761.345063] x21: ffffffc87bb75a90 x20: ffffffc87bb75aa8
[64761.350367] x19: 00000000ffffffea x18: 0000000000000400
[64761.355671] x17: 0000000000000000 x16: 0000000000000000
[64761.360975] x15: 0000000000000400 x14: 000000000000011a
[64761.366279] x13: 0000000000000000 x12: 0000000000000001
[64761.371583] x11: 0000000000000000 x10: 00000000000007f0
[64761.376879] x9 : ffffff80096638e0 x8 : ffffffc8600aeb50
[64761.382183] x7 : ffffffc87ff86a00 x6 : ffffffc8600ae3c0
[64761.387487] x5 : 0000000000000005 x4 : 0000000000004000
[64761.392782] x3 : ffffffc87bb75b08 x2 : 6b059e08b5ce2000
[64761.398086] x1 : 0000000000000000 x0 : ffffffc85fb611b8
[64761.403390] Call trace:
[64761.405823] vb2_start_streaming+0xe4/0x140
[64761.409990] vb2_core_streamon+0x88/0x180
[64761.413983] vb2_streamon+0x18/0x60
[64761.417455] vb2_ioctl_streamon+0x48/0x58
[64761.421449] v4l_streamon+0x20/0x28
[64761.424929] __video_do_ioctl+0x23c/0x498
[64761.428922] video_usercopy+0x144/0x518
[64761.432741] video_ioctl2+0x14/0x1c
[64761.436214] v4l2_ioctl+0x3c/0x58
[64761.439515] do_vfs_ioctl+0xb8/0x8a0
[64761.443081] ksys_ioctl+0x44/0x90
[64761.446380] __arm64_sys_ioctl+0x1c/0x28
[64761.450288] el0_svc_common+0x84/0xd8
[64761.453941] el0_svc_handler+0x68/0x80
[64761.457674] el0_svc+0x8/0xc
[64761.460537] ---[ end trace 091bf6f1324177c3 ]---
Unable to start streaming: Invalid argument (22).
3 buffers released.
root@vcu_llp2_xv20:~#
7.1. 参考文章
Zynq UltraScale+ MPSoC VCU TRD - Debugging - HDMI Rx Capture Pipeline
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/174784624/Zynq+UltraScale+MPSoC+VCU+TRD+-+Debugging+-+HDMI+Rx+Capture+Pipeline
Cannot capture image by yavta tool
https://forums.xilinx.com/t5/Embedded-Linux/Cannot-capture-image-by-yavta-tool/m-p/1064135#M39452
Xilinx V4L2 MIPI CSI driver
https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842522/Xilinx+V4L2+MIPI+CSI+driver