从RTSP协议的直播到 HTTP TS(APPLE的Live streaming方案)转换工作终于取得技术胜利,中间的过程有些经验值得留存.
基本代码完成后,存在几个较为严重的问题. 首先是转换之后的文件,VLC播放不了;其二是Mplayer能够播放视频,但是没有音频.
经过分析,因为TS流中的载体是H264的视频和AAC的音频;对这种情况TS流的描述,只是依照ISO-13818-1的2000版的显然不够,根据资料显示,只有2007版的13818才能满足需要. 因为2007版的13818涉及到版权和费用,获取到它也费了不少时间和周折.不过新标准最后终于拿到了.
就是这一部分:
2.14 Carriage of ITU-T Rec. H.264 | ISO/IEC 14496-10 video
2.14.1 Introduction
This Specification defines the carriage of ITU-T Rec. H.264 | ISO/IEC 14496-10 elementary stream within ITU-T Rec. H.222.0 | ISO/IEC 13818-1 systems, both for program and transport streams. Typically, an ITU-T Rec. H.264 | ISO/IEC 14496-10 stream will be an element of an ITU-T Rec. H.222.0 | ISO/IEC 13818-1 program, as defined by the PMT in a Transport Stream and the PSM in a Program Stream. The carriage and buffer management of AVC video streams is defined using existing parameters from this Recommendation | International Standard such as PTS and DTS,as well as information present within an AVC video stream. Carriage of AVC video streams in an ITU-T Rec. H.222.0 | ISO/IEC 13818-1 stream defines accurate mapping between STD parameters and HRD parameters that may be present in an AVC video stream. Requirements are defined for the presence of HRD parameters in the AVC video stream, to ensure that it can be verified whether each STD requirement is met for each AVC video stream carried in a transport stream or a program stream. NOTE 1 – Though the timing information present in the AVC video stream may not use a 90-kHz clock, the PTS and DTS timestamps need to be expressed in units of 90 kHz. When an ITU-T Rec. H.264 | ISO/IEC 14496-10 stream is carried in an ITU-T Rec. H.222.0 | ISO/IEC 13818-1 stream, the ITU-T Rec. H.264 | ISO/IEC 14496-10 coded data shall be contained in PES packets. The ITU-T Rec. H.264 | ISO/IEC 14496-10 coded data shall comply with the byte stream format defined in Annex B of ITU-T Rec. H.264 | ISO/IEC 14496-10, with the following constraints:
• Each AVC access unit shall contain an access unit delimiter NAL Unit; NOTE 2 – ITU-T Rec. H.264 | ISO/IEC 14496-10 requires that an access unit delimiter NAL Unit, if present, is the first NAL Unit within an AVC access unit. Access unit delimiter NAL Units simplify the ability to detect the boundary between pictures; they avoid the need to process the content of slice headers, and they are particularly useful for the Baseline and Extended profiles where slice order can be arbitrary.
• Each byte stream NAL Unit that carries the access unit delimiter shall contain exactly one zero_byte syntax element. NOTE 3 – The syntax and semantics of byte stream NAL units are defined in Annex B of ITU-T Rec. H.264 |ISO/IEC 14496-10.
• All Sequence and Picture Parameter Sets (SPS and PPS) necessary for decoding the AVC video stream shall be present within that AVC video stream. NOTE 4 – ITU-T Rec. H.264 | ISO/IEC 14496-10 also allows delivery of SPS and PPS by external means. This Specification does not provide support for such delivery, and therefore requires SPS and PPS to be carried within the AVC video stream.
• Each AVC video sequence that contains hrd_parameters() with the low_delay_hrd_flag set to '1', shall carry VUI parameters in which the timing_info_present_flag shall be set to '1'.
NOTE 5 – If the low_delay_hrd_flag is set to '1', then buffer underflow is allowed to occur in the STD model; see 2.14.3 and 2.14.4. Setting the timing_info_present_flag to '1' ensures that the AVC video stream contains sufficient information to determine the DPB output time and the CPB removal time of AVC access units, also in case of underflow.
To provide display specific information such as aspect_ratio, it is strongly recommended that each AVC video stream carries VUI parameters with sufficient information to ensure that the decoded AVC video stream can be displayed correctly by receivers.
对着标准,发现了ADU的问题,标准要求ADU在每个访问单元都出现一次.然后根据要求更新了代码之后,VLC仍然播放不出.
打开VLC的日志,显示CRC,PMT的错误. 这次终于在vlc上看到了视频了,然而音频不是没有声音就是不正常,上到IPHONE上实验,索性一点声音都没有.
那么音频的问题在哪?
从以下几点开始怀疑:
1) ADTS头的问题, 确实发现了问题,频率等信息是hardcode进去的,改正这个bug后,可以在VLC上正常播放了.但是在IPHONE上依然保持沉默.
TS流结构的问题: 对比可正常在IPHONE上播放,TS流,发现了有的TS流PCR专门用一个流来传送,但有的却直接使用视频的携带的PCR.
2,3天过去了音频的在IPHONE上的问题依然没有解决.
最后终于怀疑到音频的时戳上了,通过跟正确的对比,发现音频从RTSP收下来的是传竟然几帧都相同,但是可以正确播放的音频时间戳每次都有变化.时戳的变化也是有规律的.按照这个规律更改了时戳.终于听到了久违的声音.
事后回头查了TS的标准,确认时戳是要变化的.
但是RTSP为什么产生了这种时戳呢?谁知道呢,或许是所谓deepth 导致的吧