RTP H.264 stream freezing on first frame using wis-streamer and VLC player

[Eli]

Hi everyone,

I have encountered a strange problem while playing an RTP H.264 stream using VLC player.

I am developing firmware for an IP camera designed to be used with specialized software (control client).
By default, the camera's RTSP streamer (based on wis-streamer with a custom video source) had a bug where in each GOP, the SPS, PPS, SEI and IDR-frame were all grouped, and the stream simply consisted of an SPS NAL unit, followed by several P-slices. The PPS, SEI, and IDR-frame were all hidden inside the payload of the SPS packet. This caused the target software to be unable to play the stream, yet VLC player worked just fine.

After fixing the video source and properly splitting the various NAL units, the stream works properly on the target client, but now it works only partially with VLC - sometimes it works well, other times it freezes after showing 1-2 frames. Looking at the wireshark capture, the stream now looks like a proper H.264 stream: SPS, PPS, SEI, IDR, P-Slice, P-Slice, P-Slice...
I've uploaded wireshark logs and VLC logs for both scenarios (old stream, new stream):

https://www.dropbox.com/s/kk4m04toudijb82/new.log
https://www.dropbox.com/s/gyg39osmhqi7p1r/New.pcapng

https://www.dropbox.com/s/ewank7m3itt8jcr/old.log
https://www.dropbox.com/s/jk08a4nq8cuog76/Old.pcapng

I've posted this question to the VLC support boards and was advised it might be a live555 problem and that I should ask about it over here.
Any help would be greatly appreciated.

[Ross Finlayson]

Sorry, but we can't help you with 'decoding'/'freezing' problems with VLC, because VLC is not our software. (We provide only the implementation of the RTSP client protocol that VLC uses.)

I suggest instead that you start by using our "openRTSP" client application <http://www.live555.com/openRTSP/> to receive your stream (it will write the received H.264 video data into a file). Then, rename this file to have a ".h264" filename suffix, and try playing this file with VLC.

If VLC has the same 'decoding'/'freezing' problems when it plays the received file, then the problem must be with your encoder and/or VLC; it has nothing to do with us.

If, however, VLC plays the received file OK, then there might, conceivably, be some problem with the 'presentation times' that your server gives to each NAL unit that it sends. You could test this by running our "testRTSPClient" demo application.


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

[Jeff Shanab]

Sounds like you have a good understanding of the nal units and h264 traffic
in general. So It might be a decoder issue.
I have found descrepencies in interpretations of the standard.

From experience...

Each Nal unit needs a header before it goes to the decoder. This is the 00
00 00 01 before the nal unit. I have seen some streams only have 00 00 01
and different versions of libavcodec for example would be tolerent or not
tolerent of this.

Assuming [x] represents the nal unit x with the headers. and 7 is sps 8 is
pps 5 is IDR slice and 1 is P-slice

[7][8][5][1][1][1]...[1][7][8][5][1][1]... is normal
[7][8][5][5][5][1][1][1]...[1][7][8][5][5][5][1][1]... is seen on high
resolution
[7][8][5][1][1][1]...[1][5][1][1]... is legal
[7][8][5][1][1][1]...[1][7][8][1][1]...[5][1][1].... is actually legal per
standard but messes with some decoders.

Hi everyone,
I have encountered a strange problem while playing an RTP H.264 stream
using VLC player.
I am developing firmware for an IP camera designed to be used with
specialized software (control client).
By default, the camera's RTSP streamer (based on wis-streamer with a
custom video source) had a bug where in each GOP, the SPS, PPS, SEI and
IDR-frame were all grouped, and the stream simply consisted of an SPS NAL
unit, followed by several P-slices. The PPS, SEI, and IDR-frame were all
hidden inside the payload of the SPS packet. This caused the target
software to be unable to play the stream, yet VLC player worked just fine.
After fixing the video source and properly splitting the various NAL
units, the stream works properly on the target client, but now it works
only partially with VLC - sometimes it works well, other times it freezes
after showing 1-2 frames. Looking at the wireshark capture, the stream now
looks like a proper H.264 stream: SPS, PPS, SEI, IDR, P-Slice, P-Slice,
P-Slice...
I've uploaded wireshark logs and VLC logs for both scenarios (old stream,
https://www.dropbox.com/s/kk4m04toudijb82/new.log
https://www.dropbox.com/s/gyg39osmhqi7p1r/New.pcapng
https://www.dropbox.com/s/ewank7m3itt8jcr/old.log
https://www.dropbox.com/s/jk08a4nq8cuog76/Old.pcapng
I've posted this question to the VLC support boards and was advised it
might be a live555 problem and that I should ask about it over here.
Any help would be greatly appreciated.
_______________________________________________
live-devel mailing list
http://lists.live555.com/mailman/listinfo/live-devel

[Joshua Kordani]

4 byte start code is required on sps and pps nals. All others may be 3 or 4 bytes. Admittedly, the spec is way too wordy but is summarized in the previous sentence. Also, the pps and sps nals do not need to be repeated unless the future frame requires it. I have a camera that only supplies sps and pps in the rtsp setup, and then emits idr and slice in the rtp stream

_______________________________________________
live-devel mailing list
http://lists.live555.com/mailman/listinfo/live-devel

[Eli]

Thanks everyone for your replies.

My camera's H264 stream looks like the following:
[7] [8] [6] [5] [1] [1] [1] [1]....
Obviously I remove the NAL unit headers (00 00 00 01) before passing them on to H264VideoStreamDiscreteFramer (as required)
I've tried removing the SEI NAL unit [6] from the stream, this causes the stream to work for about 10 seconds and VLC stopping playback afterwards
Here's the log: https://www.dropbox.com/s/6c5gv6txkmq1120/no%20sei.log

I've tried following Ross Finlayson's advice and using openRTSP to record the stream into a file and play it back. VLC player couldn't recognize the file type even after renaming it to .264
I've looked at it with a hex editor and the NAL unit headers seem intact.

I've tried doing the same with a different camera I have. This camera's RTSP stream works perfectly with both the target software, and VLC player
I've tried to record its stream using openRTSP and play it back with VLC player without success as well.

Is there anything else I could try regarding this issue?

[Ross Finlayson]

Post by Eli
I've tried following Ross Finlayson's advice and using openRTSP to record the stream into a file and play it back. VLC player couldn't recognize the file type even after renaming it to .264

You need to rename it to have a ".h264" filename suffix - as I said in my email response - not ".264". VLC doesn't recognize the ".264" filename suffix. (I've asked the VLC developers to update VLC to recognize the ".264" suffix, and treat it just like ".h264", but for some reason they don't want to do that.)

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

[Eli]

I have renamed the file to .h264 and it played back just fine with VLC player. I then went on to check the presentation times using testRTSPclient and indeed there was something wrong with them.
After fixing the code of my custom video source used with wis-streamer, the stream works fine!

Thanks, Ross!