rtsp client -> h264 decoder

[Kenneth Forsythe]

Hello Live555,

I have an application that is based off of testRTSPClient. In afterGettingFrame I am then passing the data off to an H264 decoder which then is rendered on a video surface.

I have 2 cameras and live555MediaServer to test with. LiveMediaServer and one camera work very well. The other camera on the other hand not so much. I am seeing distorted/garbled video and lots of green. This camera, does work through VLC and other players.

What usually causes this type of behavior?

What I am doing is quite similar to what I see in H264or5VideoFileSink::afterGettingFrame. On first frame I am prepending the data with [startcodes][sps] and [startcodes] [pps], which was captured earlier on. After first frame the data is only prepended with the start codes. Is there something else I should be doing, ie analyzing the nal type and modifying differently? Or do you think the problem could be outside the stream and more in the area where I am setting up the video information?

Thank You,

Ken

[Ross Finlayson]

Post by Kenneth Forsythe
What usually causes this type of behavior?

Truncated video frames, due to your camera's frames (actually, H.264 NAL units) being too large for the outgoing "RTPSink"s buffer. If this is happening, you should be seeing warning messages on your server's 'stderr', telling you to increase "OutPacketBuffer::maxSize". You can also check this in your server code by noting whether/when you ever have to set "fNumTruncatedBytes" (due to the NAL unit size being > "fMaxSize").

While increasing "OutPacketBuffer::maxSize" will fix your problem, a better solution is to reconfigure your camera's encoder to not generate such large H.264 NAL units in the first place.


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

[Kenneth Forsythe]

Hi Ross,

I think I wrote that a little confusingly. I am not feeding the cameras into the liveMeda server. The liveMeda server is there as another test source. I am using it to play files, no changes to code.
With that said, I don't actually have the ability to change much of the camera's server properties. The most relevant setting is max packet size which is at 1400.

Since it works in other players, so I can only assume I am doing something wrong on my end.

Thank You,

Ken

From: live-devel [mailto:live-devel-***@ns.live555.com] On Behalf Of Ross Finlayson
Sent: Friday, September 26, 2014 12:49 PM
To: LIVE555 Streaming Media - development & use
Subject: Re: [Live-devel] rtsp client -> h264 decoder

What usually causes this type of behavior?

Truncated video frames, due to your camera's frames (actually, H.264 NAL units) being too large for the outgoing "RTPSink"s buffer. If this is happening, you should be seeing warning messages on your server's 'stderr', telling you to increase "OutPacketBuffer::maxSize". You can also check this in your server code by noting whether/when you ever have to set "fNumTruncatedBytes" (due to the NAL unit size being > "fMaxSize").

While increasing "OutPacketBuffer::maxSize" will fix your problem, a better solution is to reconfigure your camera's encoder to not generate such large H.264 NAL units in the first place.

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

[Ross Finlayson]

OK, so perhaps it would be best if you clarified what specific problem you are seeing. Is it a problem at the server end, at the client end, or both? And where specifically are you using the "LIVE555 Streaming Media" code? In your server (i.e., camera), in your client (i.e., media player), or both?

Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

[Kenneth Forsythe]

For servers I have two commercial H264 cameras and one instance of live555MediaServer.exe running stock. VLC can connect fine to all three. No problems here (as far as I can tell).

My client, on the other hand only works for the live555MediaServer and one of the cameras. The other camera, after receiving the data and delivering to the decoder/renderer and am seeing green distorted video, complete gobbly-gook. There doesn't appear to be any connectivity problems nor buffer size error messages.

So I wanted to see if my rtsp code handling is correct or if I need to do some extra work before the data is delivered to the decoder (I am new to this and figuring it out as I go along). As I mentioned before:

What I am doing is quite similar to what I see in H264or5VideoFileSink::afterGettingFrame. On first frame I am prepending the data with [startcodes][sps] and [startcodes] [pps], (sps,pps was captured earlier on when setting up the subsession). After first frame the data is only prepended with the start codes. Is there something else I should be doing, ie analyzing the nal type and modifying differently? Or do you think the problem could be outside the stream and more in the area where I am setting up the video information?

Thanks,

Ken

From: live-devel [mailto:live-devel-***@ns.live555.com] On Behalf Of Ross Finlayson
Sent: Friday, September 26, 2014 1:24 PM
To: LIVE555 Streaming Media - development & use
Subject: Re: [Live-devel] rtsp client -> h264 decoder

OK, so perhaps it would be best if you clarified what specific problem you are seeing. Is it a problem at the server end, at the client end, or both? And where specifically are you using the "LIVE555 Streaming Media" code? In your server (i.e., camera), in your client (i.e., media player), or both?
Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

[Ross Finlayson]

My client, on the other hand only works for the live555MediaServer and one of the cameras. The other camera, after receiving the data and delivering to the decoder/renderer and am seeing green distorted video, complete gobbly-gook. There doesn’t appear to be any connectivity problems nor buffer size error messages.

Nonetheless, I suspect that the problem is a buffer size problem - but in your client, not the server. In your "afterGettingFrame()" function, check the "numTruncatedBytes" parameter. If it's ever non-zero, then you'll need to increase the size of the buffer (in the "MediaSink" subclass that's receiving from your "H264VideoRTPSource" object).

I also suggest running the "testRTSPClient" and "openRTSP" demo applications (RTSP clients) against the camera that is causing problems, and against the camera that's not. This may give you hints as to what's going wrong.

In particular, I suggest running "openRTSP" as a client for the problematic camera, renaming the output file to have a ".h264" filename suffix, and playing it with VLC. Do you see the same artifacts that you see in your client?

What I am doing is quite similar to what I see in H264or5VideoFileSink::afterGettingFrame. On first frame I am prepending the data with [startcodes][sps] and [startcodes] [pps], (sps,pps was captured earlier on when setting up the subsession). After first frame the data is only prepended with the start codes. Is there something else I should be doing, ie analyzing the nal type and modifying differently?

No, what you're doing should be enough.


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/

[Ken Ferguson]

Hi Ken, Ross,

I just wanted to add there is one extra possibility here. We have noticed some cameras, especially HD cameras, split images across more than one slice. In a vast majority of cameras the sequence is:

SPS PPS IDR -> SLICE -> SLICE -> SLICE...

Where -> indicates the image boundary. However in some HD cameras we have seen:

SPS PPS IDR IDR -> SLICE SLICE -> SLICE SLICE -> SLICE SLICE.

If you do not group all the correct parts of an image together when you decode each *image* the you can see the green distorted video.

Just putting it out there as another possibility.

Kind Regards,

Ken Ferguson.

Post by Kenneth Forsythe
For servers I have two commercial H264 cameras and one instance of live555MediaServer.exe running stock. VLC can connect fine to all three. No problems here (as far as I can tell).
My client, on the other hand only works for the live555MediaServer and one of the cameras. The other camera, after receiving the data and delivering to the decoder/renderer and am seeing green distorted video, complete gobbly-gook. There doesn't appear to be any connectivity problems nor buffer size error messages.
/What I am doing is quite similar to what I see in H264or5VideoFileSink::afterGettingFrame. On first frame I am prepending the data with [startcodes][sps] and [startcodes] [pps], (sps,pps was captured earlier on when setting up the subsession). After first frame the data is only prepended with the start codes. Is there something else I should be doing, ie analyzing the nal type and modifying differently? Or do you think the problem could be outside the stream and more in the area where I am setting up the video information?/
Thanks,
Ken
*Sent:* Friday, September 26, 2014 1:24 PM
*To:* LIVE555 Streaming Media - development & use
*Subject:* Re: [Live-devel] rtsp client -> h264 decoder
OK, so perhaps it would be best if you clarified what specific problem you are seeing. Is it a problem at the server end, at the client end, or both? And where specifically are you using the "LIVE555 Streaming Media" code? In your server (i.e., camera), in your client (i.e., media player), or both?
Ross Finlayson
Live Networks, Inc.
http://www.live555.com/
_______________________________________________
live-devel mailing list
http://lists.live555.com/mailman/listinfo/live-devel

--
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
*Ken Ferguson
Cubitech Ltd.
* Discovery Court Business Centre
551-553 Wallisdown Road
Poole
Dorset
BH12 5AG
United Kingdom

Tel. [phone]
Email: ***@cubitech.co.uk <mailto:***@cubitech.co.uk>
Web: www.cubitech.co.uk <http://www.cubitech.com>


This email may contain confidential information. if you are not a named recipient, or believe you have been sent this email in error, please inform Cubitech Ltd immediately. All outbound email is scanned for viruses, but we cannot guarantee that it is virus-free when you receive it.