AAC/H264 RTP Multicast timestamps

[Goran Ambardziev]

Ross,
Thanks for all the help so far!

So, I have a working AAC/H264 application and I'm very close to it
working just OK, except couple of glitches that needs to be settled.

Be sure to set "fPresentationTime" properly (in your "AACFramedSource" class) for each AAC frame. To get proper audio/video sync in each media player client, this presentation time must be properly aligned with the H.264 video presentation times, and with 'wall clock' time (i.e., the time that you'd get by calling "gettimeofday()").

I implemented presentation timestamping like this:
1. There's one shared timeval start presentation time variable
(startPresentationTime ) between AACFramedSource and H264FramedSource.
Whichever gets its sample first, it will set the startPresentationTime
to gettimeofday.
2. After that, in each FramedSource (AAC/H264) fPresentationTime is set
like this:
unsigned uSeconds = PlayTimeOfTheSampleInMicroSeconds;
fPresentationTime.tv_sec = (uSeconds/1000000) +
m_presentationStartTime.tv_sec;
fPresentationTime.tv_usec = (uSeconds%1000000) +
m_presentationStartTime.tv_usec;

PlayTimeOfTheSampleInMicroSeconds I get from the AAC/H264 encoders,
respectively.

Is this a good setup?

When I run this, and preview it with VLC, it is kind of working, but
there are lots of frames and audio buffers dropped, and I get a lot of
warning messages, like this:
main warning: playback too late (177441): up-sampling
main warning: playback way too late (181443): flushing buffers
main warning: picture is too late to be displayed (missing 655 ms)
main warning: picture is too late to be displayed (missing 616 ms)
main warning: picture is too late to be displayed (missing 576 ms)
main warning: picture is too late to be displayed (missing 536 ms)
...

Why can this be happening?

I even tried to set the presentationStartTime to be 1 second ahead in
the future, but to no avail, i.e.:
gettimeofday(&m_presentationStartTime, NULL);
// Set it 1 sec into the future
m_presentationStartTime.tv_sec += 1;

Thanks,
Goran.

[Ralf Globisch]

Post by Goran Ambardziev
main warning: playback too late (177441): up-sampling
main warning: playback way too late (181443): flushing buffers
main warning: picture is too late to be displayed (missing 655 ms)
main warning: picture is too late to be displayed (missing 616 ms)
main warning: picture is too late to be displayed (missing 576 ms)
main warning: picture is too late to be displayed (missing 536 ms)

Goran, AFAICT this is usually a VLC issue.

- We see these types of messages all the time in our platform (which
is implemented using live555 on the server.) A different media player
we've tried seems to play the stream without these "glitches".
- I see the same messages using my own RTSP/RTP stack written from scratch.


Not sure if it will help at all, but you could try playing around with the various caching
values in VLC.

[Ross Finlayson]

I also suggest making sure that:
- streaming H.264 video only works OK (I think you've already done this)
- streaming AAC audio only works OK
before trying to stream both audio and video together.


Ross Finlayson
Live Networks, Inc.
http://www.live555.com/