Handling RTCP Goodbye packet with OpenRTSP

[david cailliere]

Dear Ross,

Unfortunately, I can not provide you for the stream that illustrates the problem but I
did my investigation, and I think I'm close to find the reason why openRTSP is crashing.
Actually I'm playing two different stream simultaneously using the lastest release of
openRTSP. One MPEG4 video stream and one AMR audio stream.
I get the following sequence when the problem is happenning:

.....
server -> client AUDIO RTCP GOODBYE
crash
server -> client VIDEO RTCP GOODBYE
client -> server RTSP TEARDOWN
.....


The first call to "subsessionByeHandler()" close the audio media subsession's stream, which causes some medium object to be deleted.
But "shutdown()" is not yet called since the video subession is still active. Therefore the alarm handlers are not unscheduled and the previously deleted audio medium object is used later in the process when calling fDelayQueue.handleAlarm() in order to process some pending audio buffer.

So the crash only happens when there is some delay between the reception of the RTCP bye packet for the two different stream, and in the mean time the handlertimeout is not called.

So my question is:
Would it be possible to force the call to shutdown at the first call of subsessionByeHandler ?


Regards,

David

____________________________________________________

Prenez de l'avance pour vos cadeaux de Noël, découvrez notre sélection sur Voila : http://actu.voila.fr/evenementiel/noel-2010/boutique/

[Ross Finlayson]

Post by david cailliere
Would it be possible to force the call to shutdown at the first call
of subsessionByeHandler ?

Yes, it would certainly be 'possible'. However, I'm not going to do
that, because it's not the right thing to do. It's perfectly
reasonable (though uncommon) for the server to end one substream
early, while continuing to stream the other(s). In this case, we
want to continue to receive the other, ongoing stream(s), until they
also end.

Post by david cailliere
.....
server -> client AUDIO RTCP GOODBYE
crash
server -> client VIDEO RTCP GOODBYE
client -> server RTSP TEARDOWN
.....
The first call to "subsessionByeHandler()" close the audio media
subsession's stream, which causes some medium object to be deleted.
But "shutdown()" is not yet called since the video subession is
still active. Therefore the alarm handlers are not unscheduled and
the previously deleted audio medium object is used later in the
process when calling fDelayQueue.handleAlarm() in order to process
some pending audio buffer.

This is helpful, but still not quite enough information,
unfortunately. The closing of the audio subsession's stream *should*
be stopping any additional processing related to the audio stream, so
it that's not happening, then that will be the bug.

It would be nice to know exactly what alarm handler function is
getting called (after the audio subsession is closed) that's leading
to the crash.

[david cailliere]

Dear Ross

This is helpful, but still not quite enough information, unfortunately. The closing of the audio subsession's stream *should*
be stopping any additional processing related to the audio stream, so it that's not happening, then that will be the bug.
It would be nice to know exactly what alarm handler function is getting called (after the audio subsession is closed) that's leading to the crash.

I get a fisrt chance exception at 0x004125be in openRTSP.exe : (0xC0000005: acces Violation)
when running
sink->afterGettingFrame1(frameSize, presentationTime);

Please find below a screen shot of the call stack when the crash happens:

openRTSP.exe!FileSink::afterGettingFrame(void * clientData=0x00af3768, unsigned int frameSize=17, unsigned int __formal=0, timeval presentationTime={...}, unsigned int __formal=0) Ligne 88 + 0x14 octets C++
openRTSP.exe!FramedSource::afterGetting(FramedSource * source=0x0039d2f8) Ligne 91 + 0x2f octets C++
openRTSP.exe!AMRDeinterleaver::doGetNextFrame() Ligne 468 + 0x9 octets C++
openRTSP.exe!AMRDeinterleaver::afterGettingFrame1(unsigned int frameSize=17, timeval presentationTime={...}) Ligne 504 C++
openRTSP.exe!AMRDeinterleaver::afterGettingFrame(void * clientData=0x0039d2f8, unsigned int frameSize=17, unsigned int __formal=0, timeval presentationTime={...}, unsigned int __formal=0) Ligne 493 C++
openRTSP.exe!FramedSource::afterGetting(FramedSource * source=0x0039d0b0) Ligne 91 + 0x2f octets C++
openRTSP.exe!AlarmHandler::handleTimeout() Ligne 34 + 0xf octets C++
openRTSP.exe!DelayQueue::handleAlarm() Ligne 182 C++
openRTSP.exe!BasicTaskScheduler::SingleStep(unsigned int maxDelayTime=0) Ligne 151 C++
openRTSP.exe!BasicTaskScheduler0::doEventLoop(char * watchVariable=0x00000000) Ligne 77 C++
openRTSP.exe!main(int argc=2, char * * argv=0x00393d1c) Ligne 512 C++
openRTSP.exe!__tmainCRTStartup() Ligne 266 + 0x19 octets C
openRTSP.exe!mainCRTStartup() Ligne 182 C
kernel32.dll!7c817077()


Regards,

David


____________________________________________________

Prenez de l'avance pour vos cadeaux de Noël, découvrez notre sélection sur Voila : http://actu.voila.fr/evenementiel/noel-2010/boutique/

[Ross Finlayson]

The call stack helps a little, but unfortunately still doesn't nail
down the problem for me.

A couple more questions:
1/ What command-line arguments are you giving to "openRTSP" (apart
from the "rtsp://" URL)?
2/ Does the crash happen for streams that contain other kinds of
audio, or just those that contain AMR audio (in addition to video)?

[david cailliere]

Dear Ross,

1/ What command-line arguments are you giving to "openRTSP" (apart from the "rtsp://" URL)?

I'm using options "-Q -d x" for the test
where x is a duration which is smaller than the full length of the stream. A change on the duration parameter has no impact on the issue.

2/ Does the crash happen for streams that contain other kinds of
audio, or just those that contain AMR audio (in addition to video)?ilable

Unfortunately, I think there is only one kind of audio-video stream from the server I'm testing. But I've noticed that the crash never happens when the MPEG4 video RTCP bye packet is received before the AMR audio RTCP bye packet.

Regards,

David

____________________________________________________

Prenez de l'avance pour vos cadeaux de Noël, découvrez notre sélection sur Voila : http://actu.voila.fr/evenementiel/noel-2010/boutique/

[Ross Finlayson]

Post by Ross Finlayson
1/ What command-line arguments are you giving to "openRTSP"

(apart from the "rtsp://" URL)?
I'm using options "-Q -d x" for the test
where x is a duration which is smaller than the full length of the
stream. A change on the duration parameter has no impact on the
issue.

Are you sure that the crash can still occur, even if you omit the "-Q" option?

[david cailliere]

Dear Ross

Post by Ross Finlayson
Are you sure that the crash can still occur, even if you omit the "-Q" option?

Unfortunately, yes it does. Please find below the log end generated from the following command "OpenRTSP.exe -d 10 <rtspurl> when the crash occurs. The call stack is the same as the one I provide you before:

Received RTCP "BYE" on "audio/AMR" subsession (after 22 seconds)
validated RTCP subpacket (type 3): 1, 203, 0, 0x0146b2f8
validated entire RTCP packet
schedule(0.267904->1291201051.167388)
AMRDeinterleavingBuffer::deliverIncomingFrame(): new interleave group
AMRDeinterleavingBuffer::deliverIncomingFrame(): frameIndex 2 (2,0) put in bank
0, bin 0 (1): size 17, header 0x1c, presentationTime 1291201048.132577
AMRDeinterleavingBuffer::retrieveFrame(): from bank 1, bin 0: size 17, header 0x
1c, presentationTime 1291201048.112577

Hopping it can be helpful

Regards,

David

____________________________________________________

Prenez de l'avance pour vos cadeaux de Noël, découvrez notre sélection sur Voila : http://actu.voila.fr/evenementiel/noel-2010/boutique/

[Ross Finlayson]

Hoping it can be helpful

I see what is happening to cause the crash, but unfortunately I don't
understand how it can be happening.

The call to
Medium::close(subsession->sink);
in "subsessionAfterPlaying()" should be causing
AMRDeinterleaver::doStopGettingFrames()
to get called, and that should in turn be stopping the reception (and
thus handling) of any more AMR/RTP packets.

Once again, it would be nice to be able to experiment with a stream
that actually illustrates the problem...

[david cailliere]

Dear Ross,

The call to Medium::close(subsession->sink);
in "subsessionAfterPlaying()" should be causing
AMRDeinterleaver::doStopGettingFrames()
to get called, and that should in turn be stopping the reception (and
thus handling) of any more AMR/RTP packets.

Actually, the call to AMRDeinterleaver::doStopGettingFrames() does not ensure that the reception is stopped in every case. Sometimes the fNeedAFrame attribute can be set to true when the input source is still waiting for some RTP packet.

The rewritting of the function AMRDeinterleaver::doStopGettingFrames as below should fix the issue.


void AMRDeinterleaver::doStopGettingFrames() {
fNeedAFrame = False;
fInputSource->stopGettingFrames();
}

Since that modification, I did not manage to rash OpenRTSP anymore :-)

I don't know whether the issue only concerns AMRDeinterleaver class.

Regards,

David


____________________________________________________

Prenez de l'avance pour vos cadeaux de Noël, découvrez notre sélection sur Voila : http://actu.voila.fr/evenementiel/noel-2010/boutique/

[Ross Finlayson]

Post by david cailliere
Actually, the call to AMRDeinterleaver::doStopGettingFrames() does
not ensure that the reception is stopped in every case. Sometimes
the fNeedAFrame attribute can be set to true when the input source
is still waiting for some RTP packet.
The rewritting of the function AMRDeinterleaver::doStopGettingFrames
as below should fix the issue.
void AMRDeinterleaver::doStopGettingFrames() {
fNeedAFrame = False;
fInputSource->stopGettingFrames();
}

Yes, that will likely overcome the problem. Thanks for tracking this down.

Post by david cailliere
I don't know whether the issue only concerns AMRDeinterleaver class.

The same problem existed with "QCELPAudioRTPSource". Fixes for both
will be included in the next release of the code. Thanks again.