The current FFMPEGMediaPlayer implementation call the native packet decoder natives.readNextPacket0 for both audio and video during getNextTexture(GL) http://jogamp.org/git/?p=jogl.git;a=blob;f=src/jogl/classes/jogamp/opengl/util/av/impl/FFMPEGMediaPlayer.java#l796 this implementation impede use of the FFMPEGMediaPlayer GLMediaPlayer backend for audio only playback when getNextTexture(GL) is not called periodically. FFMPEGMediaPlayer should decode audio and video packets in a separate thread during play state in order to support audio only use. By making the decoder multithreaded may as well help reduce latency by making sure that both video and audio frames are queued up in time before playback. http://jogamp.org/deployment/jogamp-next/javadoc/jogl/javadoc/com/jogamp/opengl/util/av/GLMediaPlayer.html
It is possible to reproduce this bug using the MovieSimple test by simply passing -vid -2 and produces no audio output for audio only playback. passing -vid -1 work and produce audio output for audio only playback. -2 == GLMediaPlayer.STREAM_ID_NONE == OFF -1 == AUTO Reproducer: java -cp jogamp-all-platforms/jar/jogl-test.jar:jogamp-all-platforms/jar/gluegen-rt.jar:jogamp-all-platforms/jar/jogl-all.jar:jogamp-all-platforms/jar/joal.jar com.jogamp.opengl.test.junit.jogl.demos.es2.av.MovieSimple -vid -2
Main issue: StreamWorker did not get created and started for -vid -2 inside src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java textureCount = 0; for -vid -2 StreamWorker is responsible for decoding both audio and video in a separate thread. TODO: review StreamWorker for -vid -2 TODO: review StreamWorker, GLMediaPlayer and FFMPEGMediaPlayer when to stop using an audio only stream. The current implementation decide to stop when it encounter an end of stream _video_ frame. Suggested fix: diff --git a/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java b/src index 86e19c9..7bc61b7 100644 --- a/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java +++ b/src/jogl/classes/jogamp/opengl/util/av/GLMediaPlayerImpl.java @@ -528,7 +528,7 @@ public abstract class GLMediaPlayerImpl implements GLMediaPl this.vid = vid; this.aid = aid; if ( this.streamLoc != null ) { - if( TEXTURE_COUNT_MIN < textureCount ) { + if( streamWorker == null ) { streamWorker = new StreamWorker(); } else { new Thread() {
Bug 918 (1/2): Use StreamWorker in 'Audio Only' mode, since no 'getNextTexture(..)' is issued here! Thanks to Xerxes to analyze this issue thoroughly. TODO: Implement EOS for 'Audio Only' and test seek, pause, etc .. - Apply manual tests in MovieSimple
a1be0f69bacb315e40a017b8997ef1c610da576e GLMediaPlayer: Fix Deadlock if EOS happens after pause/resume (seek) - Tested w/ seeking 'Audio Only' and Matroska Test stream was default of MovieSimple: http://video.webmfiles.org/big-buck-bunny_trailer.webm while disabling video (-vid -2) 8a8ed735f6631b2da7bf605c5c3dda4e0fc13905 Determine StreamWorker usage after init - To support audio only files, we need to determine to use StreamWorker after completion of stream-init. Fix seek(..) - FFMPeg: pos0 needs to use aPTS for audio-only - Clip target time [0..duration[ Fallback for EOS Detection In case the backend does not report proper EOS: - Utilize 'nullFramesCount >= MAX' -> EOS, where MAX is number of frames for 3s play duraction and where 'nullFramesCount' is increased if no valid packet is available and no decoded-video or -audio in the queue. - Utilize pts > duration -> EOS MovieSimple uses full GLEventListener for 'Audio Only' as well to test seek - Matroska seek for audio-only leads to EOS .. http://video.webmfiles.org/big-buck-bunny_trailer.webm - MP4 audio-only seek works http://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4 MovieSimple/MovieCube: - Use audio-pts in audio-only to calc target time Tested: - A, V and A+V - Pause, Stop and Seek - GNU/Linux d42b236ef139dcc8d8713535893c7870b55d420c ALAudioSink: Fix stop and flush of OpenAL source - stopImpl() shall always issue alSourceStop(..) if state is not STOPPED - Remove 'flush' hint for dequeueBuffer(..), we perform proper flush in respective method, see below - flush() needs to issue: - stopImpl() - which should already dequeue all buffers - Explicitly dequeue all buffers: via 'alSourcei(alSource[0], AL.AL_BUFFER, 0)' - Then dequeue manually processed buffers: dequeueBuffer( false /* wait */ ); - And dequeue _all_ buffers: dequeueForceAll(); a0b23035bf9de987f5ad1b00fe3630c102f4e513 Fix EOS Regression: Only use pts>duration for EOS if duration > 0 (camera or other sources may not have duration) Regression of commit 8a8ed735f6631b2da7bf605c5c3dda4e0fc13905 Tested on Android as well ..