public interface GLMediaPlayer extends TextureSequence
TextureSequence
state machine
using a multiplexed audio/video stream as it's source.
Audio maybe supported and played back internally or via an AudioSink
implementation.
Audio and video streams can be selected or muted via initStream(Uri, int, int, int)
using the appropriate stream id's.
Camera input can be selected using the CameraInputScheme
Uri.
Most of the stream processing is performed on the decoding thread, a.k.a. StreamWorker:
initStream(..)
- User gets notified whether the stream has been initialized or not via attributesChanges(..)
.newFrameAvailable(...)
.GLMediaPlayer.StreamException
s.GLContext
, shared with the one passed to initGL(GL)
.
The shared GLContext
allows the decoding thread to push the video frame data directly into
the designated TextureFrame
, later returned via getNextTexture(GL)
and used by the user.
Caught exceptions on StreamWorker are delivered as GLMediaPlayer.StreamException
s,
which either degrades the GLMediaPlayer.State
to GLMediaPlayer.State.Uninitialized
or GLMediaPlayer.State.Paused
.
An occurring GLMediaPlayer.StreamException
triggers a EVENT_CHANGE_ERR
event,
which can be listened to via GLMediaPlayer.GLMediaEventListener.attributesChanged(GLMediaPlayer, int, long)
.
An occurred GLMediaPlayer.StreamException
can be read via getStreamException()
.
value | request | get |
---|---|---|
STREAM_ID_NONE | mute | not available |
STREAM_ID_AUTO | auto | unspecified |
≥0 | specific stream | specific stream |
Current implementations (check each API doc link for details):
NullGLMediaPlayer
jogamp.opengl.util.av.impl.OMXGLMediaPlayer
jogamp.opengl.util.av.impl.FFMPEGMediaPlayer
jogamp.opengl.android.av.AndroidGLMediaPlayerAPI14
Implementations of this interface must implement:
public static final boolean isAvailable();to be properly considered by
GLMediaPlayerFactory.create(ClassLoader, String)
and GLMediaPlayerFactory.createDefault()
.
Timestamp type and value range has been chosen to suit embedded CPUs
and characteristics of audio and video streaming. See TimeFrameI
.
The class follows a passive A/V synchronization pattern.
Audio is being untouched, while getNextTexture(GL)
delivers a new video frame
only, if its timestamp is less than MAXIMUM_VIDEO_ASYNC
ahead of time.
If its timestamp is more than MAXIMUM_VIDEO_ASYNC
ahead of time,
the previous frame is returned.
If its timestamp is more than MAXIMUM_VIDEO_ASYNC
after time,
the frame is dropped and the next frame is being fetched.
https://en.wikipedia.org/wiki/Audio_to_video_synchronization
d_av = v_pts - a_pts;
Recommendation of audio/video pts time lead/lag at production:
Recommendation of av pts time lead/lag at presentation:
Big Buck Bunny 24f 16:9 | ||||
---|---|---|---|---|
Big Buck Bunny | 320p | h264 | aac 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4 |
Big Buck Bunny | 240p | h264 | aac 48000Hz 2 chan | http://archive.org/download/BigBuckBunny_328/BigBuckBunny_512kb.mp4 |
Big Buck Bunny | 720p | mpeg4 | ac3 48000Hz 5.1 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_surround.avi |
Big Buck Bunny | 720p | msmpeg4v2 | mp3 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.avi |
Big Buck Bunny | 720p | theora | vorbis 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.ogg |
Big Buck Bunny | 1080p | mpeg4 | ac3 48000Hz 5.1 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_surround.avi |
WebM/Matroska (vp8/vorbis) | ||||
Big Buck Bunny Trailer | 640p | vp8 | vorbis 44100Hz 1 chan | http://video.webmfiles.org/big-buck-bunny_trailer.webm |
Elephants Dream | 540p | vp8 | vorbis 44100Hz 1 chan | http://video.webmfiles.org/elephants-dream.webm |
You Tube http/rtsp | ||||
Sintel | http://www.youtube.com/watch?v=eRsGyueVLvQ | rtsp://v3.cache1.c.youtube.com/CiILENy73wIaGQn0LpXnygYbeRMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp | ||
Audio/Video Sync | ||||
Five-minute-sync-test1080p | https://www.youtube.com/watch?v=szoOsG9137U | rtsp://v7.cache8.c.youtube.com/CiILENy73wIaGQm133VvsA46sxMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp | ||
Audio-Video-Sync-Test-Calibration-23.98fps-24fps | https://www.youtube.com/watch?v=cGgf_dbDMsw | |||
sound_in_sync_test | https://www.youtube.com/watch?v=O-zIZkhXNLE |
Modifier and Type | Interface and Description |
---|---|
static interface |
GLMediaPlayer.GLMediaEventListener |
static class |
GLMediaPlayer.State
See Lifecycle.
|
static class |
GLMediaPlayer.StreamException
A StreamException encapsulates a caught exception in the decoder thread, a.k.a StreamWorker,
see See StreamWorker Error Handling.
|
TextureSequence.TexSeqEventListener<T extends TextureSequence>, TextureSequence.TextureFrame
Modifier and Type | Field and Description |
---|---|
static Uri.Encoded |
CameraInputScheme
Uri scheme name for camera input. |
static String |
CameraPropHeight
Camera property "height".
|
static String |
CameraPropRate
Camera property "rate".
|
static String |
CameraPropSizeS
Camera property "size", size as string, e.g.
|
static String |
CameraPropWidth
Camera property "width".
|
static boolean |
DEBUG |
static boolean |
DEBUG_NATIVE |
static int |
MAXIMUM_VIDEO_ASYNC
Maximum video frame async of 22 milliseconds.
|
static int |
STREAM_ID_AUTO
Constant -1 for auto or unspecified.
|
static int |
STREAM_ID_NONE
Constant -2 for mute or not available.
|
static int |
TEXTURE_COUNT_DEFAULT
Default texture count, value 4.
|
static int |
TEXTURE_COUNT_MIN
Minimum texture count, value 1.
|
sampler2D, samplerExternalOES
Modifier and Type | Method and Description |
---|---|
void |
addEventListener(GLMediaPlayer.GLMediaEventListener l)
Adds a
GLMediaPlayer.GLMediaEventListener to this player. |
Object |
attachObject(String name,
Object obj)
Attaches the user object for the given name.
|
GLMediaPlayer.State |
destroy(GL gl)
Releases the GL, stream and other resources, including
attached user objects . |
Object |
detachObject(String name)
Detaches the user object for the given name.
|
int |
getAID()
Return the audio stream id, see audio and video Stream IDs.
|
Object |
getAttachedObject(String name)
Returns the attached user object for the given name.
|
int |
getAudioBitrate()
Warning: Optional information, may not be supported by implementation.
|
String |
getAudioCodec()
Warning: Optional information, may not be supported by implementation.
|
int |
getAudioFrames()
Warning: Optional information, may not be supported by implementation.
|
int |
getAudioPTS() |
AudioSink |
getAudioSink()
If implementation uses a
AudioSink , it's instance will be returned. |
float |
getAudioVolume()
Returns the audio volume.
|
int |
getDecodedFrameCount() |
int |
getDuration() |
GLMediaPlayer.GLMediaEventListener[] |
getEventListeners()
Return all
GLMediaPlayer.GLMediaEventListener of this player. |
float |
getFramerate()
Warning: Optional information, may not be supported by implementation.
|
int |
getHeight()
Returns the height of the video.
|
TextureSequence.TextureFrame |
getLastTexture()
Returns the last updated texture.
|
TextureSequence.TextureFrame |
getNextTexture(GL gl)
Returns the next texture to be rendered.
|
String |
getPerfString()
Returns a string represantation of this player's performance values.
|
float |
getPlaySpeed()
Returns the playback speed.
|
int |
getPresentedFrameCount() |
GLMediaPlayer.State |
getState()
See Lifecycle.
|
long |
getStreamBitrate()
Warning: Optional information, may not be supported by implementation.
|
GLMediaPlayer.StreamException |
getStreamException()
Returns the
GLMediaPlayer.StreamException caught in the decoder thread, or null if none occured. |
int |
getTextureCount() |
Uri |
getUri()
Return the stream location, as set by
initStream(Uri, int, int, int) . |
int |
getVID()
Return the video stream id, see audio and video Stream IDs.
|
int |
getVideoBitrate()
Warning: Optional information, may not be supported by implementation.
|
String |
getVideoCodec()
Warning: Optional information, may not be supported by implementation.
|
int |
getVideoFrames()
Warning: Optional information, may not be supported by implementation.
|
int |
getVideoPTS() |
int |
getWidth()
Returns the width of the video.
|
void |
initGL(GL gl)
Initializes OpenGL related resources.
|
void |
initStream(Uri streamLoc,
int vid,
int aid,
int textureCount)
Issues asynchronous stream initialization.
|
boolean |
isGLOriented()
Returns
true if the video frame is oriented in
OpenGL's coordinate system, origin at bottom left. |
GLMediaPlayer.State |
pause(boolean flush)
Pauses the StreamWorker decoding thread.
|
GLMediaPlayer.State |
play()
Starts or resumes the StreamWorker decoding thread.
|
void |
removeEventListener(GLMediaPlayer.GLMediaEventListener l)
Removes a
GLMediaPlayer.GLMediaEventListener to this player. |
int |
seek(int msec)
Seeks to the new absolute position.
|
boolean |
setAudioVolume(float v)
Sets the audio volume, [0f..1f].
|
boolean |
setPlaySpeed(float rate)
Sets the playback speed.
|
void |
setTextureMinMagFilter(int[] minMagFilter)
Sets the texture min-mag filter, defaults to
GL.GL_NEAREST . |
void |
setTextureUnit(int u)
Sets the texture unit.
|
void |
setTextureWrapST(int[] wrapST)
Sets the texture min-mag filter, defaults to
GL.GL_CLAMP_TO_EDGE . |
String |
toString()
Returns a string represantation of this player, incl.
|
getRequiredExtensionsShaderStub, getTextureFragmentShaderHashCode, getTextureLookupFragmentShaderImpl, getTextureLookupFunctionName, getTextureMinMagFilter, getTextureSampler2DType, getTextureTarget, getTextureUnit, getTextureWrapST, isTextureAvailable
static final boolean DEBUG
static final boolean DEBUG_NATIVE
static final int TEXTURE_COUNT_DEFAULT
static final int TEXTURE_COUNT_MIN
static final int STREAM_ID_NONE
static final int STREAM_ID_AUTO
static final Uri.Encoded CameraInputScheme
Uri scheme
name for camera input. E.g. camera:/0
for the 1st camera device.
The Uri path
is being used to identify the camera (ID),
where the root fwd-slash is being cut-off.
The ID is usually an integer value indexing the camera ranging from [0..max-number].
The Uri query
is used to pass options to the camera
using ; as the separator. The latter avoids trouble w/ escaping.
camera:/camera://somewhere/ camera://somewhere/ ?width=640;height=480;rate=15 camera://somewhere/ ?size=640x480;rate=15
Uri: [scheme:][//authority][path][?query][#fragment] w/ authority: [user-info@]host[:port] Note: 'path' starts w/ fwd slash
static final String CameraPropSizeS
1280x720
, hd720
. May not be supported on all platforms. See CameraInputScheme
.static final String CameraPropWidth
CameraInputScheme
.static final String CameraPropHeight
CameraInputScheme
.static final String CameraPropRate
CameraInputScheme
.static final int MAXIMUM_VIDEO_ASYNC
int getTextureCount()
void setTextureUnit(int u)
void setTextureMinMagFilter(int[] minMagFilter)
GL.GL_NEAREST
.void setTextureWrapST(int[] wrapST)
GL.GL_CLAMP_TO_EDGE
.void initStream(Uri streamLoc, int vid, int aid, int textureCount) throws IllegalStateException, IllegalArgumentException
Lifecycle: GLMediaPlayer.State.Uninitialized
-> GLMediaPlayer.State.Initialized
1 or GLMediaPlayer.State.Uninitialized
GLMediaPlayer.State.Initialized
is reached asynchronous,
i.e. user gets notified via attributesChanges(..)
.
A possible caught asynchronous GLMediaPlayer.StreamException
while initializing the stream off-thread
will be thrown at initGL(GL)
.
Muted audio can be achieved by passing STREAM_ID_NONE
to aid
.
Muted video can be achieved by passing STREAM_ID_NONE
to vid
,
in which case textureCount
is ignored as well as the passed GL object of the subsequent initGL(GL)
call.
streamLoc
- the stream locationvid
- video stream id, see audio and video Stream IDsaid
- video stream id, see audio and video Stream IDstextureCount
- desired number of buffered textures to be decoded off-thread, will be validated by implementation.
The minimum value is TEXTURE_COUNT_DEFAULT
.
Ignored if video is muted.IllegalStateException
- if not invoked in GLMediaPlayer.State.Uninitialized
IllegalArgumentException
- if arguments are invalidGLMediaPlayer.StreamException getStreamException()
GLMediaPlayer.StreamException
caught in the decoder thread, or null
if none occured.
Method clears the cached GLMediaPlayer.StreamException
, hence an immediate subsequent call will return null
.
void initGL(GL gl) throws IllegalStateException, GLMediaPlayer.StreamException, GLException
Lifecycle: GLMediaPlayer.State.Initialized
-> GLMediaPlayer.State.Paused
or GLMediaPlayer.State.Initialized
gl
is ignored if video is muted, see initStream(Uri, int, int, int)
.gl
- current GL object. Maybe null
, for audio only.IllegalStateException
- if not invoked in GLMediaPlayer.State.Initialized
.GLMediaPlayer.StreamException
- forwarded from the off-thread stream initializationGLException
- in case of difficulties to initialize the GL resourcesAudioSink getAudioSink()
AudioSink
, it's instance will be returned.
The AudioSink
instance is available after initStream(Uri, int, int, int)
,
if used by implementation.
GLMediaPlayer.State destroy(GL gl)
attached user objects
.
boolean setPlaySpeed(float rate)
To simplify test, play speed is normalized, i.e.
1.0f
: if Math.abs(1.0f - rate) < 0.01f
float getPlaySpeed()
boolean setAudioVolume(float v)
To simplify test, volume is normalized, i.e.
0.0f
: if Math.abs(v) < 0.01f
1.0f
: if Math.abs(1.0f - v) < 0.01f
float getAudioVolume()
GLMediaPlayer.State play()
Lifecycle: GLMediaPlayer.State.Paused
-> GLMediaPlayer.State.Playing
GLMediaPlayer.State pause(boolean flush)
Lifecycle: GLMediaPlayer.State.Playing
-> GLMediaPlayer.State.Paused
If a new frame is desired after the next play()
call,
e.g. to make a snapshot of a camera input stream,
flush
shall be set to true
.
flush
- if true
flushes the video and audio buffers, otherwise keep them intact.int seek(int msec)
Allowed in state GLMediaPlayer.State.Playing
and GLMediaPlayer.State.Paused
, otherwise ignored,
see Lifecycle.
msec
- absolute desired time position in millisecondsGLMediaPlayer.State getState()
GLMediaPlayer.State.Uninitialized
, GLMediaPlayer.State.Initialized
, GLMediaPlayer.State.Playing
or GLMediaPlayer.State.Paused
int getVID()
int getAID()
int getDecodedFrameCount()
play()
and seek(int)
as increased by getNextTexture(GL)
or the decoding thread.int getPresentedFrameCount()
play()
and seek(int)
as increased by getNextTexture(GL)
for new frames.int getVideoPTS()
getLastTexture()
int getAudioPTS()
TextureSequence.TextureFrame getLastTexture() throws IllegalStateException
In case the instance is just initialized, it shall return a TextureFrame
object with valid attributes. The texture content may be undefined
until the first call of TextureSequence.getNextTexture(GL)
.
getLastTexture
in interface TextureSequence
IllegalStateException
- if not invoked in GLMediaPlayer.State.Paused
or GLMediaPlayer.State.Playing
TextureSequence.TextureFrame getNextTexture(GL gl) throws IllegalStateException
Implementation shall return the next frame if available, may block if a next frame may arrive soon. Otherwise implementation shall return the last frame.
Shall return null
in case no next or last frame is available.
In case the current state is not GLMediaPlayer.State.Playing
, getLastTexture()
is returned.
getNextTexture
in interface TextureSequence
IllegalStateException
- if not invoked in GLMediaPlayer.State.Paused
or GLMediaPlayer.State.Playing
addEventListener(GLMediaEventListener)
,
GLMediaEventListener#newFrameAvailable(GLMediaPlayer, TextureFrame, long)
Uri getUri()
initStream(Uri, int, int, int)
.String getVideoCodec()
String getAudioCodec()
int getVideoFrames()
int getAudioFrames()
int getDuration()
long getStreamBitrate()
int getVideoBitrate()
int getAudioBitrate()
float getFramerate()
boolean isGLOriented()
true
if the video frame is oriented in
OpenGL's coordinate system, origin at bottom left.
Otherwise returns false
, i.e.
video frame is oriented origin at top left.
false
is the default assumption for videos,
but user shall not rely on.
false
GL orientation leads to
Texture.getMustFlipVertically()
== true
,
as reflected by all TextureFrame
's Texture
s
retrieved via getLastTexture()
or getNextTexture(GL)
.
int getWidth()
int getHeight()
String toString()
String getPerfString()
void addEventListener(GLMediaPlayer.GLMediaEventListener l)
GLMediaPlayer.GLMediaEventListener
to this player.void removeEventListener(GLMediaPlayer.GLMediaEventListener l)
GLMediaPlayer.GLMediaEventListener
to this player.GLMediaPlayer.GLMediaEventListener[] getEventListeners()
GLMediaPlayer.GLMediaEventListener
of this player.Object getAttachedObject(String name)
Object attachObject(String name, Object obj)
Copyright 2010 JogAmp Community.