Bug 1494

Summary: GLMediaPlayer/GraphUI: Support Displaying Bitmap'ed Subtitles (PGS ..)
Product: [JogAmp] Jogl Reporter: Sven Gothel <sgothel>
Component: graphAssignee: Sven Gothel <sgothel>
Status: RESOLVED FIXED    
Severity: normal CC: sgothel
Priority: P4    
Version: 2.6.0   
Hardware: All   
OS: all   
Type: FEATURE SCM Refs:
a19f810c9618e2fa6829f1c157d2e1a88ca178de 670286747af423781aa4f54062aa34345d32a397 72065d9e2a9d6480c34afc9920071d5ed577d37a b8b692e9cf3120d788ae912514f54948ddccacf5 9fe460aecf8509c8305416bb9d77aadf6165c677 23cf5279472d3ae1b2d8d1904e6b1f1e7fd8f012 1639bf159810895eb24791c8128a7b2c02d15a56
Workaround: ---
Bug Depends on: 1492    
Bug Blocks: 805    

Description Sven Gothel 2024-01-31 11:01:10 CET
Bitmap Subtitles shall be made available via FFMPEGMediaPlayer implementation and utilized via GraphUI MediaButton as well as via the full feature widget MediaPlayer.
Comment 1 Sven Gothel 2024-01-31 11:02:43 CET
commit a19f810c9618e2fa6829f1c157d2e1a88ca178de

    FFMPEGPlayer: Prep for bitmap'ed subtitles: Use glEnable()/glBindTexture() func-ptr in native; readNextPacket0() passes video+subtitle texTarget and texID
    
    For bitmap subtitles we need to push the bitmap into its own texture.
    Hence readNextPacket0() must switch to used texture using glEnable() on !core and glBindTexture().

+++

commit 670286747af423781aa4f54062aa34345d32a397

    GLMediaPlayer: Add getSubtitleCodec()


+++
Comment 2 Sven Gothel 2024-02-02 07:32:42 CET
commit 72065d9e2a9d6480c34afc9920071d5ed577d37a

    Bug 1494: Add SubTextureEvent & SubEmptyEvent; Add SubtitleEvent.release() provided by owner to take back borrowed resources (texture of bitmap subtitle)

+++

commit b8b692e9cf3120d788ae912514f54948ddccacf5

    GLMediaPlayer: Split GLMediaFrameListener (rarely used) from GLMediaEventListener, easing listener callbacks; Prepare SubtitleEventListener generalization (Bug 1494)
    
    Moves pushSound(), pushSubtitle*() from FFMPEGMediaPlayer to GLMediaPlayerImpl,
    as it is handled in a generic way - even though currently only called by native FFMPEGMediaPlayer implementation.
    
    Note: This patch is incomplete, i.e. not even compile clean.
    But choses as-is to semantically split the work to ease review.

+++

commit 9fe460aecf8509c8305416bb9d77aadf6165c677

    Bug 1494: Clarify SubtitleEvent class, adding general FFmpeg analog CodecID and hence also promoting VideoPixelFormat
Comment 3 Sven Gothel 2024-02-02 08:52:25 CET
commit 23cf5279472d3ae1b2d8d1904e6b1f1e7fd8f012

Bug 1494 - GLMediaPlayer/GraphUI: Support Displaying Bitmap'ed Subtitles (PGS ..) via FFMPEGFMediaPlayer/FFmpeg
    
FFMPEGFMediaPlayer related changes:
- Add libswscale (6th FFmpeg lib used) for sws_getCachedContext(), sws_scale() and sws_freeContext(),
  used natively to convert the palette'ed bitmap into RGBA colorspace -> GL texture
    
- Handling AVSubtitleRect.type SUBTITLE_BITMAP
-- only handled if libswscale is available
-- config/adjust texture object
-- sws_scale palette'ed bitmap to texture
-- intermediate memory is cached, may be resized and free'ed at destroy
-- texture objects are managed and passed from GLMediaPlayerImpl,
   as they are also forwarded to player client via SubBitmapEvent
    
- Passing the AVCodecID to GLMediaPlayerImpl, converted to our CodecID enum.
    
- Unifying creation and opening of AVCodecContext with 'createOpenedAVCodecContext(..)'
    
+++ 
    
SubtitleEvent*
- SubTextEvent now also handles ASS.Dialogue (FFmpeg 4)
  besides ASS.Event (FFmpeg 5, 6, ..).
    
+++ 
    
GLMediaPlayerImpl
- Added ringbuffer subTexFree, managing Texture for bitmap'ed subtitles
-- Uses 1 bitmap-subtitle Texture per used textureCount in cache,
   as one bitmap-subtile can be displayed per frame.
   Could be potentially reduced to just 2 .. but resources used are 
   relatively low here.
    
- Validating subTexFree + videoFramesFree usage,
  use blocking get/put ringbuffer due to utilization from different threads.
    
- Receives subtitle content from native getNextPacket0() via callback,
  creates SubtitleEvent instance and passes it to a SubtitleEventListener - if exists.
  (See MediaButton example)
    
-- SubBitmapEvent also gets its special SubBitmapEvent.TextureOwner to handle client releasing
   the event and allowing us to put back the Texture resource to 'subTexFree'.
   This passing through of the Texture object is probably a weakness of this lifecycle
   and requires the client to ensure SubtitleEvent.release() gets called.
   See MediaButton example!
  
- Exposing CodecID, allowing clients like MediaButton to handle SubtitleEvent content according to codec
Comment 4 Sven Gothel 2024-02-02 08:53:49 CET
commit 1639bf159810895eb24791c8128a7b2c02d15a56 (HEAD -> jausoft_01)

    GLMediaPlayerImpl: Disable subDEBUG flag