725 2013-04-24 23:07:36 +0200 FFMPEGMediaPlayer audio frame decode and playback 2013-08-31 10:53:24 +0200 1 1 3 JogAmp Jogl core 2 All all RESOLVED FIXED --- enhancement --- 1 xerxes sgothel sgothel xerxes --- 062f3e9e1ecfeb63ea61eb540d94d17a2de1412a ... --- oldest_to_newest 2519 0 457 xerxes 2013-04-24 23:07:36 +0200 Created attachment 457 decode.c ffmpeg/libav audio decode and openal playback native example. Adding audio playback support to FFMPEGMediaPlayer will make it awesome. The attached decode.c demonstrate audio decode using ffmpeg/libav and playback using openal. We may choose to let the JOGL util FFMPEGMediaPlayer package depend on JOAL in order to facilitate easy playback of decoded audio frames. 2541 1 xerxes 2013-04-29 00:01:56 +0200 I have started working on this bug, a in progress git branch is available: https://github.com/xranby/jogl/commits/FFMPEGMediaPlayer-audio 2585 2 xerxes 2013-05-12 08:05:47 +0200 I have now implemented a crude proof of concept FFMPEGMediaPlayer with plain Java Sound and a most simple a/v sync. I only buffer audio frames for now. Video is played a bit ahead of time in order to generate enough audio frame buffer to prevent audio buffer under-runs during video "Thread.sleep" sync. https://github.com/xranby/jogl/commit/062f3e9e1ecfeb63ea61eb540d94d17a2de1412a 2925 3 sgothel 2013-08-31 10:53:24 +0200 .. commits are to numerous to list here, however, Xerxes's brought in audio frame decoding w/ JOAL output support! Latest FFMPEG GLMediaPlayer / ALAudioSink refined such support ranging from: [from 6332e13b2f0aa9818d37802302f04c90a4fa4239 to 1693604b4daae4571069b58bdc863516b95999a6] 457 2013-04-24 23:07:36 +0200 2013-04-24 23:07:36 +0200 decode.c ffmpeg/libav audio decode and openal playback native example. decode.c text/x-csrc 6862 xerxes LyoKICogZmZtcGVnL2xpYmF2IGF1ZGlvIGRlY29kZSBhbmQgcGxheWJhY2sgdXNpbmcgb3BlbmFs CiAqCiAqCiAqIHNvdXJjZXMgb2YgaW5zcGlyYXRpb246CiAqIGh0dHA6Ly9ibGlua2luZ2JsaXAu d29yZHByZXNzLmNvbS8yMDExLzEwLzA4L2RlY29kaW5nLWFuZC1wbGF5aW5nLWFuLWF1ZGlvLXN0 cmVhbS11c2luZy1saWJhdmNvZGVjLWxpYmF2Zm9ybWF0LWFuZC1saWJhby8KICogaHR0cDovL2dp dC5saWJhdi5vcmcvP3A9bGliYXYuZ2l0O2E9YmxvYjtmPWxpYmF2Y29kZWMvYXBpLWV4YW1wbGUu YztoYj1IRUFECiAqIGh0dHA6Ly9rY2F0LnN0cmFuZ2Vzb2Z0Lm5ldC9vcGVuYWwtdHV0b3JpYWwu aHRtbAogKgogKiBjb21waWxlOgogKiBnY2MgZGVjb2RlLmMgLWxhdmZvcm1hdCAtbG9wZW5hbAog KgogKiBDaGVlcnMKICogWGVyeGVzIFLDpW5ieQogKi8KCiNpbmNsdWRlIDxzdGRpby5oPgoKI2lu Y2x1ZGUgPGxpYmF2Y29kZWMvYXZjb2RlYy5oPgojaW5jbHVkZSA8bGliYXZmb3JtYXQvYXZmb3Jt YXQuaD4KI2luY2x1ZGUgPGxpYmF2dXRpbC9hdnV0aWwuaD4KCiNpbmNsdWRlIDxBTC9hbC5oPgoj aW5jbHVkZSA8QUwvYWxjLmg+Cgp2b2lkIGRpZShjb25zdCBjaGFyKiBtZXNzYWdlKQp7CiAgICBm cHJpbnRmKHN0ZGVyciwgIiVzXG4iLCBtZXNzYWdlKTsKICAgIGV4aXQoMSk7Cn0KCmludCBtYWlu KGludCBhcmdjLCBjaGFyKiBhcmd2W10pCnsKICAgIGlmIChhcmdjIDwgMikgewogICAgICAgIGRp ZSgiUGxlYXNlIHByb3ZpZGUgdGhlIGZpbGUgcGF0aCBhcyB0aGUgZmlyc3QgYXJndW1lbnQiKTsK ICAgIH0KCiAgICBjb25zdCBjaGFyKiBpbnB1dF9maWxlbmFtZSA9IGFyZ3ZbMV07CgogICAgLy8g VGhpcyBjYWxsIGlzIG5lY2Vzc2FyaWx5IGRvbmUgb25jZSBpbiB5b3VyIGFwcCB0byBpbml0aWFs aXplCiAgICAvLyBsaWJhdmZvcm1hdCB0byByZWdpc3RlciBhbGwgdGhlIG11eGVycywgZGVtdXhl cnMgYW5kIHByb3RvY29scy4KICAgIGF2X3JlZ2lzdGVyX2FsbCgpOwoKICAgIC8vIEEgbWVkaWEg Y29udGFpbmVyCiAgICBBVkZvcm1hdENvbnRleHQqIGNvbnRhaW5lciA9IDA7CgogICAgaWYgKGF2 Zm9ybWF0X29wZW5faW5wdXQoJmNvbnRhaW5lciwgaW5wdXRfZmlsZW5hbWUsIE5VTEwsIE5VTEwp IDwgMCkgewogICAgICAgIGRpZSgiQ291bGQgbm90IG9wZW4gZmlsZSIpOwogICAgfQoKICAgIGlm IChhdl9maW5kX3N0cmVhbV9pbmZvKGNvbnRhaW5lcikgPCAwKSB7CiAgICAgICAgZGllKCJDb3Vs ZCBub3QgZmluZCBmaWxlIGluZm8iKTsKICAgIH0KCiAgICBpbnQgc3RyZWFtX2lkID0gLTE7Cgog ICAgLy8gVG8gZmluZCB0aGUgZmlyc3QgYXVkaW8gc3RyZWFtLiBUaGlzIHByb2Nlc3MgbWF5IG5v dCBiZSBuZWNlc3NhcnkKICAgIC8vIGlmIHlvdSBjYW4gZ3VyYXJhbnRlZSB0aGF0IHRoZSBjb250 YWluZXIgY29udGFpbnMgb25seSB0aGUgZGVzaXJlZAogICAgLy8gYXVkaW8gc3RyZWFtCiAgICBp bnQgaTsKICAgIGZvciAoaSA9IDA7IGkgPCBjb250YWluZXItPm5iX3N0cmVhbXM7IGkrKykgewog ICAgICAgIGlmIChjb250YWluZXItPnN0cmVhbXNbaV0tPmNvZGVjLT5jb2RlY190eXBlID09IEFW TUVESUFfVFlQRV9BVURJTykgewogICAgICAgICAgICBzdHJlYW1faWQgPSBpOwogICAgICAgICAg ICBicmVhazsKICAgICAgICB9CiAgICB9CgogICAgaWYgKHN0cmVhbV9pZCA9PSAtMSkgewogICAg ICAgIGRpZSgiQ291bGQgbm90IGZpbmQgYW4gYXVkaW8gc3RyZWFtIik7CiAgICB9CgogICAgLy8g RmluZCB0aGUgYXByb3ByaWF0ZSBjb2RlYyBhbmQgb3BlbiBpdAogICAgQVZDb2RlY0NvbnRleHQq IGNvZGVjX2NvbnRleHQgPSBjb250YWluZXItPnN0cmVhbXNbc3RyZWFtX2lkXS0+Y29kZWM7CiAg ICBBVkNvZGVjKiBjb2RlYyA9IGF2Y29kZWNfZmluZF9kZWNvZGVyKGNvZGVjX2NvbnRleHQtPmNv ZGVjX2lkKTsKCiAgICBpZiAoIWF2Y29kZWNfb3Blbihjb2RlY19jb250ZXh0LCBjb2RlYykgPCAw KSB7CiAgICAgICAgZGllKCJDb3VsZCBub3QgZmluZCBvcGVuIHRoZSBuZWVkZWQgY29kZWMiKTsK ICAgIH0KCiAgICAvLyBUbyBpbml0YWxpemUgb3BlbmFsIGZvciBwbGF5YmFjawogICAgQUxDZGV2 aWNlICpkZXY7CiAgICBBTENjb250ZXh0ICpjdHg7CiAgICBkZXYgPSBhbGNPcGVuRGV2aWNlKE5V TEwpOwogICAgaWYoIWRldikKICAgIHsKICAgICAgICBmcHJpbnRmKHN0ZGVyciwgIk9vcHNcbiIp OwogICAgICAgIHJldHVybiAxOwogICAgfQogICAgY3R4ID0gYWxjQ3JlYXRlQ29udGV4dChkZXYs IE5VTEwpOwogICAgYWxjTWFrZUNvbnRleHRDdXJyZW50KGN0eCk7CiAgICBpZighY3R4KQogICAg ewogICAgICAgIGZwcmludGYoc3RkZXJyLCAiT29wczJcbiIpOwogICAgICAgIHJldHVybiAxOwog ICAgfQoKICAgIGludCBpbml0QnVmZmVyID0gMTsKICAgIGludCBidWZmZXJOdW1iZXIgPSAwOwoj ZGVmaW5lIE5VTV9CVUZGRVJTIDMKI2RlZmluZSBCVUZGRVJfU0laRSA0MDk2CiAgICBBTHVpbnQg c291cmNlLCBidWZmZXJzW05VTV9CVUZGRVJTXTsKICAgIEFMdWludCBmcmVxdWVuY3kgPSA0NDEw MDsKICAgIEFMZW51bSBmb3JtYXQgPSBBTF9GT1JNQVRfU1RFUkVPMTY7CgogICAgYWxHZW5CdWZm ZXJzKE5VTV9CVUZGRVJTLCBidWZmZXJzKTsKICAgIGFsR2VuU291cmNlcygxLCAmc291cmNlKTsK ICAgIGlmKGFsR2V0RXJyb3IoKSAhPSBBTF9OT19FUlJPUikKICAgIHsKICAgICAgICBmcHJpbnRm KHN0ZGVyciwgIkVycm9yIGdlbmVyYXRpbmcgOihcbiIpOwogICAgICAgIHJldHVybiAxOwogICAg fQoKICAgIC8vIGxpYmF2CiAgICBBVlBhY2tldCBwYWNrZXQ7CiAgICBBVkZyYW1lKiBkZWNvZGVk RnJhbWU7CiAgICBkZWNvZGVkRnJhbWU9YXZjb2RlY19hbGxvY19mcmFtZSgpOwogICAgaW50IGxl bjsKCiAgICB3aGlsZSAoMSkgewoKICAgICAgICAvLyBSZWFkIG9uZSBwYWNrZXQgaW50byBgcGFj a2V0YAogICAgICAgIGlmIChhdl9yZWFkX2ZyYW1lKGNvbnRhaW5lciwgJnBhY2tldCkgPCAwKSB7 CiAgICAgICAgICAgIGJyZWFrOyAgLy8gRW5kIG9mIHN0cmVhbS4gRG9uZSBkZWNvZGluZy4KICAg ICAgICB9CgogICAgICAgIGlmKHBhY2tldC5zdHJlYW1faW5kZXghPXN0cmVhbV9pZCkgewoJCQkv LyBTa2lwIFZpZGVvIGFuZCBzdWJ0aXRsZSBwYWNrZXRzCiAgICAgICAgICAgIHByaW50ZigiViIp OwogICAgICAgICAgICBjb250aW51ZTsKICAgICAgICB9CiAgICAgICAgcHJpbnRmKCJBIik7CiAg ICAgICAgLy8gRGVjb2RlcyBhdWRpbyBkYXRhIGZyb20gYHBhY2tldGAgaW50byB0aGUgZnJhbWUK CiAgICAgICAgd2hpbGUoIHBhY2tldC5zaXplID4wKSB7CiAgICAgICAgICAgIGludCBmaW5pc2hl ZEZyYW1lPTA7CiAgICAgICAgICAgIGxlbiA9IGF2Y29kZWNfZGVjb2RlX2F1ZGlvNChjb2RlY19j b250ZXh0LCBkZWNvZGVkRnJhbWUsICZmaW5pc2hlZEZyYW1lLCAmcGFja2V0KTsKCiAgICAgICAg ICAgIGlmKGxlbjwwKSB7CiAgICAgICAgICAgICAgICBwcmludGYoImVycm9yIik7CiAgICAgICAg ICAgICAgICBicmVhazsgIC8vIEVycm9yIGluIGRlY29kaW5nCiAgICAgICAgICAgIH0KCiAgICAg ICAgICAgIGlmIChmaW5pc2hlZEZyYW1lKSB7CgkJCSAgIC8vIERlY29kZWQgZGF0YSBpcyBub3cg YXZhaWxhYmxlIGluCgkJCSAgIC8vIGRlY29kZWRGcmFtZS0+ZGF0YVswXQoJCQkgICAKICAgICAg ICAgICAgICAgaW50IGRhdGFfc2l6ZSA9IGF2X3NhbXBsZXNfZ2V0X2J1ZmZlcl9zaXplKE5VTEws IGNvZGVjX2NvbnRleHQtPmNoYW5uZWxzLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgZGVjb2RlZEZyYW1lLT5uYl9zYW1wbGVzLAogICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29kZWNfY29u dGV4dC0+c2FtcGxlX2ZtdCwgMSk7CgogICAgICAgICAgICAgICAvLyBPcGVuQUwgY29uc3VtZXMg YnVmZmVycyBpbiB0aGUgYmFja2dyb3VuZAogICAgICAgICAgICAgICAvLyB3ZSBmaXJzdCBuZWVk IHRvIGluaXRpYWxpemUgdGhlIE9wZW5BTCBidWZmZXJzIHRoZW4KICAgICAgICAgICAgICAgLy8g c3RhcnQgY29udGlub3VzIHBsYXliYWNrLgogICAgICAgICAgICAgICBpZihpbml0QnVmZmVyKSB7 CiAgICAgICAgICAgICAgICAgICBhbEJ1ZmZlckRhdGEoYnVmZmVyc1tidWZmZXJOdW1iZXJdLCBm b3JtYXQsIGRlY29kZWRGcmFtZS0+ZGF0YVswXSwgZGF0YV9zaXplLCBmcmVxdWVuY3kpOwogICAg ICAgICAgICAgICAgICAgaWYoYWxHZXRFcnJvcigpICE9IEFMX05PX0VSUk9SKSB7CiAgICAgICAg ICAgICAgICAgICAgICAgZnByaW50ZihzdGRlcnIsICJFcnJvciBsb2FkaW5nIDooXG4iKTsKICAg ICAgICAgICAgICAgICAgICAgICByZXR1cm4gMTsKICAgICAgICAgICAgICAgICAgIH0KCiAgICAg ICAgICAgICAgICAgICBpZihidWZmZXJOdW1iZXI9PTIpewogICAgICAgICAgICAgICAgICAgICAg IC8vIGFsbCBidWZmZXJzIHF1ZXVlZAogICAgICAgICAgICAgICAgICAgICAgIGFsU291cmNlUXVl dWVCdWZmZXJzKHNvdXJjZSwgTlVNX0JVRkZFUlMsIGJ1ZmZlcnMpOwogICAgICAgICAgICAgICAg ICAgICAgIC8vIHN0YXJ0IHBsYXliYWNrCiAgICAgICAgICAgICAgICAgICAgICAgYWxTb3VyY2VQ bGF5KHNvdXJjZSk7CiAgICAgICAgICAgICAgICAgICAgICAgaWYoYWxHZXRFcnJvcigpICE9IEFM X05PX0VSUk9SKSB7CiAgICAgICAgICAgICAgICAgICAgICAgICAgIGZwcmludGYoc3RkZXJyLCAi RXJyb3Igc3RhcnRpbmcgOihcbiIpOwogICAgICAgICAgICAgICAgICAgICAgICAgICByZXR1cm4g MTsKICAgICAgICAgICAgICAgICAgICAgICB9CiAgICAgICAgICAgICAgICAgICAgICAgaW5pdEJ1 ZmZlcj0wOwogICAgICAgICAgICAgICAgICAgfQoKICAgICAgICAgICAgICAgICAgIC8vIHVwZGF0 ZSBidWZmZXIgbnVtYmVyIHRvIGZpbGwKICAgICAgICAgICAgICAgICAgIGJ1ZmZlck51bWJlcj0o YnVmZmVyTnVtYmVyKzEpJTM7CiAgICAgICAgICAgICAgIH0gZWxzZSB7CgkJCQkgICAvLyBPcGVu QUwgaXMgcGxheWluZyBpbiB0aGUgYmFja2dyb3VuZC4KCQkJCSAgIC8vIG9uZSBuZXcgZnJhbWUg d2l0aCBhdWRpbyBkYXRhIGlzIHJlYWR5CgkJCQkgICAKICAgICAgICAgICAgICAgICAgIC8vIGZp cnN0IHdhaXQgZm9yIG9wZW5hbCB0byByZWxlYXNlIG9uZSBidWZmZXIKICAgICAgICAgICAgICAg ICAgIEFMdWludCBidWZmZXI7CiAgICAgICAgICAgICAgICAgICBBTGludCB2YWw7CiAgICAgICAg ICAgICAgICAgICBkbyB7CiAgICAgICAgICAgICAgICAgICAgICAgYWxHZXRTb3VyY2VpKHNvdXJj ZSwgQUxfQlVGRkVSU19QUk9DRVNTRUQsICZ2YWwpOwogICAgICAgICAgICAgICAgICAgfSB3aGls ZSAodmFsIDw9MCk7CiAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAvLyBmaWxs IGFuZCByZXF1ZXVlIHRoZSBlbXB0eSBidWZmZXIKICAgICAgICAgICAgICAgICAgIGFsU291cmNl VW5xdWV1ZUJ1ZmZlcnMoc291cmNlLCAxLCAmYnVmZmVyKTsKICAgICAgICAgICAgICAgICAgIGFs QnVmZmVyRGF0YShidWZmZXIsIGZvcm1hdCwgZGVjb2RlZEZyYW1lLT5kYXRhWzBdLCBkYXRhX3Np emUsIGZyZXF1ZW5jeSk7CiAgICAgICAgICAgICAgICAgICBhbFNvdXJjZVF1ZXVlQnVmZmVycyhz b3VyY2UsIDEsICZidWZmZXIpOwogICAgICAgICAgICAgICAgICAgaWYoYWxHZXRFcnJvcigpICE9 IEFMX05PX0VSUk9SKSB7CiAgICAgICAgICAgICAgICAgICAgICAgZnByaW50ZihzdGRlcnIsICJF cnJvciBidWZmZXJpbmcgOihcbiIpOwogICAgICAgICAgICAgICAgICAgICAgIHJldHVybiAxOwog ICAgICAgICAgICAgICAgICAgfQogICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAg Ly9SZXN0YXJ0IG9wZW5hbCBwbGF5YmFjayBpZiBuZWVkZWQKICAgICAgICAgICAgICAgICAgIGFs R2V0U291cmNlaShzb3VyY2UsIEFMX1NPVVJDRV9TVEFURSwgJnZhbCk7CiAgICAgICAgICAgICAg ICAgICBpZih2YWwgIT0gQUxfUExBWUlORykKICAgICAgICAgICAgICAgICAgICAgICBhbFNvdXJj ZVBsYXkoc291cmNlKTsKICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgfQoKICAgICAgICAg ICAgLy9UaGVyZSBtYXkgYmUgbW9yZSB0aGFuIG9uZSBmcmFtZSBvZiBhdWRpbyBkYXRhCiAgICAg ICAgICAgIC8vaW5zaWRlIHRoZSBwYWNrZXQuCiAgICAgICAgICAgIHBhY2tldC5zaXplLT1sZW47 CiAgICAgICAgICAgIHBhY2tldC5kYXRhKz1sZW47CiAgICAgICAgfSAvLyB3aGlsZSBwYWNrZXQu c2l6ZSA+IDAKICAgIH0gLy8gd2hpbGUgbW9yZSBwYWNrZXRzIGV4aXN0IGluc2lkZSBjb250YWlu ZXIuCgogICAgYXZfY2xvc2VfaW5wdXRfZmlsZShjb250YWluZXIpOwoKICAgIC8vIFdhaXQgZm9y IG9wZW5hbCB0byBmaW5pc2ggcGxheWJhY2sgb2YgYW55IHJlbWFpbmluZyBidWZmZXJzLgogICAg QUxpbnQgdmFsOwogICAgZG8gewogICAgICAgYWxHZXRTb3VyY2VpKHNvdXJjZSwgQUxfU09VUkNF X1NUQVRFLCAmdmFsKTsKICAgIH0gd2hpbGUodmFsID09IEFMX1BMQVlJTkcpOwoKICAgIC8vIEZy ZWUgT3BlbkFMIHJlc291cmNlcwogICAgYWxEZWxldGVTb3VyY2VzKDEsICZzb3VyY2UpOwogICAg YWxEZWxldGVCdWZmZXJzKE5VTV9CVUZGRVJTLCBidWZmZXJzKTsKCiAgICBhbGNNYWtlQ29udGV4 dEN1cnJlbnQoTlVMTCk7CiAgICBhbGNEZXN0cm95Q29udGV4dChjdHgpOwogICAgYWxjQ2xvc2VE ZXZpY2UoZGV2KTsKCiAgICBmcHJpbnRmKHN0ZG91dCwgIkRvbmUgcGxheWluZy4gRXhpdGluZy4u LiIpOwoKICAgIHJldHVybiAwOwp9Cg==