Summary: | Add generic support for stereoscopic renderer, demo w/ OculusVR | ||
---|---|---|---|
Product: | [JogAmp] Jogl | Reporter: | Sven Gothel <sgothel> |
Component: | util | Assignee: | Sven Gothel <sgothel> |
Status: | RESOLVED FIXED | ||
Severity: | enhancement | CC: | rami.santina |
Priority: | --- | ||
Version: | 2 | ||
Hardware: | All | ||
OS: | all | ||
Type: | --- | SCM Refs: |
40d7d0d18ed11b8dbb0f16137f82eef828b822a4
8a4dacf38c9d78bc0aafae91cbd256148835ee9b
589070da2be7e097b7415503011387ad1f6e14d5
b54e87d2cebf6aa0def18f172cbfa111c4efc837
d7d8d11abad1e4e17532068333dfe9dddf2942a2
ae17eca383cce89a000062c3a86e415a6753066b
36327e24cf586b50bf18e87d7d13d53eb41cf1d9
d44e8ada30d62149c5d4d4b8fdba7cc33f8c765b
62bc3219736ea003e69d8a63dc4ce29841d4129e
f8f0f051604721bceaee214b8e5218fd47d2eb9e
jogl 38e51e4a5f6f35c658df10f6d48a33e3ffaea2f3
jogl 8be4b87a9740229f09b1dc2b7265e3e73c4397dd
jogl 609b3f6d28bb13b589cde815acdb0e72e58ebe44
oculusvr-sdk a18c0cec906c4ac8c171d9000fd1c54e9302c9a4
|
Workaround: | --- | ||
Bug Depends on: | 1022, 1025 | ||
Bug Blocks: | 1116 |
Description
Sven Gothel
2014-06-14 08:41:00 CEST
To seamlessly use libovr via GlueGen/Java, we require compound [array] call-by-value support. See Bug 1022. with the last required GlueGen, i.e. Bug 1011 comment 5, i.e. supporting compound-arrays in structs and allowing enum-c identifiers to define array-length, bring-up on GNU/Linux x86_64 is successful. Bring-up test only initializes the ovrHmdHandle data structure by OVR native code. Current evaluation build compiles all OVR-SDK source files itself w/o using provided libovr.a. We also skip the GL dependent renderer of the SDK, i.e. we prefer to utilize our JOGL 'barrel distortion' renderer. This eases the 'chicken-egg' problem of OVR SDK dependencies, i.e. libGL, libGLESv2 .. etc Since the OVR source code is C++, we may still have to figure out how to / and whether we shall link libstdc++ statically to remove platform dependencies. Next bring-up test shall include a working visual demo. (In reply to comment #2) > with the last required GlueGen, i.e. Bug 1011 comment 5, Ooops - Bug 1022 comment 5, http://jogamp.org/git/?p=jogl.git;a=shortlog;h=refs/heads/oculusvr Branch: oculusvr baf8aba50ea5a73467298bb7d20f9028b8223434 Add OculusSDK 0.3.2 Source Code w/o Samples, docs or binaries (libs or tools) 44686e9bdc4ce7b603935ef5d908353dac9c956e As described in comment 2 40d7d0d18ed11b8dbb0f16137f82eef828b822a4 ... Re-created oculusvr branch: Please delete previous one! <http://jogamp.org/git/?p=jogl.git;a=shortlog;h=refs/heads/oculusvr> +++ Moved OculusVR SDK to it's own sub-module: <http://jogamp.org/git/?p=oculusvr-sdk.git;a=summary> with merged sources for - Linux - Windows - OSX +++ Continue w/ bring-up for Windows .. b54e87d2cebf6aa0def18f172cbfa111c4efc837 - Bring-up on Windows w/ Mingw64 - Solve static linkage of stdc++ d7d8d11abad1e4e17532068333dfe9dddf2942a2 - Bring-up on OSX w/ clang/clang++ - Split native build: cc, c++, linkage ae17eca383cce89a000062c3a86e415a6753066b - Add note about 'Oculus VR Rift SDK Software License' in LICENSE.txt - Merged branch oculusvr -> master, .. - Removed branch oculusvr 36327e24cf586b50bf18e87d7d13d53eb41cf1d9 Add OculusVR distortion renderer (single FBO and dual FBO) Screenshots: <http://jogamp.org/files/screenshots/oculusvr/ovr-snap03.png> <http://jogamp.org/files/screenshots/oculusvr/> Adoption (aka usage) is quite simple, see: <http://jogamp.org/git/?p=jogl.git;a=blob;f=src/test/com/jogamp/opengl/test/junit/jogl/stereo/ovr/OVRDemo01.java;h=c20af1389c2351e566f0b1823a97072e6016f630;hb=36327e24cf586b50bf18e87d7d13d53eb41cf1d9> It uses the new support classes to render stereo manually w/ all the distortions. The sensors shall work implicit via the 'eye pose' and the derived modelview matrix. However, we have to see how that works w/ 'rotating the head', no DK2 available here. In case more stereo renderer are required, we shall add subsequent bug reports. Closing for now. Commit d44e8ada30d62149c5d4d4b8fdba7cc33f8c765b: Refine API in regards to proper package names, interface and high-level access to eye specific constant parameter and variable eye movement. +++ Commit 36327e24cf586b50bf18e87d7d13d53eb41cf1d9 introduced 'GLEventListener2' Move javax.media.opengl.GLEventListener2 -> com.jogamp.opengl.util.CustomRendererListener -> com.jogamp.opengl.util.stereo.StereoRendererListener StereoRendererListener adds stereoscopic specific: public void reshapeEye(final GLAutoDrawable drawable, final int x, final int y, final int width, final int height, final EyeParameter eyeParam, final EyePose eyePose); .. see below. ++ Add com.jogamp.opengl.util.stereo: - EyeParameter (Constant eye parameters, like IPD and FOV) - EyePose (Current eye position and orientation) +++ Add com.jogamp.opengl.math.FovHVHalves to support non-centered bi-directional FOV for lenses. Add respective FloatUtil.makePerspective(.. FovHVHalves fovhv ) variant. Commit 62bc3219736ea003e69d8a63dc4ce29841d4129e Turns out that using bilinear texture filtering w/o multisampling is 'good enough' .. - OVRSBSRenderer*: - Add texture filtering parameter (bilinear/none) - Adjust FBO texture params (alpha, filtering) No alpha needed in FBO sink/target! - Demo: Pass all parameters w/ arguments, use sane good defaults. Make OVR access vendor agnostic: Package 'com.jogamp.opengl.util.stereo' contains all public interfaces/classes Renamed interfaces: CustomRendererListener -> CustomGLEventListener StereoRendererListener -> StereoGLEventListener New vendor agnostic 'stuff' in com.jogamp.opengl.util.stereo: 1 - StereoDeviceFactory To create a vendor specific StereoDeviceFactory instance, which creates the StereoDevice. 2 - StereoDevice For vendor specific implementation. Can create StereoDeviceRenderer. 3 - StereoDeviceRenderer For vendor specific implementation. 4 - StereoClientRenderer Vendor agnostic client StereoGLEventListener renderer, using a StereoDeviceRenderer. Now supports multiple StereoGLEventListener, via add/remove. - MovieSBSStereo demo-able via StereoDemo01 can show SBS 3D movies. I did another round of abstraction adding generic impl. of StereoDevice and StereoDeviceRenderer and isolating the lense distortion math from the OVR SDK[1]. The generic impl. may utilize: - monoscopic display (*) - stereoscopic display (*) - Side-By-Side (SBS) (*) - without lenses (*) - with lenses (*) - Other techniques - OpenGL Quad Stereo Buffer (+) - Top-Bottom (+) - Interlaced (+) - Color Separation (+) Goal is to harmonize stereo rendering w/ this package. Note, the isolating lense distortion math from the OVR SDK[1] is bound to the SDK's license since I have copied parts of it. Hence we may only be able to distribute this part within the oculusvr-sdk sub-module due to it's license which sort of forbids deploying only parts IMHO. (In reply to comment #12) > The generic impl. may utilize: > - monoscopic display (*) > - stereoscopic display (*) > - Side-By-Side (SBS) (*) > - without lenses (*) > - with lenses (*) Done. commit jogl 38e51e4a5f6f35c658df10f6d48a33e3ffaea2f3 Bug 1021: Add GenericStereoDevice* Supporting custom configurations; Hook-in oculusvr-sdk java distortion-mesh calculation if available StereoDeviceFactory support new GenericStereoDeviceFactory, with it's GenericStereoDevice and GenericStereoDeviceRenderer. GenericStereoDevice maintains different configurations, triggered either by passing a GenericStereoDevice.Config instance directly or by the device-index parameter: - 0: monoscopi device: No post-processing - 1: stereoscopic device SBS: No post-processing - 2: stereoscopic device SBS + Lenses: Distortion post-processing (only available w/ oculusvr-sdk sub-module) Producing a 'GenericStereoDevice.Config' instance is self containing and may extend if supporting more device types like top-bottom, interlaced etc. StereoDemo01 handles all use-cases and may be used as a test-bed to add and experiment with stereoscopy, devices and settings. XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX commit oculusvr-sdk a18c0cec906c4ac8c171d9000fd1c54e9302c9a4 Add java code of DistortionMeshProducer for lens usage, as included in Oculus VR RIFT SDK 0.3.2 (w/ dependency on JOGL) This code has been added to the oculusvr-sdk sub-module due to the unclear license situation. While including these parts w/ the whole SDK, we do not violate the RIFT SDK licence - to our knowledge. +++ Code implements JOGL's 'jogamp.opengl.util.stereo.DistortionMesh.Producer' interface. The implementation heavily uses RIFT SDK mathematical equations, comments, algorithms and constants (DK1). RIFT SDK C++ code has been partially ported to Java. Purpose it to add a development testbed for JOGL without requiring to bind to the native SDL library and to experiment w/ some parameters. This module may be used for educational purposes with JOGL's 'jogamp.opengl.util.stereo.GenericStereoDevice' If 'compiled in' with the SDK, reflection it utilized in one of GenericStereoDevice configuration to load this functionality. This module is not useful for real VR integration, since it missies proper parameter detection as well as sensor support. For real VR integration, the fully fledged 'jogamp.opengl.oculusvr.OVRStereoDevice' shall be used instead. The latter is part of JOGL and binds dynamically into the RIFT SDK native library, which is provided within this sub-module (oculusvr-sdk). 8be4b87a9740229f09b1dc2b7265e3e73c4397dd Fix OculusVR detection, need to probe HMD count, otherwise native SDK code crashes w/ NPE 609b3f6d28bb13b589cde815acdb0e72e58ebe44: - StereoDevice.DeviceType: Add API doc - StereoDevice: Add getFactory() - GenericStereoDevice - Use common static vars for configurations for simplicity - Fix createRenderer(..)'s eyeViewport in case no post-processing is performed, i.e. needs viewport X offset. - StereoDemo01 - Use 'movie' eyePosition instead of default if: - using a movie player _and_ using lenses! - Fix NEWT window pixel-unit size after window creation! |