Because the ARM JVM still cant differensiate ABI in its system properties #JogAmp #Gluegen now implement an ELF header parserxerxes // Feb 8, 2013 10:55:28 PM
This post is about an solution to an old chest nut that still plague ARM Java users on both servers, desktop and mobile. The problem it the impossibility to query the JVM system properties to find out the ABI armel/armhf in use, on ARM systems and possible other ARCH ABI variants as well such as x32, because no information about the ABI is exposed using said system properties. http://download.java.net/jdk8/docs/api/java/lang/System.html#getProperties%28%29 It is mandatory for developers to know the ABI in use at runtime in order to load matching JNI libraries.
About one year ago in 2012 the question got raised inside the #OpenJDK IRC channel http://icedtea.classpath.org/wiki/New_os.arch_namespace_Architecture but nothing really happened.
Later in 2012 the same question got raised inside the Raspberry Pi community during the move to the new Rasbian armhf system, http://www.raspberrypi.org/phpBB3/viewtopic.php?p=201105 but nothing really happened except adding more or less creative workarounds to all ARM Java applications and libraries.
The Raspberry Pi community eventually added a Java specific forum and now Oracle engaged in the community and asked for questions: So the question got raised again.
In the end of 2012 Oracle launched their first armhf JVM for the Raspberry Pi but the os.arch namespace was still not fixed, after reporting a workaround for the new JDK 8 EA http://www.raspberrypi.org/phpBB3/viewtopic.php?p=238135#p238135 Oracle took notice.
In the beginning of Jan 2013 Oracle Bob Vandette started an discussion on how to introduce an extended properties namespace that would cover the ABI differences to let applications discover the ABI at runtime.
During the FOSDEM 2013 JogAmp re-raised the question in front of the Oracle audience at 19m23s into the JogAmp talk and the response was "We just discussed that last week". So our reply is "So we can not wait for that". This issue is still causing hairpulling for ARM Java server/desktop/mobile deployments where for example the JVM tries to load armel JNI on the new armhf ARM Linux systems because the system properties set by the new armhf JVM is identical to an armel JVM, loading a library of the wrong ABI do not work.
Q: So what can we do about it?
A: Include an #freejava ELF header parser of course!
#JogAmp #gluegen now implements and use the following ELF header parser: http://jogamp.org/git/?p=gluegen.git;a=commit;h=371e1dbff6f5f255ab27ed0ab32368abb06eed82 … in order to find out the ABI at runtime without relying on if proper #JVM os.arch & ABI properties are set or not.
Kudos to Sven Gothel for implementing the new ELF header parser using the Gluegen StructAccessor.
Kudos to Andrew Haley, aph, who hatched the idea inside the #OpenJDK IRC channel to scan /proc/self/exe using an elf header parser from within the Java process itself and look for the Tag_ABI_VFP_args ELF header flag, this is what the implemented gluegen solution is based on.
I have just returned from FOSDEM where we Julen Gouesse, Sven Gothel and Xerxes Rånby presented a JogAmp freejava love talk with some live demonstrations running hardware accelerated on x86 laptop, android/meego phone/tables and GNU/LInux systems such as the AC100 and RaspberryPi.
Slides, Video and Teaser from the JogAmp FOSDEM freejava love talk are now online:
If you want to design a game then we recommend you to use JogAmp indirect through a game engine library such as libgdx or JMonkeyEngine 3. We have a forum thread inside the JogAmp community where we work on improving engine support for embedded devices such as the Raspberry Pi using said engines. By using a game engine will get you up to speed developing professional games that can be run across all devices and formfactors.
The video and teaser recordings also includes footage of the JMonkeyEngine 3 JOGL 2 backend initialized by Julien Gouesse that we for time reason never managed to show during the strict 40min talk and live demo at FOSDEM.
Inside the FOSDEM talk we ran the open-source libgdx game pax-britannica using the new libgdx JogAmp JOGL 2 port initialized by Julien Gouesse in combination with the new hardfloat fixed CACAO ARM JVM found in the new IcedTea 6 1.12 release on the Raspberry Pi.
we also ran the JogAmp Jake2 port, a port done by Sven Gothel, using the armhf JamVM from IcedTea 7 on the ac100.
Both opensource games of course rocked running using freejava!
The point that we wanted to show is that if you start to use the dedicated love using the media accelerator found in all new devices your java applications rendering will run *equally* fast regardless of the JVM implementation, since the rendering is then performed by the GPU instead of the CPU.
For demonstation purposes: I had to extend the libgdx backend with a custom mouse pointer in order for otherwise touchscreen oriented games such as pax-britannica to work on the Raspberry Pi from console, the reason why this is needed is because there is no highlevel compositing windowmanager running on the RaspPi adding the overlay mousepointer for you like what you are custom to see when running desktop applications using X11. This libgdx RaspberryPi mouse pointer branch allowed me to test all touch oriented libgdx games and demos from console using a mouse input device.
While we know that compilation of custom JVM can be tricky I have prepared a RaspberryPi armhf CACAO libjvm.so that you can use as an drop in replacement into any OpenJDK 6 installation (/usr/lib/jvm/java-6-openjdk-armhf/jre/lib/arm/server/libjvm.so) This libjvm.so was built using the IcedTea 6 1.12 release from inside a Raspbian chroot.
For JamVM simply install the openjdk-7-jdk package and run it using java -jamvm its already built and packaged by the Raspbian team and work great! The new CACAO armhf libjvm.so is found here: http://labb.zafena.se/cacao/armhf-armv6/libjvm.so
The Raspberry Pi Rasbian armhf distribution have now packaged IcedTea 6 1.12.1 and included it in the distribution, this means that you can test the new armhf CACAO JVM and JamVM JVM by simply installing the openjdk-6-jdk package.
sudo apt-get update
sudo apt-get install openjdk-6-jdk
java -jamvm -version
java -cacao -version
Also a great KUDOS for Qun, our dear camera-woman!
Cheers and enjoy the love!
On behalf of the JogAmp community - Xerxes
2008: Sun Microsystems invested heavily into using hardware acceleration on mobile devices using JOGL. This was the foundation to get JavaFX 1.3 running across devices. Video of James Gosling, Ken Russell and Sven Gothel on stage at JavaOne 2008 keynote http://www.youtube.com/watch?v=DeupVAMnvFA.
Ken Russel later wrote his last Sun blog-post that covers the technology demonstrated in this first OpenGL ES 2 JOGL demonstration: https://blogs.oracle.com/kbr/entry/java_on_the_nvidia_apx.
Later during the same year Sven Gothel demonstrated hardware accelerated OpenMAX video decoding on the same Tegra 1 mobile device using the new re-architectured JOGL 2 with stellar mobile support: http://www.youtube.com/watch?v=D6Lkw3eZK1w
2009: The core members of the JOGL and Java3D team left Sun before Oracle took over and forked JOGL.
2010: The JogAmp community was created.
2011: Oracle Gives up on Java3D (and JOGL) for RIA (Webstart and Applets)
The Oracle decision to remove all signed Java3D and JOGL builds from their servers, and hence break all existing online Java3D/JOGL applications using the SUN/Oracle builds, this have only been mentioned by Oracle inside the java.net support forum.
Fortunately JOGL and Java3D was originally released under a BSD license so it was possible for the community to keep-on maintaining the bindings and provide JogAmp signed builds.
2012: Oracle work on flushing out its own use of JOGL in JavaFX by removing jogl-prism. Oracle no longer give programmers direct access to OpenGL using their APIs.
Oracle announced during JavaOne 2012 that Oracle would like to get help from the community to port JavaFX to new platforms under the OpenJFX project. Surely the community can help by re-implement jogl-prism and adding raw JOGL support into OpenJFX to ease future OpenJFX porting efforts.
JOGL v2 is actively maintained by the JogAmp community.
JOGL v2 now runs on the latest GPU cores and work on top of any JVM. JOGL v2 contains its own platform independent NEWT windowing toolkit. NEWT applications targeting the GL2ES2 or GL2ES1 profile, both using a common subset of OpenGL and ES calls, can be deployed across different platforms and devices from desktop to mobile without code change.
http://jogamp.org/doc - Documentation, slides, and videorecording from the latest live JogAmp demonstrations
Java3D is now also maintained by some members of the JogAmp community.
Dear Team & Users,
it’s crunchtime ..
JogAmp will have it’s annual BOF @ SIGGRAPH 2012.
If you are around, we hope you have a chance to to attend
and meet w/ us.
Afterwards we may like to dine & wine together, if possible.
JogAmp: 2D/3D & Multimedia Across Devices Tuesday, 7 August 3:30 pm - 5:00 pm JogAmp provides OpenGL and OpenCL across devices using Java. Showcasing font, UI and video, high-level API utilization (Ardor3D, Java3D, etc.), and applications on Android, Linux, Window, OSX, and Solaris. JogAmp Community
If you have contributions for our BOF,
please don’t be shy and come forward.
We would like you to present it.
If you cannot attend personally, you can have us showing your content.
You will be referenced of course.
Please reply to in forum.
Today the v2.0-rc9 release has been notified w/ details.
Mostly contains stabilization and bug fixes no new features.
Android ARM-v7a APKs are included this time,
next market update is targeted w/ the next release.
… comment on the forum.
Besides, some wiki pages got updated.
Last weeks the new video streaming feature GLMediaPlayer
was added for Android and machines w/ Libav/FFmpeg pre-installed.
We produced a presentation video showcasing JogAmp’s objectives:
RC6 is now underway, I still have to add some supplied patches and walk through the buglist though.
However, this time I have to complete this task until tomorrow regardless whether I could complete the bug walk or not.
IFC is one of the oldest standards for data exchange within the AEC industry. It became one of the most supported exchange formats available. Using a standard which was started about ten years ago, brings with it obsolete ways for defining objects and exchange files. Example to that is usage of EXPRESS language to define the schema and use of ASCII as the format of the file.
While it is great to have a format that is being adopted by a growing list of publishers, issues with the schema and its implementations makes it hard to adopt and rely on. Below are some of the issues that I guess are mostly visible, and a possible solution to consider!
Highly Typed: The IFC schema (ISO 10303-21) has an object definition for each type of object defined in the AEC industry. This results in a huge schema; instead of having one type of data object. The attributes of the objects are based on the object type, thus producing a complex exporter and reader of the file. As a developer, the highly specific format of the objects makes the specs your best friend while writing a parser, reader or an exporter to this format.
Mapping vs Correctness: This issue is mainly a problem with the usage of the format. The generosity of options in describing a single object or part of objects, is that a publisher usually chooses one path of representation which is usually the simplest to map too or the first option they see (since it’s a huge schema). This has the risk of losing data quality and/or having bad representation of the data. As an example, a circle can be defined in lots of ways: like IfcCircle, IfcArc between 0 degrees and 360 degrees, IfcTrimmedCurve of an IfcArc (between 360 and 0) trimmed at 0 and 360. The last one might seem odd, but I have seen it a lot. The three options give a correct result but it’s an over kill, and in my view wrong. Another example to fast routes taken is exporting all the geometry as triangle meshes losing all the quality…
Readability: There exists an XML version of the format, Part-28, but it’s not much used. ifc file (part 21) format is an ASCII file, where each object is a line by itself. Parent-Child relationship and associations are defined by #object_number that acts as a link between objects, which makes tracking an object in a text editor practically impossible.
Predefined Data fields: each object type found in the file has a predefined set of properties that should be available. The problem with this restriction is that when a publisher doesn’t have the required fields they include bogus data just to conform to the schema of the object, which sometimes makes the whole dataset bogus. This is also found in the required predefined hierarchy of the object definitions. Example: most design systems set the building name to be the filename or some string, when it’s not available!
A call for Conformance Level: Changing the schema is a very optimistic approach, since the time frame required for the industry to support it and implement it is long. What would be nice to have is a test suite where publishers are asks to run to check the quality of the data sent. Then publishers will consider updating their implementations to get a higher ranking on the conformance. In my view, with such test suites we will see less publishers producing files with geometric cache-triangulation (a very common observation) instead of the actual CAD data.
Besides adding proper Mac OS X support (10.5.8 – 10.7.*, incl. OpenJDK7),
OpenGL 4.2 and latest EGL, ES1 and ES2 extension updates and lot’s of stabilization’s,
Xerxes Rånby and myself worked on a proper Linux ARMv7 support.
Both were able to test on Omap4 (Pandaboard ES), Tegra2 (AC100), where Xerxes also tested on other machines, eg. Nokia N9 MeeGo.
Even though GlueGen and JOGL in general support EGL and ES1/ES2 since 2008 incl. the GL profile selection,
we figured we need better support for multiple GL implementations on one platform, Mesa3D software and the hardware EGL/ES ones.
Without tweaking your default configuration, JOGL chooses the right implementation for the desired profile,
e.g. hardware accelerate GLES2 for the desired common GL2ES2 profile on your mobile device, even though Mesa is installed.
Besides tiny big fixes and workarounds the biggest amount of work was to attach the Linux ARMv7 job to Jenkins.
We use a cross-compile and cross-test environment, where the build host cross-compiles and makes the Pandaboard ES
fetch the artifacts via rsync and execute the tests. Later the build host pulls the results and forwards them to the Jenkins master.
All JogAmp modules are now build for Linux-Armv7,
a test release is made available here.
You may like to try the test Applets.
Note: This week RC6 will be released under the usual location and the above test URL will cease to exist.
I dared to test browser support via OpenJDK and the IcedTea Plugin,
and for some reason .. it just works
Please feel welcome to join the discussion.
After tons of bug fixes and Mac OS X, Solaris and Android platform support
we finally have v2.0-rc4.
Besides many important bug fixes this release supports
Mac OS X 10.6.4 and 10.7.
The Applet browser plugin is also enhanced and validated
on all platforms for FF, Chrome, Safari and IE where supported.
We have to thank each other for our ongoing support,
bug reports, inspiration and pushing for results.
The following aliasing of URL location has been made,
ie. all are aligned to v2.0-rc4 for now.
- v2.0-rc4 -> archive/rc/v2.0-rc4
- jogamp-current -> v2.0-rc4
- jogamp-next -> v2.0-rc4
- webstart -> v2.0-rc4
- webstart-next -> v2.0-rc4
Developer downloads at:
Git repositories have been tagged w/ v2.0-rc4.
Planned for RC5 so far:
- Maven2 integration (I know .. a bit late it is already)
- Mobile/Android autobuilds / release
- More bugfixes
- Update the graph package
Please join the discussion in our forum.
You may like to comment on this release here.
To conclude my today series of blog entries, I thought it might be a good idea to show our automatic test statistics.
Here are the latest good and failed test charts for all platforms:
And finally the progress on OS X with all tests included:
Besides adding running all unit tests to OSX, we also made sure that it’s performance is now equal to the other platforms, ie. around 4 minutes runtime for all tests.
But be aware that these features will be promoted to the next release RC4,
so you would need to wait or use the autobuilds, see Downloading the latest automatic build.