Bug 626

Summary: Support X11/EGL-blit on Rasp.Pi allowing use of AWT and Applets (was: NApplet fail using newt bcm.vc.iv driver)
Product: [JogAmp] Jogl Reporter: Xerxes Rånby <xerxes>
Component: appletAssignee: Sven Gothel <sgothel>
Status: CONFIRMED ---    
Severity: enhancement CC: gouessej
Priority: P2    
Version: 2   
Hardware: All   
OS: all   
Type: --- SCM Refs:
Workaround: ---

Description Xerxes Rånby 2012-10-06 01:38:26 CEST
I decided to test if it was possible to run the JogAmp applet demos from the midori browser on the RaspberryPi.

Good news: the broadcom vc iv autodetection worked and the GPU is initialized.
Bad news: something else broke, log below:

Tested using Rasbian wheezy armhf.

pi@raspberrypi ~ $ ldd /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
	not a dynamic executable
pi@raspberrypi ~ $ ls -la /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
-rw-r--r-- 1 pi pi 4992 Oct  5 23:26 /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
pi@raspberrypi ~ $ file /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
/tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so: ELF 32-bit LSB shared object, ARM, version 1 (SYSV), dynamically linked, BuildID[sha1]=0xb624f0122ee6952661f57ea736481c8653f1ed6e, stripped


pi@raspberrypi ~ $ midori http://jogamp.org/deployment/archive/master/gluegen_598-joal_364-jogl_824-jocl_686-signed/jogl-applet-runner-newt-gears-normal-napplet2.html
The configuration couldn't be loaded: Not a directory

** (midori:2468): WARNING **: Can't open /home/pi/.config/midori/cookies.db

** (midori:2468): WARNING **: Failed to execute query: (null)

** (midori:2468): WARNING **: Failed to execute query: (null)

** (midori:2468): CRITICAL **: midori_browser_new_history_item: assertion `db != NULL' failed

** (midori:2468): CRITICAL **: midori_browser_update_history_title: assertion `db != NULL' failed
java version "1.7.0_07"
OpenJDK Runtime Environment (IcedTea7 2.3.2) (7u7-2.3.2a-1+rpi1)
JamVM (build 1.6.0-devel, inline-threaded interpreter with stack-caching)

** (midori:2468): CRITICAL **: midori_browser_new_history_item: assertion `db != NULL' failed

** (midori:2468): CRITICAL **: midori_browser_update_history_title: assertion `db != NULL' failed

** (midori:2468): CRITICAL **: midori_browser_new_history_item: assertion `db != NULL' failed

** (midori:2468): CRITICAL **: midori_browser_update_history_title: assertion `db != NULL' failed
BCM.Display initIDs ok
BCM.Screen initIDs ok
BCM.Window initIDs ok
class com.jogamp.opengl.test.junit.jogl.demos.es2.GearsES2 has no 'window' field
class com.jogamp.opengl.test.junit.jogl.demos.es2.GearsES2 has no 'glWindow' field
java.awt.IllegalComponentStateException: component must be showing on the screen to determine its location
	at java.awt.Component.getLocationOnScreen_NoTreeLock(Component.java:2044)
	at java.awt.Component.getLocationOnScreen(Component.java:2018)
	at com.jogamp.newt.awt.applet.JOGLNewtApplet1Run.start(JOGLNewtApplet1Run.java:234)
	at sun.applet.AppletPanel.run(AppletPanel.java:475)
	at java.lang.Thread.run(Thread.java:722)
JNLPAppletLauncher: static initializer
java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: Can't load library: /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
	at com.jogamp.newt.awt.applet.JOGLNewtApplet1Run.init(JOGLNewtApplet1Run.java:220)
	at org.jdesktop.applet.util.JNLPAppletLauncher.startSubApplet(JNLPAppletLauncher.java:1995)
	at org.jdesktop.applet.util.JNLPAppletLauncher.access$200(JNLPAppletLauncher.java:662)
	at org.jdesktop.applet.util.JNLPAppletLauncher$5.run(JNLPAppletLauncher.java:1325)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:251)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:721)
	at java.awt.EventQueue.access$200(EventQueue.java:103)
	at java.awt.EventQueue$3.run(EventQueue.java:682)
	at java.awt.EventQueue$3.run(EventQueue.java:680)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:691)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:244)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:163)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:151)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:147)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:139)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:97)
Caused by: java.lang.UnsatisfiedLinkError: Can't load library: /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
	at java.lang.Runtime.load0(Runtime.java:792)
	at java.lang.System.load(System.java:1059)
	at org.jdesktop.applet.util.JNLPAppletLauncher.loadLibrary(JNLPAppletLauncher.java:2035)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.jogamp.common.jvm.JNILibLoaderBase.loadLibraryInternal(JNILibLoaderBase.java:412)
	at com.jogamp.common.jvm.JNILibLoaderBase.access$000(JNILibLoaderBase.java:59)
	at com.jogamp.common.jvm.JNILibLoaderBase$DefaultAction.loadLibrary(JNILibLoaderBase.java:90)
	at com.jogamp.common.jvm.JNILibLoaderBase.loadLibrary(JNILibLoaderBase.java:328)
	at jogamp.nativewindow.NWJNILibLoader.access$000(NWJNILibLoader.java:39)
	at jogamp.nativewindow.NWJNILibLoader$1.run(NWJNILibLoader.java:51)
	at jogamp.nativewindow.NWJNILibLoader$1.run(NWJNILibLoader.java:42)
	at java.security.AccessController.doPrivileged(Native Method)
	at jogamp.nativewindow.NWJNILibLoader.loadNativeWindow(NWJNILibLoader.java:42)
	at jogamp.nativewindow.jawt.JAWTUtil.<clinit>(JAWTUtil.java:183)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at javax.media.nativewindow.NativeWindowFactory$2.run(NativeWindowFactory.java:232)
	at javax.media.nativewindow.NativeWindowFactory$2.run(NativeWindowFactory.java:229)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.media.nativewindow.NativeWindowFactory.initSingleton(NativeWindowFactory.java:229)
	at javax.media.opengl.GLProfile.initProfilesForDefaultDevices(GLProfile.java:1385)
	at javax.media.opengl.GLProfile.access$100(GLProfile.java:76)
	at javax.media.opengl.GLProfile$1.run(GLProfile.java:174)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.media.opengl.GLProfile.initSingleton(GLProfile.java:140)
	at javax.media.opengl.GLProfile.getProfileMap(GLProfile.java:1887)
	at javax.media.opengl.GLProfile.get(GLProfile.java:785)
	at javax.media.opengl.GLProfile.get(GLProfile.java:801)
	at com.jogamp.newt.awt.applet.JOGLNewtApplet1Run.init(JOGLNewtApplet1Run.java:179)
	... 17 more
Comment 1 Xerxes Rånby 2012-10-06 01:58:27 CEST
By checking sha256sum i can confirm that it did try to load the armv6 libnativewindow_x11.so on a armv6hf system.

sha256sum /home/pi/jogamp-all-platforms/lib/linux-armv6hf/libnativewindow_x11.so 
cc0dc2b7104aef41dd7047384cfb2859061a95a8ac4a2b103f695040dbf31b3e  /home/pi/jogamp-all-platforms/lib/linux-armv6hf/libnativewindow_x11.so

sha256sum /home/pi/jogamp-all-platforms/lib/linux-armv6/libnativewindow_x11.so 
865a9438691f9ca027bdf51604ab831a452b9f9e71414c5eaefb47c75f3af4b9  /home/pi/jogamp-all-platforms/lib/linux-armv6/libnativewindow_x11.so

cd /tmp/jnlp-applet/jln558084847869472732/jln7859347536866624904/
sha256sum libnativewindow_x11.so 865a9438691f9ca027bdf51604ab831a452b9f9e71414c5eaefb47c75f3af4b9  libnativewindow_x11.so
Comment 2 Xerxes Rånby 2012-10-06 02:22:16 CEST
Test result when running using a Debian armel Wheezy,
On this system the correct library got loaded.
still the .bcm.vc.iv newt driver do not play along with the browser applet setup:


pi@raspberrypi ~ $ midori http://jogamp.org/deployment/archive/master/gluegen_598-joal_364-jogl_824-jocl_686-signed/jogl-applet-runner-newt-gears-normal-napplet2.html
java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.4) (6b24-1.11.4-3)
OpenJDK Zero VM (build 20.0-b12, mixed mode)
Unable to use Firefox's proxy settings. Using "DIRECT" as proxy type.
BCM.Display initIDs ok
BCM.Screen initIDs ok
BCM.Window initIDs ok
class com.jogamp.opengl.test.junit.jogl.demos.es2.GearsES2 has no 'window' field
class com.jogamp.opengl.test.junit.jogl.demos.es2.GearsES2 has no 'glWindow' field
java.lang.RuntimeException: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Native windowing type .bcm.vc.iv (custom) not yet supported, platform reported native windowing type: .bcm.vc.iv
	at com.jogamp.newt.awt.applet.JOGLNewtApplet1Run.init(JOGLNewtApplet1Run.java:220)
	at sun.applet.AppletPanel.run(AppletPanel.java:436)
	at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Native windowing type .bcm.vc.iv (custom) not yet supported, platform reported native windowing type: .bcm.vc.iv
	at jogamp.nativewindow.NativeWindowFactoryImpl.getAWTNativeWindow(NativeWindowFactoryImpl.java:99)
	at jogamp.nativewindow.NativeWindowFactoryImpl.getNativeWindowImpl(NativeWindowFactoryImpl.java:65)
	at javax.media.nativewindow.NativeWindowFactory.getNativeWindow(NativeWindowFactory.java:477)
	at jogamp.newt.awt.NewtFactoryAWT.getNativeWindow(NewtFactoryAWT.java:70)
	at com.jogamp.newt.awt.NewtCanvasAWT.reparentWindow(NewtCanvasAWT.java:428)
	at com.jogamp.newt.awt.NewtCanvasAWT.addNotify(NewtCanvasAWT.java:391)
	at java.awt.Container.addImpl(Container.java:1064)
	at java.awt.Container.add(Container.java:923)
	at com.jogamp.newt.awt.applet.JOGLNewtApplet1Run.init(JOGLNewtApplet1Run.java:216)
	... 2 more
Caused by: java.lang.IllegalArgumentException: Native windowing type .bcm.vc.iv (custom) not yet supported, platform reported native windowing type: .bcm.vc.iv
	at jogamp.nativewindow.NativeWindowFactoryImpl.getAWTNativeWindow(NativeWindowFactoryImpl.java:92)
	... 10 more
Comment 3 Sven Gothel 2012-10-06 02:32:05 CEST
This is expected, since we use the BCM IV headless toolkit we worked on together.
It cannot support AWT, since AWT from openjdk for Linux requires X11.

If we would use std X11 w/ the EGL blit solution, 
we may could it make working w/ AWT.

Hence we would need to already discussed mode-switch
for Rasp-Pi, i.e.
  - Headless BCM IV core
  - X11 w/ EGL blit standard

~Sven
Comment 4 Sven Gothel 2012-10-06 02:54:02 CEST
JOGL would already support X11/EGL (afaik), 
but now we would need to disable the BCM IV auto detection.

We may need a smart heuristic to do that automatic, i.e.:
  - if JVM is running w/ AWT enabled (!headless) _and_
    native windowing TK is X11 _and_ a display connection to local
    could be established - then use X11/EGL-blit.

  - otherwise use BCM.IV if available
Comment 5 Xerxes Rånby 2013-09-27 12:03:14 CEST
Example code on how to do efficient DMA GPU to CPU memory copy on the Pi.
https://github.com/shirro/pi-eglonx - "OpenGL ES rendering to X window"