Bug 626 - Support X11/EGL-blit on Rasp.Pi allowing use of AWT and Applets (was: NApplet fail using newt bcm.vc.iv driver)
Summary: Support X11/EGL-blit on Rasp.Pi allowing use of AWT and Applets (was: NApplet...
Status: CONFIRMED
Alias: None
Product: Jogl
Classification: JogAmp
Component: applet (show other bugs)
Version: 2
Hardware: All all
: P2 enhancement
Assignee: Sven Gothel
URL:
Depends on:
Blocks:
 
Reported: 2012-10-06 01:38 CEST by Xerxes Rånby
Modified: 2013-09-27 12:03 CEST (History)
1 user (show)

See Also:
Type: ---
SCM Refs:
Workaround: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Xerxes Rånby 2012-10-06 01:38:26 CEST 
I decided to test if it was possible to run the JogAmp applet demos from the midori browser on the RaspberryPi.

Good news: the broadcom vc iv autodetection worked and the GPU is initialized.
Bad news: something else broke, log below:

Tested using Rasbian wheezy armhf.

pi@raspberrypi ~ $ ldd /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
	not a dynamic executable
pi@raspberrypi ~ $ ls -la /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
-rw-r--r-- 1 pi pi 4992 Oct  5 23:26 /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
pi@raspberrypi ~ $ file /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
/tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so: ELF 32-bit LSB shared object, ARM, version 1 (SYSV), dynamically linked, BuildID[sha1]=0xb624f0122ee6952661f57ea736481c8653f1ed6e, stripped


pi@raspberrypi ~ $ midori http://jogamp.org/deployment/archive/master/gluegen_598-joal_364-jogl_824-jocl_686-signed/jogl-applet-runner-newt-gears-normal-napplet2.html
The configuration couldn't be loaded: Not a directory

** (midori:2468): WARNING **: Can't open /home/pi/.config/midori/cookies.db

** (midori:2468): WARNING **: Failed to execute query: (null)

** (midori:2468): WARNING **: Failed to execute query: (null)

** (midori:2468): CRITICAL **: midori_browser_new_history_item: assertion `db != NULL' failed

** (midori:2468): CRITICAL **: midori_browser_update_history_title: assertion `db != NULL' failed
java version "1.7.0_07"
OpenJDK Runtime Environment (IcedTea7 2.3.2) (7u7-2.3.2a-1+rpi1)
JamVM (build 1.6.0-devel, inline-threaded interpreter with stack-caching)

** (midori:2468): CRITICAL **: midori_browser_new_history_item: assertion `db != NULL' failed

** (midori:2468): CRITICAL **: midori_browser_update_history_title: assertion `db != NULL' failed

** (midori:2468): CRITICAL **: midori_browser_new_history_item: assertion `db != NULL' failed

** (midori:2468): CRITICAL **: midori_browser_update_history_title: assertion `db != NULL' failed
BCM.Display initIDs ok
BCM.Screen initIDs ok
BCM.Window initIDs ok
class com.jogamp.opengl.test.junit.jogl.demos.es2.GearsES2 has no 'window' field
class com.jogamp.opengl.test.junit.jogl.demos.es2.GearsES2 has no 'glWindow' field
java.awt.IllegalComponentStateException: component must be showing on the screen to determine its location
	at java.awt.Component.getLocationOnScreen_NoTreeLock(Component.java:2044)
	at java.awt.Component.getLocationOnScreen(Component.java:2018)
	at com.jogamp.newt.awt.applet.JOGLNewtApplet1Run.start(JOGLNewtApplet1Run.java:234)
	at sun.applet.AppletPanel.run(AppletPanel.java:475)
	at java.lang.Thread.run(Thread.java:722)
JNLPAppletLauncher: static initializer
java.lang.RuntimeException: java.lang.UnsatisfiedLinkError: Can't load library: /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
	at com.jogamp.newt.awt.applet.JOGLNewtApplet1Run.init(JOGLNewtApplet1Run.java:220)
	at org.jdesktop.applet.util.JNLPAppletLauncher.startSubApplet(JNLPAppletLauncher.java:1995)
	at org.jdesktop.applet.util.JNLPAppletLauncher.access$200(JNLPAppletLauncher.java:662)
	at org.jdesktop.applet.util.JNLPAppletLauncher$5.run(JNLPAppletLauncher.java:1325)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:251)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:721)
	at java.awt.EventQueue.access$200(EventQueue.java:103)
	at java.awt.EventQueue$3.run(EventQueue.java:682)
	at java.awt.EventQueue$3.run(EventQueue.java:680)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:691)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:244)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:163)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:151)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:147)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:139)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:97)
Caused by: java.lang.UnsatisfiedLinkError: Can't load library: /tmp/jnlp-applet/jln6481281403825762696/jln2486257265421406286/libnativewindow_awt.so
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
	at java.lang.Runtime.load0(Runtime.java:792)
	at java.lang.System.load(System.java:1059)
	at org.jdesktop.applet.util.JNLPAppletLauncher.loadLibrary(JNLPAppletLauncher.java:2035)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.jogamp.common.jvm.JNILibLoaderBase.loadLibraryInternal(JNILibLoaderBase.java:412)
	at com.jogamp.common.jvm.JNILibLoaderBase.access$000(JNILibLoaderBase.java:59)
	at com.jogamp.common.jvm.JNILibLoaderBase$DefaultAction.loadLibrary(JNILibLoaderBase.java:90)
	at com.jogamp.common.jvm.JNILibLoaderBase.loadLibrary(JNILibLoaderBase.java:328)
	at jogamp.nativewindow.NWJNILibLoader.access$000(NWJNILibLoader.java:39)
	at jogamp.nativewindow.NWJNILibLoader$1.run(NWJNILibLoader.java:51)
	at jogamp.nativewindow.NWJNILibLoader$1.run(NWJNILibLoader.java:42)
	at java.security.AccessController.doPrivileged(Native Method)
	at jogamp.nativewindow.NWJNILibLoader.loadNativeWindow(NWJNILibLoader.java:42)
	at jogamp.nativewindow.jawt.JAWTUtil.<clinit>(JAWTUtil.java:183)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at javax.media.nativewindow.NativeWindowFactory$2.run(NativeWindowFactory.java:232)
	at javax.media.nativewindow.NativeWindowFactory$2.run(NativeWindowFactory.java:229)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.media.nativewindow.NativeWindowFactory.initSingleton(NativeWindowFactory.java:229)
	at javax.media.opengl.GLProfile.initProfilesForDefaultDevices(GLProfile.java:1385)
	at javax.media.opengl.GLProfile.access$100(GLProfile.java:76)
	at javax.media.opengl.GLProfile$1.run(GLProfile.java:174)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.media.opengl.GLProfile.initSingleton(GLProfile.java:140)
	at javax.media.opengl.GLProfile.getProfileMap(GLProfile.java:1887)
	at javax.media.opengl.GLProfile.get(GLProfile.java:785)
	at javax.media.opengl.GLProfile.get(GLProfile.java:801)
	at com.jogamp.newt.awt.applet.JOGLNewtApplet1Run.init(JOGLNewtApplet1Run.java:179)
	... 17 more
Comment 1 Xerxes Rånby 2012-10-06 01:58:27 CEST 
By checking sha256sum i can confirm that it did try to load the armv6 libnativewindow_x11.so on a armv6hf system.

sha256sum /home/pi/jogamp-all-platforms/lib/linux-armv6hf/libnativewindow_x11.so 
cc0dc2b7104aef41dd7047384cfb2859061a95a8ac4a2b103f695040dbf31b3e  /home/pi/jogamp-all-platforms/lib/linux-armv6hf/libnativewindow_x11.so

sha256sum /home/pi/jogamp-all-platforms/lib/linux-armv6/libnativewindow_x11.so 
865a9438691f9ca027bdf51604ab831a452b9f9e71414c5eaefb47c75f3af4b9  /home/pi/jogamp-all-platforms/lib/linux-armv6/libnativewindow_x11.so

cd /tmp/jnlp-applet/jln558084847869472732/jln7859347536866624904/
sha256sum libnativewindow_x11.so 865a9438691f9ca027bdf51604ab831a452b9f9e71414c5eaefb47c75f3af4b9  libnativewindow_x11.so
Comment 2 Xerxes Rånby 2012-10-06 02:22:16 CEST 
Test result when running using a Debian armel Wheezy,
On this system the correct library got loaded.
still the .bcm.vc.iv newt driver do not play along with the browser applet setup:


pi@raspberrypi ~ $ midori http://jogamp.org/deployment/archive/master/gluegen_598-joal_364-jogl_824-jocl_686-signed/jogl-applet-runner-newt-gears-normal-napplet2.html
java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.4) (6b24-1.11.4-3)
OpenJDK Zero VM (build 20.0-b12, mixed mode)
Unable to use Firefox's proxy settings. Using "DIRECT" as proxy type.
BCM.Display initIDs ok
BCM.Screen initIDs ok
BCM.Window initIDs ok
class com.jogamp.opengl.test.junit.jogl.demos.es2.GearsES2 has no 'window' field
class com.jogamp.opengl.test.junit.jogl.demos.es2.GearsES2 has no 'glWindow' field
java.lang.RuntimeException: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Native windowing type .bcm.vc.iv (custom) not yet supported, platform reported native windowing type: .bcm.vc.iv
	at com.jogamp.newt.awt.applet.JOGLNewtApplet1Run.init(JOGLNewtApplet1Run.java:220)
	at sun.applet.AppletPanel.run(AppletPanel.java:436)
	at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Native windowing type .bcm.vc.iv (custom) not yet supported, platform reported native windowing type: .bcm.vc.iv
	at jogamp.nativewindow.NativeWindowFactoryImpl.getAWTNativeWindow(NativeWindowFactoryImpl.java:99)
	at jogamp.nativewindow.NativeWindowFactoryImpl.getNativeWindowImpl(NativeWindowFactoryImpl.java:65)
	at javax.media.nativewindow.NativeWindowFactory.getNativeWindow(NativeWindowFactory.java:477)
	at jogamp.newt.awt.NewtFactoryAWT.getNativeWindow(NewtFactoryAWT.java:70)
	at com.jogamp.newt.awt.NewtCanvasAWT.reparentWindow(NewtCanvasAWT.java:428)
	at com.jogamp.newt.awt.NewtCanvasAWT.addNotify(NewtCanvasAWT.java:391)
	at java.awt.Container.addImpl(Container.java:1064)
	at java.awt.Container.add(Container.java:923)
	at com.jogamp.newt.awt.applet.JOGLNewtApplet1Run.init(JOGLNewtApplet1Run.java:216)
	... 2 more
Caused by: java.lang.IllegalArgumentException: Native windowing type .bcm.vc.iv (custom) not yet supported, platform reported native windowing type: .bcm.vc.iv
	at jogamp.nativewindow.NativeWindowFactoryImpl.getAWTNativeWindow(NativeWindowFactoryImpl.java:92)
	... 10 more
Comment 3 Sven Gothel 2012-10-06 02:32:05 CEST 
This is expected, since we use the BCM IV headless toolkit we worked on together.
It cannot support AWT, since AWT from openjdk for Linux requires X11.

If we would use std X11 w/ the EGL blit solution, 
we may could it make working w/ AWT.

Hence we would need to already discussed mode-switch
for Rasp-Pi, i.e.
  - Headless BCM IV core
  - X11 w/ EGL blit standard

~Sven
Comment 4 Sven Gothel 2012-10-06 02:54:02 CEST 
JOGL would already support X11/EGL (afaik), 
but now we would need to disable the BCM IV auto detection.

We may need a smart heuristic to do that automatic, i.e.:
  - if JVM is running w/ AWT enabled (!headless) _and_
    native windowing TK is X11 _and_ a display connection to local
    could be established - then use X11/EGL-blit.

  - otherwise use BCM.IV if available
Comment 5 Xerxes Rånby 2013-09-27 12:03:14 CEST 
Example code on how to do efficient DMA GPU to CPU memory copy on the Pi.
https://github.com/shirro/pi-eglonx - "OpenGL ES rendering to X window"