Bug 1246

- dunno whether it is scattered -> static MachineDescription

- Since it is an ELF binary, we should be able to detect it.

- As you mentioned, the PAGESIZE is different and the
  static (assumed) may not match the queried one at runtime,
  but AFAIK we prefer the runtime one .. no?
  So does it crash -> exception?

- Then you can simply try to remove that criteria from the 
  comparison .. IMHO OK.

- Does the ELF reader properly detect the platform?

- Also -> version 2.4.0

Created attachment 754 [details]
ppc64le support proposal

Hi,

I gave it another try and got it to build eventually.
Patch attached. Comments welcome.

Thanks.

(In reply to gilles.filippini from comment #3)
> Hi,
> 
> I gave it another try and got it to build eventually.
> Patch attached. Comments welcome.
> 
> Thanks.

Hi Gilles(?),

first of all KUDOS for your approach diving into our
ant build system and data!

After the 2.3.2 release I like to take your patches!

As mentioned earlier, we may be able to enhance the generic
validation, so the pagesize criteria does not hit.
This will allow us not needing to add the static MachineData config
and be more versatile for other similar LP64 arch, 
see comment 1.

But this can be done in a followup commit.

For now, it would be great, if you can offer your patch
either as:
  - git patch (email formatted)
  - git pull request

This way your authorship is being preserved
and this is required to allow an code audit to defend our IP.
(i.e. that we don't violate rights of others .. i.e. your employer)

Thank you, Sven

(In reply to Sven Gothel from comment #4)
> first of all KUDOS for your approach diving into our
> ant build system and data!

Thanks :)

> 
> After the 2.3.2 release I like to take your patches!
> 
> As mentioned earlier, we may be able to enhance the generic
> validation, so the pagesize criteria does not hit.
> This will allow us not needing to add the static MachineData config
> and be more versatile for other similar LP64 arch, 
> see comment 1.
> 
> But this can be done in a followup commit.
> 
> For now, it would be great, if you can offer your patch
> either as:
>   - git patch (email formatted)
>   - git pull request
> 
> This way your authorship is being preserved
> and this is required to allow an code audit to defend our IP.
> (i.e. that we don't violate rights of others .. i.e. your employer)

Will do, but before I need to test this patch usability.
I've somewhat adapted it to the release 2.2.4 currently in Debian but failed to run the classical OneTriangleAWT example:

Exception in thread "main" java.lang.ExceptionInInitializerError
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:278)
	at javax.media.nativewindow.NativeWindowFactory$3.run(NativeWindowFactory.java:338)
	at javax.media.nativewindow.NativeWindowFactory$3.run(NativeWindowFactory.java:334)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.media.nativewindow.NativeWindowFactory.initSingleton(NativeWindowFactory.java:334)
	at javax.media.opengl.GLProfile.initProfilesForDefaultDevices(GLProfile.java:1690)
	at javax.media.opengl.GLProfile.access$000(GLProfile.java:77)
	at javax.media.opengl.GLProfile$1.run(GLProfile.java:201)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.media.opengl.GLProfile.initSingleton(GLProfile.java:187)
	at javax.media.opengl.GLProfile.getProfileMap(GLProfile.java:2246)
	at javax.media.opengl.GLProfile.get(GLProfile.java:959)
	at javax.media.opengl.GLProfile.getDefault(GLProfile.java:693)
	at javax.media.opengl.GLProfile.getDefault(GLProfile.java:704)
	at OneTriangleAWT.main(OneTriangleAWT.java:19)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 9
	at jogamp.nativewindow.jawt.JAWT.size(JAWT.java:29)
	at jogamp.nativewindow.jawt.JAWT.create(JAWT.java:33)
	at jogamp.nativewindow.jawt.JAWTUtil.getJAWT(JAWTUtil.java:246)
	at jogamp.nativewindow.jawt.JAWTUtil.<clinit>(JAWTUtil.java:330)
	... 16 more

Does this backtrace tell you something?

Thanks,

_gilles.

(In reply to gilles.filippini from comment #5)
> I've somewhat adapted it to the release 2.2.4 currently in Debian but failed
> to run the classical OneTriangleAWT example:

As I understand it jogl needs to be patched as well for this example to work :/

Is there a simple way to check for gluegen usability which does not require jogl?

Thanks,

_g.

(In reply to gilles.filippini from comment #6)
> (In reply to gilles.filippini from comment #5)
> > I've somewhat adapted it to the release 2.2.4 currently in Debian but failed
> > to run the classical OneTriangleAWT example:
> 
> As I understand it jogl needs to be patched as well for this example to work
> :/

Yes, sorry, this is a bit annoying I know.
That would be the next thing on you platter of support:
  - joal, jogl and jocl

> 
> Is there a simple way to check for gluegen usability which does not require
> jogl?

- just run the gluegen unit tests.

all via ant, or a single one as I do:

cd gluegen/make
vi scripts/runtest.sh
bash scripts/runtest.sh ../build

IMHO the following test should be sufficient,
triggering the 'loading path':
  com.jogamp.common.util.TestPlatform01 

~Sven

How are these source files in src/java/jogamp/common/os/elf/ supposed to be generated?

Ehdr.java
Shdr.java

Is there an option to force the target generate.os.sources during the build?

Thanks,

_g.

One more step. The ArrayIndexOutOfBoundsException is solved and the OneTriangleAWT now fails with this error:

libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00003fffa95ee284, pid=9911, tid=70366626443680
...

Since I test through a vnc session on a remote Debian porterbox I can't tell where the problem is. Could this kind of error come from a faulty gluegen / jogl build?

Thanks,

_g.

(In reply to gilles.filippini from comment #9)
> One more step. The ArrayIndexOutOfBoundsException is solved and the
> OneTriangleAWT now fails with this error:
> 
> libGL error: No matching fbConfigs or visuals found
> libGL error: failed to load driver: swrast
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00003fffa95ee284, pid=9911, tid=70366626443680
> ...
> 
> Since I test through a vnc session on a remote Debian porterbox I can't tell
> where the problem is. Could this kind of error come from a faulty gluegen /
> jogl build?
> 
> Thanks,
> 
> _g.

Found a stack trace:
Stack: [0x00003fff26cc0000,0x00003fff26ec0000],  sp=0x00003fff26ebd710,  free space=2037k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x54e284]  jni_GetLongField+0x74
V  [libjvm.so+0x566b68]  jni_GetDirectBufferAddress+0x118
C  [libnativewindow_x11.so+0x42ec]  Java_jogamp_nativewindow_x11_X11Lib_XRenderFindVisualFormat1+0x54
j  jogamp.nativewindow.x11.X11Lib.XRenderFindVisualFormat1(JJ)Ljava/nio/ByteBuffer;+0
j  jogamp.nativewindow.x11.X11Lib.XRenderFindVisualFormat(JJ)Ljogamp/nativewindow/x11/XRenderPictFormat;+2
j  jogamp.opengl.x11.glx.X11GLXGraphicsConfiguration.XVisual2XRenderMask(JJ)Ljogamp/nativewindow/x11/XRenderDirectFormat;+2
j  jogamp.opengl.x11.glx.X11GLXGraphicsConfiguration.XVisualInfo2GLCapabilities(Lcom/jogamp/nativewindow/x11/X11GraphicsDevice;Ljavax/media/opengl/GL
Profile;Ljogamp/nativewindow/x11/XVisualInfo;IZ)Ljogamp/opengl/x11/glx/X11GLCapabilities;+256
j  jogamp.opengl.x11.glx.X11GLXGraphicsConfigurationFactory.chooseGraphicsConfigurationXVisual(Ljavax/media/opengl/GLCapabilitiesImmutable;Ljavax/med
ia/opengl/GLCapabilitiesImmutable;Ljavax/media/opengl/GLCapabilitiesChooser;Lcom/jogamp/nativewindow/x11/X11GraphicsScreen;I)Ljogamp/opengl/x11/glx/X
11GLXGraphicsConfiguration;+269
j  jogamp.opengl.x11.glx.X11GLXGraphicsConfigurationFactory.chooseGraphicsConfigurationStatic(Ljavax/media/opengl/GLCapabilitiesImmutable;Ljavax/medi
a/opengl/GLCapabilitiesImmutable;Ljavax/media/opengl/GLCapabilitiesChooser;Lcom/jogamp/nativewindow/x11/X11GraphicsScreen;I)Ljogamp/opengl/x11/glx/X1
1GLXGraphicsConfiguration;+164
j  jogamp.opengl.x11.glx.X11GLXDrawableFactory.createMutableSurfaceImpl(Ljavax/media/nativewindow/AbstractGraphicsDevice;ZLjavax/media/opengl/GLCapab
ilitiesImmutable;Ljavax/media/opengl/GLCapabilitiesImmutable;Ljavax/media/opengl/GLCapabilitiesChooser;Ljavax/media/nativewindow/UpstreamSurfaceHook;
)Ljavax/media/nativewindow/ProxySurface;+69
j  jogamp.opengl.x11.glx.X11GLXDrawableFactory.createDummySurfaceImpl(Ljavax/media/nativewindow/AbstractGraphicsDevice;ZLjavax/media/opengl/GLCapabil
itiesImmutable;Ljavax/media/opengl/GLCapabilitiesImmutable;Ljavax/media/opengl/GLCapabilitiesChooser;II)Ljavax/media/nativewindow/ProxySurface;+24
j  jogamp.opengl.x11.glx.X11GLXDrawableFactory$SharedResourceImplementation.createSharedResource(Ljava/lang/String;)Ljogamp/opengl/SharedResourceRunn
er$Resource;+133
j  jogamp.opengl.SharedResourceRunner.run()V+267
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub
V  [libjvm.so+0x53a59c]  JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x46c
V  [libjvm.so+0x7c5c14]  os::os_exception_wrapper(void (*)(JavaValue*, methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x34
V  [libjvm.so+0x539014]  JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Thread*)+0x224
V  [libjvm.so+0x5856a8]  thread_entry(JavaThread*, Thread*)+0xb8
V  [libjvm.so+0x945a58]  JavaThread::thread_main_inner()+0x198
V  [libjvm.so+0x945d68]  JavaThread::run()+0x2a8
V  [libjvm.so+0x7bc050]  java_start(Thread*)+0x160
C  [libpthread.so.0+0x89a4]  start_thread+0xf4

(In reply to gilles.filippini from comment #8)
> How are these source files in src/java/jogamp/common/os/elf/ supposed to be
> generated?
> 
> Ehdr.java
> Shdr.java
> 
> Is there an option to force the target generate.os.sources during the build?
> 

you could issue their generation manually via ant.

however, we choose to pre-generate certain files.
I would need to check whether those were post-edited or not.

(In reply to gilles.filippini from comment #9)
> One more step. The ArrayIndexOutOfBoundsException is solved and the
> OneTriangleAWT now fails with this error:
> 
> libGL error: No matching fbConfigs or visuals found
> libGL error: failed to load driver: swrast
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00003fffa95ee284, pid=9911, tid=70366626443680
> ...
> 
> Since I test through a vnc session on a remote Debian porterbox I can't tell
> where the problem is. Could this kind of error come from a faulty gluegen /
> jogl build?
> 

I assume you produce jogl similar to gluegen, i.e. w/ your compiler flags?

But 1st things 1st: did you succeed w/ running some or all 
GlueGen unit tests?
Not only report the failures - reporting success is also very welcome! :)

(In reply to Sven Gothel from comment #12)
> But 1st things 1st: did you succeed w/ running some or all 
> GlueGen unit tests?
> Not only report the failures - reporting success is also very welcome! :)

All tests but 6 pass:

TestVersionSemantics doesn't build (I don't have semver).

test01URLCompositioning:
There was 1 failure:
1) test01URLCompositioning(com.jogamp.common.util.TestIOUtilURICompose)
java.net.MalformedURLException: unknown protocol: asset

testTempJarCache02AddNativeLibs:
There was 1 failure:
1) testTempJarCache02AddNativeLibs(com.jogamp.common.util.TestTempJarCache)
java.io.FileNotFoundException: /home/pini/gluegen2-2.2.4/build/gluegen2-rt-natives-linux-ppc64le.jar (No such file or directory)

testJarUtilFlat01:
There were 4 failures:TempFileCache: removeAll(/tmp/jogamp_0000/file_cache/jln8133976196119522112/jln7418965195467938792/jogamp/common/jvm)

1) testJarUtilFlat01(com.jogamp.common.util.TestJarUtil)TempFileCache: removeAll(/tmp/jogamp_0000/file_cache/jln8133976196119522112/jln7418965195467938792/jogamp/common/jvm/JVMUtil.class)

test00TempJarCacheSimplePath:
There were 2 failures:
1) test00TempJarCacheSimplePath(com.jogamp.common.net.TestNetIOURIReservedCharsBug908)
java.lang.UnsatisfiedLinkError: Can't load library: /home/pini/gluegen2-2.2.4/libgluegen2-rt.so

pcppMacroDefinitionTest:
There was 1 failure:
1) pcppMacroDefinitionTest(com.jogamp.gluegen.test.junit.generation.PCPPTest)
java.lang.ExceptionInInitializerError

_g.

I'm back on the 2.3.1 release because somebody has just uploaded this version of gluegen to Debian experimental.

I'm now puzzled by the tests results where I read in many places:
Duplicate/Compatible MachineDataInfo in StaticConfigs: Elements [7: PPC_64_UNIX(7)] and [6: LP64_UNIX(6)]
MachineDataInfoStatic: PPC_64_UNIX(7): MachineDataInfo: runtimeValidated false, 32Bit false, primitive size / alignment:
  int8    1 / 1, int16   2 / 2
  int     4 / 4, long    8 / 8
  int32   4 / 4, int64   8 / 8
  float   4 / 4, double  8 / 8, ldouble 16 / 16
  pointer 8 / 8, page    65536
MachineDataInfoStatic: LP64_UNIX(6): MachineDataInfo: runtimeValidated false, 32Bit false, primitive size / alignment:
  int8    1 / 1, int16   2 / 2
  int     4 / 4, long    8 / 8
  int32   4 / 4, int64   8 / 8
  float   4 / 4, double  8 / 8, ldouble 16 / 16
  pointer 8 / 8, page    4096

Why is this an error? Obviously the page sizes are different.

Thanks,

_g.

I've inhibited the "Duplicate/Compatible MachineDataInfo" error and there is only 4 failures left:

1) TestJarUtil fails loading inexistant gluegen2-rt-natives-linux-ppc64le.jar which is not generated by the Debian package. I guess I can ignore this error.

2) TestUri99LaunchOnReservedCharPathBug908 fails loading libgluegen2-rt.so with this error: 
n/a - Native Library /home/pini/debian/gluegen2/debian/libgluegen2-jni/usr/lib/jni/libgluegen2-rt.so already loaded in another classloader

3) and 4) have the same pattern: TestStructGen01 and TestStructGen02 fail with:
1) test01(com.jogamp.gluegen.test.junit.structgen.TestStructGen02)
java.lang.InternalError: Not set
        at jogamp.common.os.MachineDataInfoRuntime.getStatic(MachineDataInfoRuntime.java:78)
        at com.jogamp.gluegen.test.junit.structgen.Pixel.<clinit>(Pixel.java:18)
        at com.jogamp.gluegen.test.junit.structgen.TestStructGen02.test01(TestStructGen02.java:23)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
...

I guess the latter is the most annoying. What do you think?

Thanks,

_g

Hi,

The good new is that I have exactly the same failures on arch amd64 :)

Thanks,

_g.

Pull request created.
==> https://github.com/sgothel/gluegen/pull/29

Thanks,

_g.

Good news: the patch was finally tested on a baremetal ppc64le machine, and it works.
===> https://lists.debian.org/debian-powerpc/2015/10/msg00035.html

(In reply to gilles.filippini from comment #17)
> Pull request created.
> ==> https://github.com/sgothel/gluegen/pull/29
> 
> Thanks,
> 
> _g.

Great job :) I have just looked at your pull request, it's minimal, it does the job, big kudos to you.

Hi,

Now I better understand comment 1. I've canceled the previous pull request and created another one with relaxed constraint on runtime pagesize.
==> https://github.com/sgothel/gluegen/pull/30

Thanks,

_g.

Merged Gilles pull request, GlueGen commit 6d87df8b109f045433575cd94b22ba8d8150903a

In case this is not sufficient to build all modules, please re-open and advise.