#jogamp @ irc.freenode.net - 20130930 05:05:30 (UTC)

20130930 05:05:30 -jogamp- Previous @ http://jogamp.org/log/irc/jogamp_20130929050530.html
20130930 05:05:30 -jogamp- This channel is logged @ http://jogamp.org/log/irc/jogamp_20130930050530.html
20130930 06:29:24 * hharrison (~chatzilla@anon) Quit (Remote host closed the connection)
20130930 07:40:56 * monsieur_max (~maxime@anon) has joined #jogamp
20130930 10:38:14 * cryobacterium (~ilias@anon) has joined #jogamp
20130930 10:38:29 <cryobacterium> hello all
20130930 10:40:13 <cryobacterium> I'm still experiencing some problems regarding deployment of my jogl project in jars, i.e. in its own jar including all its libraries plus an additional one for the native libs
20130930 10:42:26 <xranby> cryobacterium: what is the unexpected behaviour?
20130930 10:43:46 <cryobacterium> I'm building a jar containing my project + gluegen-rt + jogl-all, and another next to it, named project-native-linux-i586.jar that contains the gluegen-rt and jogl natives. the problem is that while the native jar is found, it can't load the libnativewindow_awt.so library (even if it's still in the correct jar)
20130930 10:44:57 <xranby> cryobacterium: did you place unpack the contents of gluegen-rt.jar and jogl-all.jar into your project jar?
20130930 10:45:26 <cryobacterium> yes
20130930 10:46:10 <cryobacterium> I'm building everything through ant, so my main jar consists of my stuff plus the two jars you're mentioning, and the second jar is just a merge of the two native libs I need
20130930 10:46:24 <xranby> check ldd libnativewindow_awt.so on this system to verify that this library can resolve its symbols
20130930 10:48:15 <cryobacterium> I get (as expected) "ldd: ./libnativewindow_awt.so: No such file or directory"
20130930 10:48:32 <cryobacterium> however the natives jar does contain this file
20130930 10:49:04 <xranby> please run ldd on the unpacked file to verify if the library can be used on this system
20130930 10:49:51 <xranby> i want to make sure the file is ok and not in a compressed state
20130930 10:50:17 <cryobacterium> if I unpack the jar, I can both ldd and also run my project without problems
20130930 10:50:30 <xranby> some webbrowsers downloads and stores files from the jogamp server in .gz form
20130930 10:50:33 <xranby> ok
20130930 10:53:00 <cryobacterium> actually if I start my project with an extracted native jar, the program starts but I get some catched zipexceptions regarding the (unzipped now) natives jar file
20130930 10:53:22 <cryobacterium> but still, ldd displays properly the libs so it should be fine
20130930 10:54:40 <cryobacterium> I noticed this just now, from ldd: libjawt.so => not found
20130930 10:54:59 <cryobacterium> the rest of the contents seem fine
20130930 11:02:32 <xranby> libjawt.so is part of the java jre and is traditionally loaded dynamically by invoking java.awt.Toolkit.getDefaultToolkit()
20130930 11:03:00 <cryobacterium> so it should be fine then
20130930 11:03:14 <cryobacterium> what is the problem though? could it be the way I'm creating the natives jar?
20130930 11:07:35 <xranby> libjawt.so can still be the issue.. depending on how you initialize your appication. according to http://mail.openjdk.java.net/pipermail/jdk7-dev/2011-August/002257.html is java.awt.Toolkit.getDefaultToolkit() no longer sufficient to load libjawt.so
20130930 11:07:43 <xranby> on java 7 systems
20130930 11:08:30 <xranby> cryobacterium: i would like you to run your application with more debug output enabled
20130930 11:09:06 <xranby> in order to see the exact paths used to load the librarys
20130930 11:10:57 <xranby> cryobacterium: is awt a requirement for your project?
20130930 11:11:46 <xranby> cryobacterium: the reason i ask is because jogl can operate without using awt and instead use the newt window and input system
20130930 11:23:45 <xranby> Discussion from jul 2012: http://web.archiveorange.com/archive/v/rp9RwolB2idA9so0qVe3 - <AWT Dev> JAWT Breakage in OpenJDK 7/8
20130930 11:31:17 <xranby> thus the fix that got added back then was one update to the RPATH used by the traditional java launchers: http://hg.openjdk.java.net/jdk8/awt/jdk/rev/f003387c33ad
20130930 11:32:06 <xranby> cryobacterium: how do you launch your application?
20130930 11:51:25 <xranby> cryobacterium: jogamp's solution is to load jawt using JAWTJNILibLoader
20130930 11:52:33 <xranby> JAWTJNILibLoader.initSingleton();
20130930 11:53:54 <xranby> this gets loaded if you use JAWTUtil in your code
20130930 11:57:21 <cryobacterium> xranby: sorry, I was away shortly
20130930 12:00:26 <cryobacterium> xranby: I'm using awt for some cases, but mostly using swing
20130930 12:01:03 <cryobacterium> and also, my application is launched typically by launching the jar; I'm not sure if I'm answering your question
20130930 12:03:27 <xranby> cryobacterium: it may help if you publish your ant scripts and a toy example project how to setup a reproducer
20130930 12:03:36 <xranby> something that we can test here
20130930 12:05:54 <xranby> also debug logs is of interest http://jogamp.org/wiki/index.php/Jogl_FAQ#Detailed_Debug_Log
20130930 12:13:34 <cryobacterium> xranby: thanks for the help -- my project actually is hosted on sourceforge. for example, my ant script is located here: http://sourceforge.net/p/cuttlefish/code/HEAD/tree/build.xml
20130930 12:13:56 <cryobacterium> I'll try to get hold of debug logs
20130930 12:15:36 <xranby> cryobacterium: i try build it here and see if i run into the same issue
20130930 12:16:17 <xranby> using revision 282
20130930 12:16:23 <cryobacterium> xranby: that would be awesome! this script is only for the linux-i586 architecture though, you could easily modify it for your architecture if different
20130930 12:17:14 <cryobacterium> running target dist should do the jar building/deploying. target run successfully runs my project, but when I try to run it with java -jar it fails
20130930 12:19:57 <xranby> cryobacterium: ok .. looks like I reproduced it https://gist.github.com/xranby/a95192dd6c3d7c35447a
20130930 12:20:38 <xranby> and.. ant run work
20130930 12:20:49 <cryobacterium> yes, indeed
20130930 12:21:53 <cryobacterium> xranby: I've also got the debug log that you mentioned
20130930 12:23:23 <cryobacterium> xranby: my exception is slightly different: http://pastebin.com/0jc8eyXE
20130930 12:24:36 <cryobacterium> and my test_dbg.log: https://dl.dropboxusercontent.com/u/4500209/test_dbg.log
20130930 12:31:33 <xranby> this is the cuttlefish debug log https://gist.github.com/xranby/a95192dd6c3d7c35447a#file-java-djnlp-newt-debug-all-djnlp-nativewindow-debug-all-djnlp-jogl-debug-all-jar-cuttlefish-jar
20130930 12:31:35 <xranby> java -Djnlp.newt.debug=all -Djnlp.nativewindow.debug=all -Djnlp.jogl.debug=all -jar cuttlefish.jar
20130930 12:35:11 <cryobacterium> so does this mean that I'm indeed doing something wrong with initialization?
20130930 12:35:55 <cryobacterium> that's weird, since I can run the project fine through ant or eclipse...
20130930 12:36:49 <cryobacterium> (my cuttlefish debug log: https://dl.dropboxusercontent.com/u/4500209/cuttlefish_dbg.log )
20130930 12:47:23 <xranby> cryobacterium: i do not see anything suspicious yet.. i am still investigating
20130930 12:47:47 <cryobacterium> thanks a lot, I'm also looking into it
20130930 12:52:49 <monsieur_max> not jogamp related but still somewhat interesting : https://fr.dolphin-emu.org/blog/2013/09/26/dolphin-emulator-and-opengl-drivers-hall-fameshame/?cr=fr
20130930 12:55:09 <xranby> monsieur_max: interesting how nvidia is both at the top and the bottom (dolphin is not working using the mobile nvidia tegra 2 & 3 at all)
20130930 12:55:47 <monsieur_max> haha, yes, did not really noticed it
20130930 13:00:53 <xranby> cryobacterium: the reason why ant run work is because it is placing all jars on the classpath like this: java -classpath /home/xranby/cuttlefish-code/lib/collections-generic-4.01.jar:/home/xranby/cuttlefish-code/lib/commons-cli-1.2.jar:/home/xranby/cuttlefish-code/lib/gephi-toolkit.jar:/home/xranby/cuttlefish-code/lib/gson-1.7.1.jar:/home/xranby/cuttlefish-code/lib/jogl-all-platforms/gluegen-rt.jar:/home/xranby/cuttlefish-code/lib/jogl-all-platforms/jogl-all
20130930 13:01:37 <cryobacterium> but what about the native libs?
20130930 13:01:50 <xranby> ant run placed the native libs on the classpath as well
20130930 13:01:55 <xranby> i think eclipse run do the same
20130930 13:02:03 <cryobacterium> hmm
20130930 13:02:44 <cryobacterium> I noticed that if I extract the .so files cuttlefish runs normally
20130930 13:02:59 <cryobacterium> so there seems to be a problem with the classloader and the jar'ed .so files
20130930 13:03:14 <cryobacterium> it looks like I should avoid this approach, right?
20130930 13:03:45 <xranby> eclipse "export runnable jar" actually adds the native jar to the manifest
20130930 13:04:17 <cryobacterium> I've also tried adding it to the manifest of the main jar, still same luck
20130930 13:04:35 <cryobacterium> you mean, in the class-path
20130930 13:04:56 <xranby> indeed
20130930 13:07:41 <cryobacterium> xranby: even so, I get exactly the same behaviour...I
20130930 13:08:23 <cryobacterium> xranby: I'll have to try it again a bit later, I'll be afk for an hour or so...but thanks for all the support!
20130930 13:08:52 <xranby> cryobacterium: thank you for publishing the code, its easier to understand when i have the issue in my face :)
20130930 13:09:15 <cryobacterium> xranby: yeah, I completely understand! thanks for the great support! :)
20130930 13:43:19 <xranby> cryobacterium: please help investigating run java -Djogamp.debug=all -jar cuttlefish.jar it will debug the JNILibLoaderBase class that do the actual loading
20130930 13:47:07 <xranby> sgothel: can you explain this? https://gist.github.com/xranby/a95192dd6c3d7c35447a#file-java-djogamp-debug-all-jar-cuttlefish-jar JNILibLoaderBase: loaded nativewindow_x11 yet i still run into the Exception in thread "main" java.lang.UnsatisfiedLinkError: jogamp.nativewindow.x11.X11Lib.QueryExtension0(JLjava/lang/String;)Z
20130930 13:50:25 <xranby> sgothel: cryobacterium's http://sourceforge.net/p/cuttlefish/code/HEAD/tree/ src archive is a good reproducer for this bug, on my machine it triggers early in the code during GLProfile.getDefault() which should imho work
20130930 13:55:56 <xranby> cryobacterium: i am not sure why your machine fail to load nativewindow_awt , at least that library loaded fine here on my machine
20130930 13:59:24 <xranby> cryobacterium: actually.. if i use jamvm, java -jamvm ,... i also fail to load nativewindow_awt like on your machine
20130930 14:20:16 <cryobacterium> xranby: I ran cuttlefish with all debug on, here's the output https://dl.dropboxusercontent.com/u/4500209/jogamp_dbg_all.log
20130930 14:20:34 <cryobacterium> also, I'm not using jamvm (at least not to my knowledge!)
20130930 14:29:47 <xranby> the obvious error IMHO is TempJarCache: null
20130930 14:30:08 <xranby> if i understand this correctly this prevents loading the library from the unpacked location
20130930 14:30:32 <xranby> instead it sometime loads using "mode 3"
20130930 14:31:00 <cryobacterium> what is mode 3? I've seen mode 4 used in the libraries loaded propery
20130930 14:31:03 <cryobacterium> what is mode 3? I've seen mode 4 used in the libraries loaded properly
20130930 14:32:31 <xranby> gluegen src/java/com/jogamp/common/jvm/JNILibLoaderBase.java
20130930 14:33:33 <xranby> http://jogamp.org/git/?p=gluegen.git;a=blob;f=src/java/com/jogamp/common/jvm/JNILibLoaderBase.java;h=caa8702336d3625c8c66d632dc7a45846a02946c;hb=HEAD#l458
20130930 14:34:22 <xranby> this is the code for JNILibLoaderBase.loadLibraryInternal mode 3 is System.loadLibrary( name )
20130930 14:34:40 <xranby> mode 2 is System.load( TempJarCache )
20130930 14:34:55 <xranby> thus mode 2 is somehow broken
20130930 14:35:19 <cryobacterium> yes I see this now
20130930 14:36:10 <cryobacterium> thanks for pointing it out...however, is this dependent on my system? were you able to run cuttlefish normally on yours?
20130930 14:37:11 <xranby> its broken on my system as well. i have added all java -jar logs using the hotspot server and jamvm jvm here: https://gist.github.com/xranby/a95192dd6c3d7c35447a
20130930 14:37:33 <xranby> when i use jamvm i hit the same exception like on your system
20130930 14:38:37 <xranby> https://gist.github.com/xranby/a95192dd6c3d7c35447a/raw/3153f87f429d701f3b07f97eceda1a9625c909d2/java+-jamvm+-Djogamp.debug%3Dall+-jar+cuttlefish.jar+.txt
20130930 14:47:59 <cryobacterium> hmm I see, it is the same
20130930 14:50:00 <xranby> cryobacterium: on the same machine.. if i instead use hotspot, both nativewindow_awt and nativewindow_x11 is loaded using mode 3 https://gist.github.com/xranby/a95192dd6c3d7c35447a/raw/112257a06df5e22c652457d3d98c1096f71aca33/java+-Djogamp.debug%3Dall+-jar+cuttlefish.jar+.txt
20130930 14:50:12 <xranby> and then later run into a new issue
20130930 14:51:56 <xranby> mode 3 is actually prefferd since then jogl do not need to unpack the library into a tmp dir, instead it loads the library directly from inside the jar
20130930 14:53:14 <xranby> the differences between you and my system are, you are using IcedTea6 1.12.6
20130930 14:53:42 <xranby> I am using IcedTea 2.3.10
20130930 14:54:33 <xranby> and we still do not know the root cause of the issue
20130930 15:01:15 <cryobacterium> I'll consider your comments on this, I'll try to dig a bit deeper
20130930 15:01:48 <cryobacterium> I see what you mean for the modes though
20130930 15:08:32 <cryobacterium> xranby: in need of a imminent solution, I'll try extracting the required natives from within the source code, in order to have a single jar, until I (or somebody) find a solution for this proble
20130930 15:08:36 <cryobacterium> *problem
20130930 16:02:02 * monsieur_max (~maxime@anon) Quit (Quit: Leaving.)
20130930 16:18:55 * cryobacterium (~ilias@anon) Quit (Quit: Bye all, thanks for helping!)
20130930 17:12:52 * monsieur_max (~maxime@anon) has joined #jogamp
20130930 17:37:14 * cryobacterium (~ilias@anon) has joined #jogamp
20130930 17:49:39 * cryobacterium (~ilias@anon) Quit (Quit: Leaving)
20130930 18:33:20 * xranby_ac100 (~xranby@anon) has joined #jogamp
20130930 19:59:25 * void256 (~chatzilla@anon) has joined #jogamp
20130930 20:03:42 * void256 (~chatzilla@anon) Quit (Ping timeout: 240 seconds)
20130930 21:02:55 * monsieur_max (~maxime@anon) Quit (Quit: Leaving.)
20130930 21:37:57 <xranby_ac100> sgothel: when using extract all classes into one jar + -natives-os.arch.jar then TempJarCache only extracts the gluegen-jar natives from the native jar during bootstrapNativeLib()
20130930 21:42:16 <xranby_ac100> i have a feeling that a TempJarCache.addNativeLibs row is missing, please suggest where ... perhaps JNILibLoaderBase.addNativeJarLibs0 ? , that extracts all other natives into the TempJarCache before JNILibLoaderBase tries to load them
20130930 21:42:54 <xranby_ac100> in order for JNILibLoaderBase to be able to find librarys using "mode - 2"
20130930 21:43:53 <xranby_ac100> logs is added in the backlog
20130930 21:43:57 <xranby_ac100> logs are
20130930 21:49:38 <xranby_ac100> hmm.. never mind.. already added in addNativeJarLibsImpl
20130930 21:53:15 <xranby_ac100> ahh... TempJarCache.addNativeLibs will skip loading libs due to the LoadState
20130930 21:53:46 <xranby_ac100> (already loaded by gluegen-rt during bootstrapNativeLib)
20130930 21:56:06 <xranby_ac100> ok i will remove the nativeLibJars.put(jarURI, LoadState.LOOKED_UP); line from bootstrapNativeLib and expect this bug to be fixed
20130930 22:29:28 <xranby_ac100> i need to add change the LoadState enum compilesWith test as well to allow the test to fail when LOOKED_UP != LOADED
20130930 22:37:55 <xranby_ac100> this patch fixed the issue https://gist.github.com/xranby/a95192dd6c3d7c35447a#file-oneclassesjar-nativesosarchjar-patch
20130930 22:39:59 <xranby_ac100> allows more native jars to be extracted into the TempJarCache after the gluegen-rt native have been extracted during the bootstrap process.
20130930 22:40:15 <xranby_ac100> in situations where all natives are stored inside one jar
20131001 04:23:46 <sgothel> I am not so sure about 'LOOKED_UP != LOADED' failure, but removing 'nativeLibJars.put(jarURI, LoadState.LOOKED_UP);' in bootstrapping is a remedy I can see ..
20131001 04:25:31 <sgothel> IMHO a proper patch would be to allow bootstrapping .. to behave like addNativeLibs(..) .. which removes the 'extra semantics' and gives full functionality
20131001 04:33:48 <sgothel> I will check the dependency problems .. i.e. remove Platform <-> JarUtil bootstrap dependencies .. I guess that would be a much cleaner solution
20131001 05:05:30 -jogamp- Continue @ http://jogamp.org/log/irc/jogamp_20131001050530.html