Bug 1118 - Add all-natives.jar with fat-jar directory structure we can download using maven
Summary: Add all-natives.jar with fat-jar directory structure we can download using maven
Status: RESOLVED DUPLICATE of bug 1172
Alias: None
Product: General
Classification: JogAmp
Component: builds (show other bugs)
Version: 2.3.2
Hardware: All all
: P5 enhancement
Assignee: Sven Gothel
Depends on: 1172
  Show dependency treegraph
Reported: 2015-01-11 15:19 CET by Johan
Modified: 2015-09-27 03:14 CEST (History)
2 users (show)

See Also:
SCM Refs:
Workaround: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Johan 2015-01-11 15:19:10 CET
This feature request is based on the discussion with Goussej and the forum thread here: http://forum.jogamp.org/Help-How-to-Build-an-executable-JAR-with-SBT-JOGL-Scala-tt4033830.html 

I propose building and publishing the following Gluegen and JO[]L jar files:

That all have the internal directory structure like:

jogl-all-natives.jar would contain

This way JOGL/Gluegen would easily integrate with automatic fat-jar tools without any extra configuration. 
Example: I use SBT to build scala applications every day and would love to be able to build some scala OpenGL applications using JOGL. Fat jar creation with sbt assembly is done using the sbt assembly plugin (https://github.com/sbt/sbt-assembly/). 

Currently it's not possible to build fat jars with jogl and sbt - Having all-natives jars would most likely solve this. Then I'd just need to specify one more maven/sbt dependency i.e. org.jogamp:jogl-all-natives:2.2.4.
Comment 1 Johan 2015-01-11 15:20:27 CET
Here is the discussion IRC log with Goussej:
<GiGurra> Hi. I spoke to goussej on the forums (http://forum.jogamp.org/Help-How-to-Build-an-executable-JAR-with-SBT-JOGL-Scala-td4033830.html#none) about a jar loading/packaging problem and he told me to come here and see if anyone knew how to solve my problem
<GiGurra> In particular he told me to contact someone named Mark, not sure if Mark is here atm but I dont see his name on the right
* monsieur_max (~maxime@ip-46.net-81-220-153.grenoble.rev.numericable.fr) has joined
* gouessej (52794ad6@gateway/web/freenode/ip. has joined
<gouessej> Hi
<gouessej> GiGurra: Please read my reply
<gouessej> GiGurra: rmk0 knows Maven far better than me
<GiGurra> Thanks, I'll try to figure out some solution. Do you know if the question has come up before?
<GiGurra> Or are there too few people interested in both JOGL and scala
<gouessej> I don't think that the question has been asked before
<gouessej> Why not contacting the sbt maintainers to see what they suggest?
<GiGurra> Yes I'm thinking about that too
<GiGurra> but I'm not sure. Assembly is not part of sbt
<GiGurra> it's justa plugin
<GiGurra> sbt does not provide any ways for building an executable jar afaik, so you need to use one of the packaging tools
<GiGurra> First id need to figure out how the file structure inside my jar should work for JOGL to run. Then once I know that, Ican contact the sbt assembly, pack or one-jar people and see if they want to support it
* monsieur_max1 (~maxime@ip-46.net-81-220-153.grenoble.rev.numericable.fr) has joined
* monsieur_max has quit (Ping timeout: 264 seconds)
<gouessej> Then, you should look at the paragraph I quoted
<gouessej> It shows in which location you must put the native libraries to allow GlueGen to find them, extract them and load them
<GiGurra> yes, thats the page Ive been studying mroe hehe
<GiGurra> The simplest might be multi jar, even though it's discouraged
<gouessej> Then, what is wrong??? /natives/<os.and.arch>/
<gouessej> I have an idea...
<gouessej> This should help: http://forum.jogamp.org/Fat-jar-deployment-method-ant-recipe-tp4032213p4032229.html
<gouessej> Look at the prefixes
<gouessej> Now don't tell me that you don't know where to put the native libraries :)
<gouessej> https://jogamp.org/jogl/doc/deployment/JOGL-DEPLOYMENT.html#NativeJARFiles
<GiGurra> Goussej, yes thx. Thats not the problem. I know there are many places to put them. Here's the problem:
<GiGurra> I need to come up with a simple enough suggestion for proposing a new build setting to the assembly/pack/one-jar plugin createors, good enough for them to accept. Otherwise we'll just run into "you're doing it wrong" -> "no you are"
<GiGurra> maven/sbt/gradle, all being fairly high level declarative build tools compared to ant, you're not supposed to work on details in them
<GiGurra> so I may just end up with a custom script to build my jar file, and just use SBT for development
<gouessej> Gradle allows to call Ant
<gouessej> Maven too
<GiGurra> I'm sure it does that is not the point
<GiGurra> sbt can all any console command too
<GiGurra> I may just end up calling ant to produce my jar in the end
<GiGurra> just because I cannot get it to work with sbt for example
<gouessej> There is already a limitation with One-JAR
<GiGurra> but then id need to define the dependencies again... The point of sbt/maven/etc is to be able to define deps once and be done with it, let the tool figure out the rest :)
<GiGurra> yeah, I dont care about one-jar tbh
<GiGurra> getting assembly to work would be the best, and I think the subfolder naming scheme is the best solution for that
<gouessej> Do you think that you have a chance to make it work with assembly then?
<GiGurra> yes
<GiGurra> assembly just merges everything in the dir structure given by the original jars. I'll just need to have them implement some retarget-filter
<gouessej> Good idea
<gouessej> In my humble opinion, the only helpful thing that could be done in JogAmp consists in providing a JAR with all native libraries respecting the naming convention
<GiGurra> YES
<GiGurra> if that is possible
<GiGurra> :)
<GiGurra> that would be great. I have to leave now. Good talking to you, I'll leave irc on
<gouessej> It breaks nothing :) You can make a bug report and mark it as a RFE
<gouessej> Ok bye
<GiGurra> bye, thx
<gouessej> you're welcome
* gouessej has quit (Quit: Page closed)
Comment 2 Julien Gouesse 2015-01-12 10:27:54 CET
Writing an Ant task that merges the JARs containing the native libraries into a single JAR by copying all files in /natives/*/ is trivial but I don't know where and when it should be called as a machine is able to build only the native libraries of its platform whereas the other native libraries are necessary to create gluegen-rt-all-natives.jar, jogl-all-natives.jar, jocl-all-natives.jar and joal-all-natives.jar.
Comment 3 Sven Gothel 2015-01-27 10:55:29 CET
Currently we use Mark's scripting to produce maven artifacts.

Then .. we would need to have those all-natives etc in the first place,
for all jogamp modules.

I don't really see the benefit here,
as Julien mentioned - far-jar is supported but we don't produce 'em yet.

.. quite undecided.
Comment 4 Julien Gouesse 2015-01-27 11:27:57 CET
(In reply to comment #3)
> Currently we use Mark's scripting to produce maven artifacts.
> Then .. we would need to have those all-natives etc in the first place,
> for all jogamp modules.
> I don't really see the benefit here,
> as Julien mentioned - far-jar is supported but we don't produce 'em yet.
> .. quite undecided.

The problem is that some build tools aren't flexible enough (unlike Ant) to create those JARs easily. Johan's suggestion is the least invasive solution we found as a compromise to make it work with SBT. It doesn't require any change inside GlueGen.

Moreover, each time someone has to make a fat JAR, we have to show her/him the documentation. If we provided some *-all-natives JARs, we would just have to tell her/him to keep the same structure when merging the JARs. If the build tool is unable to do that, then we'll have to blame it ;)

Finally, it would help to obtain a single JAR (at least for desktop environments) for all JogAmp APIs, which are often used in academical contexts by newbies. I often have to enumerate tons of JARs in the tutorials and there is always someone who picks the wrong JARs for the native libraries.
Comment 5 Julien Gouesse 2015-03-31 16:57:13 CEST
Maybe this RFE is a better solution:
Comment 6 Xerxes Rånby 2015-08-28 01:40:41 CEST
We expect the implementation of Bug 1172 to resolved this bug all native jars uses the fat-jar internal layout. All plugins that combine jars into one should work. jogamp 2.3.2 will be the first release using the natives/os.and.arch internal jar layout.

*** This bug has been marked as a duplicate of bug 1172 ***