(Continued from Bug 1404.) Add an "Automatic-Module-Name" entry to the manifest of each jar, to avoid issues when the jars are on the Java 11+ modulepath. Does not affect Java 8 compatibility. ============= WHY DO THIS? ============= When a jar on the Java 11+ modulepath does not specify its module name, Java uses the jar's filename to make up a module name. To declare a dependency on the jar, a dependent module must use the filename-based identifier ... which means the dependency will break if the jar file gets renamed. Even worse, Java's heuristic for making up module names can result in collisions between jars. When the 2.4.0 release jars are eventually deployed to Maven Central, they will face this problem if they use the same Maven-module hierarchy that previous releases have used -- specifically, if "natives" jars are deployed using the same artifactId and different classifiers. For example, the core JOGL library ends up with the following filenames in Maven (see https://repo1.maven.org/maven2/org/jogamp/jogl/jogl-all/2.3.2/): * jogl-all-X.X.X.jar * jogl-all-X.X.X-natives-linux-amd64.jar * jogl-all-X.X.X-natives-windows-amd64.jar * etc. When Java generates filename-based module names, those jars all end up with the same module name: "jogl.all". (This is not a problem for the "fat" jar, which contains both classes and natives in a single jar. But it would be a problem for the separate jars -- jogl-all, gluegen-rt, etc.) The simplest way to address these problems is to add an "Automatic-Module-Name" line to each jar's manifest. No other changes are required. ========= WHY NOT? ========= The only reason NOT to do this is if the jars aren't intended to work when they are on the modulepath. Based on Bug 1404, I assume (and hope) that these jars ARE intended to work either way -- on classpath or on modulepath. (It's annoying to have to test a library twice -- once on classpath, once on modulepath -- but it's apparently unavoidable if you want Java 11+ compatibility. Discussed here: https://blog.joda.org/search/label/modules.) ==================== REMAINING QUESTIONS ==================== What should the module names be? ... I will do some reading about how modules should be named, and then post my suggestions here.
I can try to post a patch, but it would take me a while to get to it. I would have to figure out the build setup, so for me this wouldn't be just a quick task.
There are two schools of thought about JPMS module names: short names that look like jar filenames (e.g. "jogl"), and fully qualified names that look like java-package names (e.g. "com.jogamp.opengl"). Fully qualified names could go: * com.jogamp.gluegen * com.jogamp.opengl * com.jogamp.opencl * com.jogamp.openal * com.jogamp.gluegen.natives.linux.amd64 * com.jogamp.opengl.natives.linux.amd64 * com.jogamp.opencl.natives.linux.amd64 * com.jogamp.openal.natives.linux.amd64 * ... and so on for various OS/arch combinations Alternatively, short names could go: * gluegen * jogl * jocl * joal * gluegen.natives.linux.amd64 * jogl.natives.linux.amd64 * jocl.natives.linux.amd64 * joal.natives.linux.amd64 * ... and so on for various OS/arch combinations Stephen Colebourne makes a good case for fully qualified module names here: https://blog.joda.org/2017/04/java-se-9-jpms-module-naming.html
How is "jogl-all.jar" related to the jars in the "atomic" subdirectory ("newt-awt.jar", "newt-driver-linux.jar", etc.)? Is "jogl-all.jar" just the aggregation of a bunch of smaller jars? That Colebourne link says something about what he calls "all" jars, and how problematic they are in Java 9+. It just dawned on me what he means ... so I'm trying to understand how all the various Jogamp jars (and there are a lot of them!) are related to each other. "jogamp-fat.jar" is pretty obvious. But I'm not sure how the jars in the "atomic" subdir (e.g. "newt-awt.jar") relate to the ones in the parent dir (e.g. "jogl-all.jar"): https://jogamp.org/deployment/v2.4.0-rc-20200202/jar/atomic/ Do the jars under "atomic" get aggregated into "jogl-all.jar", or something?
After giving it some thought, I believe the best thing to do is: 1. Identify a set of jars with mutually disjoint contents -- jars that don't "overlap" each other. (Empty packages are okay -- even empty parent packages. But no collisions between non-empty packages.) 2. Document that only those non-overlapping jars should be used as jpms-module dependencies. (That is, anyone writing a "module-info.java" should AVOID doing "requires jogamp.fat" or "requires jogl.all".) 3. Add "Automatic-Module-Name" to the non-overlapping jars only, because those are the only ones that should be depended upon as jpms-modules. So the question is: Q: What is the set of non-overlapping jars that, together, contain all the Jogamp classes? I can do some digging to try to figure this out. But -- Sven or others -- if you know off the top of your head, please let me know. It might be something simple like: all the jars under the "atomic" subdir.
There doesn't seem to be a set of non-overlapping jars that, together, contain all classes. In the "atomic" subdir: * jogl,gluegen-gl have identical copies of com/jogamp/gluegen/runtime/opengl * newt-event,newt have identical copies of com/jogamp/newt/event * newt-event,newt-awt have identical copies of com/jogamp/newt/event/awt * newt-event contains part of newt and part of newt-awt * jogl,jogl-awt,jogl-util have com/jogamp/opengl/util with different contents In the top-level dir: * Each jogl jar has 4 variants: desktop, noawt, android, mobile * gluegen-rt is a subset of gluegen
The jars in the "atomic" subdir have some overlap, so they can't be used as modules without some reorganization. But the jars in the top-level directory should actually work fine. In gluegen.jar, gluegen-rt.jar: Automatic-Module-Name: com.jogamp.gluegen In jogl-all, jogl-all-noawt, jogl-all-android, jogl-all-mobile: Automatic-Module-Name: com.jogamp.opengl In jocl.jar: Automatic-Module-Name: com.jogamp.opencl In joal.jar: Automatic-Module-Name: com.jogamp.openal Note that gluegen.jar and gluegen-rt.jar get the same module name, which means they can't both be on the modulepath at runtime. Same with the jogl-all variants. But that's fine; seems like that's the intended usage anyway. (And, critically, the application developer gets to decide which variant to use -- the decision is not compiled into the "module-info.class" of an intermediate dependency jar.)
The "main" jars don't need "Automatic-Module-Name", because they aren't used as modules. (In fact, they are empty jars, and aren't really used at all.) The "natives" jars don't need "Automatic-Module-Name", because they are accessed through the filesystem, rather than through the classloader. The "fat" jar doesn't need "Automatic-Module-Name", because it shouldn't be used as a module. (Conceptually, it contains multiple modules ... but JPMS doesn't support multi-module jars. Argh!) I can describe why it shouldn't be used as a jar in more detail if that would be helpful. The "atomic" jars don't need "Automatic-Module-Name", because they can't really be used as modules, due to package-name collisions among various jars. (This could be addressed by reorganizing the jars a little bit -- but IIUC the current goal is to reach Java 11+ compatibility without disruptive changes.) So my recommendation is to add the manifest entries listed in my previous post, and leave it at that. Does that sound reasonable?
Thank you very much Mike for your detailed elaboration and proposal! I need to read through all your details within the next days with more attention. I glanced over it a little already and I am already impressed how thorough you analyzed the current state of things. Thank you!
(In reply to Mike Hogye from comment #0) > ========= > WHY NOT? > ========= > > The only reason NOT to do this is if the jars aren't intended to work when they > are on the modulepath. Based on Bug 1404, I assume (and hope) that these jars > ARE intended to work either way... Before I get an opinion on the naming scheme of things, let us debate whether it is technically feasible. (First of all, I like the overall idea of course). Usually, as you have well seen, I modularized the jar files already for certain purposes and sub-sections. JOGL is generally split into Nativewindow, JOGL and NEWT. However, we do depend on other modules as well. In case of AWT, we sadly require using certain not-exported symbols. These were traditionally accessible and widely used for native bindings. Sadly a few symbols haven't been exported in the new module scheme. I have experienced that we can't get access to these symbols, if we add ourselves or them (awt, etc) to the module path. Using the classpath scheme still allows using these symbols. For this purpose I had to add the UnsafeUtil class in GlueGen, see commit 07c1885e9a3d1f3a3853414648c06fb3864bc69f <https://jogamp.org/cgit/gluegen.git/commit/?id=07c1885e9a3d1f3a3853414648c06fb3864bc69f> A simple call hierarchy of 'com.jogamp.common.util.UnsafeUtil.doWithoutIllegalAccessLogger(PrivilegedAction<T>)' shows where it is being used. If we cannot resolve this issue, at least we need to add a DISCLAIMER that these AWT related features are not available anymore. Personally I prefer NEWT anyways, however the above might be an issue.
(In reply to Sven Gothel from comment #9) Yes, we have similar issues with DirectBuffer, AWT tweaks, and memory-mapped files. Our workaround is to document that applications must pass several "--add-opens" args to the JVM, thereby making the relevant classes accessible to our library at runtime. Docs for --add-opens are here: https://docs.oracle.com/en/java/javase/11/migrate/index.html#JSMIG-GUID-12F945EB-71D6-46AF-8C3D-D354FD0B1781 We're currently using JOGL 2.4.0-rc-20200202 on the modulepath with these JVM args: --add-opens java.desktop/sun.awt=jogl.all --add-opens java.desktop/sun.java2d=jogl.all (And maybe we need a couple more, for the doWithoutIllegalAccessLogger() calls.) This is not ideal, and it's frustrating that it's necessary ... but in practice it's not too terrible. Occasionally we forget to add the JVM args to a run configuration, and end up with some sort of "X is not accessible" exception. After grumbling briefly, we open the doc page with the required args, copy the args, and paste them into the run configuration. It's an occasional inconvenience, rather than a showstopper.
(In reply to Mike Hogye from comment #10) sound very good, thank you for elaborating. will be earmarked, need time to push this in.
Maybe some information could be moved into module-info.java