Bug 741

Summary: Enable Hi-Dpi Mode on OSX (Retina)
Product: [JogAmp] Jogl Reporter: ac <andres.colubri>
Component: macosxAssignee: Sven Gothel <sgothel>
Status: RESOLVED FIXED    
Severity: normal CC: askinner, gouessej, harvey.harrison, org.jogamp, rami.santina, wwalker3
Priority: ---    
Version: 2   
Hardware: pc_x86_64   
OS: macosx   
Type: --- SCM Refs:
83be0fa0ffe96c1acbc232965c2c03f40768184d 0ffba122ea5c4b8cc247234ca9f48ccfcce833cd f9a00b91dcd146c72a50237b62270f33bd0da98e fb57c652fee6be133990cd7afbbd2fdfc084afaa 98ed02cdb7b325d8afde596a5ef04f97be2018d4 56d60b36798fa8dae48bf2aa5e2de6f3178ab0d1 d149c554b453c86a880a4f0595bb8e340c80d041 bcda2dad1a6569ffd4eba07b231d50fdafc60b7f 8b255eb303bba045b4eb087da1d1cb33b2e89e96 fcd59768d776e202d3b03b7a9fee2aac69b3b663 f9ce025372b0b1cb9b8db78a0d4294861172bc1e ebe980ad6ac40148bc84913d1ba1f7adf6200490 2571ed0b5ef14155d204540d38b564a7d4cd47b6 58153310faa4089417037e67e06c0812908cecd2 4686a652d821efe04045333026be79270bc19bfd
Workaround: ---
Bug Depends on:    
Bug Blocks: 1373    

Description ac 2013-05-29 17:50:49 CEST
In Hi-Dpi mode, the actual high resolution is only used by the OpenGL context, but not by the windowing API (Cocoa/Quartz, etc), which remain at the original non-retina pixel size.

Relevant discussion about this issue available here:

http://forum.jogamp.org/Retina-support-in-OSX-td4029248.html#a4029251
Comment 1 Sven Gothel 2014-05-20 20:54:40 CEST
High-DPI (HiDPI) as understood by OSX:
  - using a different DPI value for window- and surface/GL coordinates,
    i.e. native display DPI is higher than used for window coordinates

  - hence window-dim != pixel-dim
    (dim == dimension)

  - may use special API workarounds to achieve native display DPI,
    otherwise OS specific up-scaling is performed (-> blurry)

Analysis of other OS/UI, e.g. Windows, X11, .. 
shows that such a 'DPI separation' (window/surface) does not exists (?).

To support HiDPI thoroughly in JOGL (NativeWindow, JOGL, NEWT)
we need to separate window- and surface dimension (dim),
i.e. NativeWindow and NativeSurface must have distinguished
access methods for window-dim and surface-dim (or pixel-dim).
  NativeWindow:
      - get[Win]Width()
      - get[Win]Height()

  NativeSurface
      - getPix[el]Width()
      - getPix[el]Height()

The final method names have to be determined.

++++

Distinguishing window-dim and pixel-dim using 
unambiguous method-names will also solve 
currently existing API collisions of
  [1] NativeSurface's and GLDrawable's getWidth()/getHeight()
with e.g.
  [2] AWT's getWidth()/getHeight().

W/o HiDPI support these collisions are not crucial
since the semantics were the same.

With HiDPI semantics changed, i.e.
   AWT getWidth() == NativeWindow getWinWidth(), ..
   AWT getWidth() * pixelScale == NativeSurface getPixWidth(), ..

where pixelScale := getPixWidth() / getWinWidth()

....

While changing a most common core JOGL API's method name
cannot be considered a good thing, 
solving this API collision would further API stability 
for possible future use-cases .. while solving HiDPI.

Status of API Change: TBD
Comment 2 Sven Gothel 2014-05-20 21:01:25 CEST
- We also have to re-validating AWTPrintLifecycle's DPI semantics,
  since we currently are based on pixel dimension w/ 72 dpi!
Comment 3 Sven Gothel 2014-05-20 21:14:00 CEST
going ahead w/ API change proposal in branch 'bug742_hidpi': getPixelWidth() and getWindowWidth() ..
Comment 4 Sven Gothel 2014-05-20 21:15:20 CEST
(In reply to comment #3)
> going ahead w/ API change proposal in branch 'bug742_hidpi': getPixelWidth()
> and getWindowWidth() ..

branch: bug741_hidpi
Comment 5 Sven Gothel 2014-05-21 09:03:13 CEST
Branch bug741_hidpi pushed to repo:
  http://jogamp.org/git/?p=jogl.git;a=shortlog;h=refs/heads/bug741_hidpi

+++

83be0fa0ffe96c1acbc232965c2c03f40768184d
  Add access to private HiDPI in AWT pixelScale value in JAWTUtil and JAWTWindow

0ffba122ea5c4b8cc247234ca9f48ccfcce833cd
  Add prelim HiDPI support to GLJPanel w/o API change 
  and w/o fixing AWTPrintLifecycle DPI evaluation

+++

f9a00b91dcd146c72a50237b62270f33bd0da98e

- Distinguish window-units and pixel-units; 
- Add HiDPI for AWT GLCanvas w/ OSX CALayer

Core API Change:

To support HiDPI thoroughly in JOGL (NativeWindow, JOGL, NEWT)
we need to separate window- and pixel units.

NativeWindow and NativeSurface now have distinguished
access methods for window units and pixel units.

  NativeWindow: Using window units
      - getWindowWidth()    * NEW Method *
      - getWindowHeight()   * NEW Method *
      - getX(), getY(), ...

  NativeSurface: Using pixel units
      - getWidth()  -> getSurfaceWidth()   * RENAMED *
      - getHeight() -> getSurfaceHeight()  * RENAMED *

  GLDrawable:  Using pixel units
      - getWidth()  -> getSurfaceWidth()   * RENAMED, aligned w/ NativeSurface *
      - getHeight() -> getSurfaceHeight()  * RENAMED, aligned w/ NativeSurface *

Above changes also removes API collision w/ other windowing TK,
e.g. AWT's getWidth()/getHeight() in GLCanvas
and the same method names in GLDrawable before this change.

+++

Now preliminary 'working':
  - AWT GLCanvas
  - AWT GLJPanel

Tested manually on OSX w/ and w/o HiDPI Retina:
  java com.jogamp.opengl.test.junit.jogl.demos.es2.awt.TestGearsES2AWT -manual -noanim -time 1000000
  java com.jogamp.opengl.test.junit.jogl.demos.es2.awt.TestGearsES2GLJPanelAWT -manual -noanim -time 1000000

+++

TODO:
  - NEWT
    - Change Window.setSize(..) to use pixel units ?
    - OSX HiDPI support

  - Testing ..

  - API refinement
Comment 6 Julien Gouesse 2014-05-21 13:12:41 CEST
What does GLCanvas.getWidth() return now? How should I modify any existing source code calling this method?
Comment 7 Sven Gothel 2014-05-21 18:38:42 CEST
(In reply to comment #6)
> What does GLCanvas.getWidth() return now? How should I modify any existing
> source code calling this method?

After the change, GLCanvas.getWidth() no more overrides 
GLDrawable's getWidth() interface.

Hence GLCanvas.getWidth() will be solely in AWT's domain,
i.e. returns the AWT components width (in window units on OSX).

To use the pixel units GLCanvas.getSurfaceWidth() has to be used,
which uses [GLDrawable/NativeSurface].getSurfaceWidth() semantics.
Comment 8 Sven Gothel 2014-05-21 18:43:35 CEST
We could now rename 

  NativeWindow.getWindowWidth() -> NativeWindow.getWidth() 

w/o having a semantic API collision.

The impact of reintroducing 'getWidth()/getHeight()'
to NativeWindow and hence NEWT Window 
would allow NEWT Window users to have source compatibility.

I am not sure whether this is a good thing, since the
semantics have changed in case 'window units' are different of 'pixel units'.

I.e. w/o this source compatibility the user would need to change all their
getWidth()/getHeight() usage and *think*.

On the other hand, the simple method names would be semantically compatible
with other TKs .. and w/ NativeWindow's getX()/getY() w/o 'Window' identifier
in it's method names.

Hence I would vote for the simpler name.
Comment 9 Sven Gothel 2014-05-26 19:26:04 CEST
fb57c652fee6be133990cd7afbbd2fdfc084afaa

Bug 742 HiDPI: [Core API Change] Distinguish window-units and pixel-units: Refine commit f9a00b91dcd146c72a50237b62270f33bd0da98e

- Using comment tag 'FIXME HiDPI' to locate remaining issues

- Fix remaining 'getPixel*(..)' -> 'getSurface*(..)'
  - UpstreamSurfaceHook

- Fix usage (one by one) of
  - NativeWindow: getWindowWidth() / getWindowHeight()
  - NativeSurface/GLDrawable: getSurfaceWidth() / getSurfaceHeight()

- mention window- or pixel units in API doc where required

- use 'setSurfaceSize(..)' where appropriate to match 'getSurface*()'
  - GLFBODrawable
  - GLOffscreenAutoDrawable
  - UpstreamSurfaceHook.MutableSize

- NativeWindow's Point: Add API doc and 'Point scaleInv(..)'

- NativeSurface
  Simplify new conversion methods and use single in-place storage
  - 'int[] getWindowUnitXY(int[], int[])' -> 'int[] convertToWindowUnits(int[], int[])'
  - 'int[] getPixelUnitXY(int[], int[])' -> 'int[] convertToPixelUnits(int[], int[])'

- NEWT Screen/Monitor
  - Assume screen/window units
  - TODO: Refine semantics - Monitor resolution probably is in pixel units ?!
    - Including the Rectangle/Monitor association etc etc

- NEWT Window
  - Add setSurfaceSize(..) for convenience
  - Add 'Point convertToWindowUnits(final Point pixelUnitsAndResult)', etc ..
  - All window ops are using window units (size, pos, ..),
    but methods operating on the surface/drawable: windowRepaint(..) ..

- TODO: Consider changing method names 'window*(..)' to 'surface*(..)'
  actually operating on surface/drawable
  - Window.windowRepaint(..)
  - GLAutoDrawableDelegate.windowResizedOp(..) (maybe all similar methods in here)

- NEWT Mouse/Pointer Events
  - Using pixel units
Comment 10 Sven Gothel 2014-05-26 19:26:37 CEST
commit 98ed02cdb7b325d8afde596a5ef04f97be2018d4

Bug 742 HiDPI: [Core API Change] Distinguish window-units and pixel-units: Refine commit fb57c652fee6be133990cd7afbbd2fdfc084afaa

- NEWT Screen, Monitor, MonitorMode, ..
  - All Units are in pixel units, not window units!

  - On OSX HiDPI, we report the current scaled monitor resolution,
    instead of the native pixel sized.
    Need to filter out those, i.e. report only native unscaled resolutions,
    since out MonitorMode analogy is per MonitorDevice and not per window!

- Fix usage (one by one) of
   - Screen and Monitor viewport usage
Comment 11 Sven Gothel 2014-05-26 19:28:15 CEST
56d60b36798fa8dae48bf2aa5e2de6f3178ab0d1

Bug 741 HiDPI: Refine Monitor/Screen [virtual] Viewport Definition / Add NEWT Support / Fix JAWT getPixelScale deadlock

- NativeWindow/Surface/NEWT API DOC: Define Coordinate System of Window and Screen

- OSXUtil: Add getPixelScale(..) via Screen index and 'windowOrView'

- JAWTWindow/JAWTUtil.getPixelScale(..): Use pre-fetched AWT GraphicsConfiguration to solve AWT-TreeLock (deadlock)

- [Virtual] Viewport of MonitorDevice and Screen:
  - Properly calculate and expose [virtual] viewport in window and pixel units
    - OSX Monitor viewports in pixel units are 'reconstructed'
  - Window/Viewport to Monitor selection shall be perfomed via window units (unique)

- OSX NEWT Window create/init (native): Use given size and coordinates even in fullscreen mode
  Don't override by quering NSScreen coordinates, trust given values.

- Fix test cases, i.e. usage of pixel- and window-units
Comment 12 Sven Gothel 2014-05-27 00:06:11 CEST
commit d149c554b453c86a880a4f0595bb8e340c80d041

Bug 741 HiDPI: [Core API Change] Bring back get[Width|Height]() in NativeWindow, i.e. getWindow[Width|Height]() ->  get[Width|Height]()
    
We have distinguished pixel- and window units in commit f9a00b91dcd146c72a50237b62270f33bd0da98e
and introduced NativeWindow.getWindow[Width|Height]() and NativeSurface.getSurface[Width|Height]().
    
To have a unique naming scheme, we could rename all method using 'Window',
but for simplicity and since there will be no 'semantic override'
just use the simple version.
Comment 13 Sven Gothel 2014-05-27 00:08:56 CEST
Merged branch bug741_hidpi -> master and pushed to all repos.
Comment 14 Sven Gothel 2014-05-27 00:11:31 CEST
Ran all unit tests manually on OSX w/ HiDPI and
GNU/Linux w/ AMD Radeon (OSS driver) - no regressions.

W/ all other bugfixes Bug 1009, Bug 1010, Bug 1012 and Bug 1013
unit tests are stable on 
  - AMD Radeon (OSS driver)
  - NVIDIA 337.19 (w/ EGL ES 3.1)
Comment 15 Sven Gothel 2014-05-27 00:21:34 CEST
TODO with AWT:
   Update scale if moving from monitor w/ HiDPI -> non HiDPI monitor

Sadly the current method of querying via AWT doesn't work here.
Comment 16 Sven Gothel 2014-05-27 11:02:36 CEST
commit bcda2dad1a6569ffd4eba07b231d50fdafc60b7f

 Bug 741 HiDPI: Fix regression MIN_MONITOR_DEVICE_PROPERTIES: Adding missing 'Rotated Viewport window-units' / Refine API doc in MonitorModeProps
    
 Regression of commit 56d60b36798fa8dae48bf2aa5e2de6f3178ab0d1

+++

JOGL Build 1253:
 - https://jogamp.org/chuck/view/fwd/job/jogl/1253/

Aggregations:
 - http://jogamp.org/deployment/archive/master/gluegen_789-joal_534-jogl_1253-jocl_961
 - http://jogamp.org/deployment/archive/master/gluegen_789-joal_534-jogl_1253-jocl_961-signed/
Comment 17 Sven Gothel 2014-05-27 16:47:28 CEST
Add setting/toggling HiDPI, i.e. diff DPI for window- and surface
per GLAutoDrawable upstream component:
  - GLCanvas, GLJPanel, GLWindow, ..

Add  interface and let it be impl. by above classes 
exposing setting a desired pixelScale for x- and y-component.

Magic numbers for desired pixelScale:
  - 1: Same DPI for window and surface
  - 0: Native Surface and Window DPI (usually maximum)
Comment 18 Sven Gothel 2014-05-27 23:50:21 CEST
commit 8b255eb303bba045b4eb087da1d1cb33b2e89e96

Add missing window -> pixel unit conversion in AWTNewtEventFactory (e.g. for NewtCanvasAWT)

+++

commit fcd59768d776e202d3b03b7a9fee2aac69b3b663

Fix OSX NEWT Offscreen Size Regression from commit 56d60b36798fa8dae48bf2aa5e2de6f3178ab0d1
    
Fix regression of commit 56d60b36798fa8dae48bf2aa5e2de6f3178ab0d1:
  createWindow(..) was issuing sizeChanged(..) to ensure size notification,
  however - the offscreen case used the dummy size 64x64.
    
  Fix issues the notifications in caller w/ true size.
Comment 19 Sven Gothel 2014-06-06 13:07:06 CEST
commit f9ce025372b0b1cb9b8db78a0d4294861172bc1e

  - Fix missing window -> pixel unit conversion in AWTNewtEventFactory 
    of commit 8b255eb303bba045b4eb087da1d1cb33b2e89e96

commit ebe980ad6ac40148bc84913d1ba1f7adf6200490

  - Add new NativeSurfaceHolder interface to GLDrawable and NativeWindow;
  - [AWT|SWT]NewtEventFactory use NativeSurfaceHolder as source, 
    fixes pixel unit conversion
Comment 20 Sven Gothel 2014-06-08 16:40:06 CEST
commit 2571ed0b5ef14155d204540d38b564a7d4cd47b6

Add ScalableSurface interface to get/set pixelScale w/ full OSX impl.

Add ScalableSurface interface
  - To set pixelScale before and after realization
  - To get pixelScale

  - Implemented on:
    - NEWT Window
      - Generic impl. in WindowImpl
      - OSX WindowDriver impl.
        - Also propagetes pixelScale to parent JAWTWindow if offscreen (NewtCanvasAWT)
      - AWT WindowDriver impl.

    - JAWTWindow / OSXCalayer
      - AWT GLCanvas
      - AWT GLJPanel
      - NEWTCanvasAWT:
        - Propagates NEWT Window's pixelScale to underlying JAWTWindow

    - WrappedSurface for pixelScale propagation
      using offscreen drawables, i.e. GLJPanel

    - Generic helper in SurfaceScaleUtils (nativewindow package)

  - Fully implemented on OSX

  - Capable to switch pixelScale before realization,
    i.e. native-creation, as well as on-the-fly.

  - Impl. uses int[2] for pixelScale to support
    non-uniform scale.

Test cases:
  - com.jogamp.opengl.test.junit.jogl.demos.es2.newt.TestGearsES2NEWT
  - com.jogamp.opengl.test.junit.jogl.demos.es2.awt.TestGearsES2AWT
  - com.jogamp.opengl.test.junit.jogl.demos.es2.awt.TestGearsES2GLJPanelAWT
  - com.jogamp.opengl.test.junit.jogl.demos.es2.newt.TestGearsES2NewtCanvasAWT

  - Press 'x' to toggle HiDPI
  - Commandline '-pixelScale <value>'
  - Added basic auto unit test (setting pre-realization)
Comment 21 Sven Gothel 2014-06-08 16:40:43 CEST
commit 3fb76fcef1e6dd552ec0f677af67baf3186a1434

Simplify ScalableSurface [set|get]SurfaceScale(..) spec, 
which also fixed JAWTWindow getSurfaceScale() issue on Windows
    
Let setSurfaceScale(..) return the validated requested values
and getSurfaceScale(..) always the current values.
    
This removes complication and solves a bug w/ JAWTWindow on Windows,
where we used 'drawable' as an indicator for 'previous locked' state.
The latter is not true since on Windows 'drawable' is set to null in unlock,
getWindowHandle() should be taken instead.
Comment 22 ac 2014-06-08 17:05:01 CEST
This is great, thanks! Which stable release will include the fix?
Comment 23 Sven Gothel 2014-06-08 18:10:42 CEST
commit 58153310faa4089417037e67e06c0812908cecd2

Simplify ScalableSurface (2): Add request pixelScale API entry, fixed NewtCanvasAWT use-case
    
We require the requested pixelScale in NewtCanvasAWT if the NEWT window (child)
is not yet realized, so the JAWTWindow can receive the request,
since realized/current pixelScale is still 1.
    
Remove return value (requested pixel scale):
  -  public int[] setSurfaceScale(final int[] result, final int[] pixelScale);
  +  public void setSurfaceScale(final int[] pixelScale);
    
Add API hook to query requested pixel scale:
  +  int[] getRequestedSurfaceScale(final int[] result);
    
Unique name for get[Current]*:
  -  public int[] getSurfaceScale(final int[] result);
  +  public int[] getCurrentSurfaceScale(final int[] result);
Comment 24 Sven Gothel 2014-06-08 18:11:44 CEST
(In reply to comment #22)
> This is great, thanks! Which stable release will include the fix?

Release 2.2.0

<https://jogamp.org/wiki/index.php/SW_Tracking_Report_Objectives_for_the_release_2.2.0>

But I will add aggregated test builds ..
since we have a few API changes especially for this feature.
Comment 25 Sven Gothel 2014-06-09 04:00:12 CEST
commit 4686a652d821efe04045333026be79270bc19bfd

Add ScalableSurface.getNativeSurfaceScale(..) to compute surface DPI ; Add NEWT Window.getPixelsPerMM(..) to query surface DPI
    
With HiDPI and surface scale, we need knowledge of the native surface's
pixel-scale matching the monitor's pixel-per-millimeter value.
    
Preserving the queried native pixel-scale and exposing it via
ScalableSurface.getNativeSurfaceScale(..) to compute surface DPI.
    
Add NEWT Window.getPixelsPerMM(..) to query surface DPI.
    
Surface DPI is demonstrated in GraphUI's GPUUISceneGLListener0A .. and TestRulerNEWT01, etc ..