Bug 1422

Summary: NEWT: Handle SWT's DPI Scaling on non native dpi scaling platforms (Linux, Windows, ..)
Product: [JogAmp] Newt Reporter: Sven Gothel <sgothel>
Component: swtAssignee: Sven Gothel <sgothel>
Status: VERIFIED FIXED    
Severity: normal CC: marcel.au
Priority: P4    
Version: 2.4.1   
Hardware: All   
OS: all   
Type: DEFECT SCM Refs:
d671b2ab3badbcfdbe0ff57f55ff997ba7bcb060 78609202731252f0024e6330cc94c52b05c1d146 f63b94cccc71cf154a7a6d3359ceface3a683229
Workaround: ---
Bug Depends on: 1358, 1421    
Bug Blocks: 674, 1373, 1423    

Description Sven Gothel 2020-01-06 23:57:39 CET
Bug 1358 and Bug 1421 discuss the dpi scaling size (1358) 
in general and the positioning (1421) in particular for MacOS.

This bug shall discuss the mapping of SWT's dpi-scaling
to NEWT on platforms other than MacOS where it is not actually implemented
in the native toolkits: Windows and Linux GTK.
This issue is in particular for NewtCanvasSWT.

Both latter platforms may have a high level artificial dpi-scaling for 
so called unaware applications to map the 75 dpi on 150 dpi or higher.
However, this is not part of their native low level interfaces (X11 and GDI) 
as used by e.g. NEWT.

This bug entry resides in product NEWT,
as JOGL's SWT GLCanvas should be complete by now and reflect
all SWT dpi scaling API entries.

How and if we shall impose this non-native toolkit scaling in NEWT
for SWT or even expose it in general for NEWT is the question.
Comment 1 Sven Gothel 2020-01-07 00:00:38 CET
Marcel found still issues on Windows and Linux re scaling,
see Bug 1421 comment 11 through Bug 1421 comment 16.
Comment 2 Sven Gothel 2020-01-14 09:00:35 CET
See Marcel's on Bug 1421 comment 31
Comment 3 Sven Gothel 2020-01-14 09:01:45 CET
(In reply to Sven Gothel from comment #2)
Marcel wrote:

Just one note to the highDPI implementation on Window and Linux.

With the last built the canvas still occupies only 1/4 of the swt canvas.

I had to correct the scale at three positions where I scale up the NewtCanvasSWT:
 
int scale=(int)DPIUtil.getDeviceZoom()/100;


at the three positions where the call to newtChild.setSize(...) occurs.  I correct the implementation with:

newtChild.setSize(clientAreaWindow.width*scale, clientAreaWindow.height*scale);
Comment 4 Sven Gothel 2020-01-15 06:06:22 CET
(In reply to Sven Gothel from comment #3)
> int scale=(int)DPIUtil.getDeviceZoom()/100;

It is actually 'float scaleFactor = DPIUtil.getDeviceZoom()/100f;'

Something like this:

public static int autoScaleUp (int size) {
  final int deviceZoom = DPIUtil.getDeviceZoom();
  if (deviceZoom == 100) return size;
  float scaleFactor = deviceZoom/100f;
  return Math.round (size * scaleFactor);
}
Comment 5 Sven Gothel 2020-01-15 08:04:00 CET
commit d671b2ab3badbcfdbe0ff57f55ff997ba7bcb060

Bug 1422: Emulate DPI Scaling on non-native DPI autoscale platforms (!MacOS)
 
Bug 1422 shows that it seems to be desired to emulate DPI scaling where
the native toolkit does not implmement the same.
    
On GTK, DPIUtil.mapDPIToZoom (int dpi) reads:
  double zoom = (double) dpi * 100 / 96; 
  int roundedZoom = (int) Math.round (zoom);
  return roundedZoom;
   
While having dpi calculated as: 
    
  dpi = 96 * GDK.gdk_monitor_get_scale_factor(monitor);
    
Well, this seems to exist to allow 96 dpi fixed layout to
'look' OK on high-dpi screens.
However, you get in trouble if you layout high-dpi aware,
i.e. using percentages etc.
    
There is one exception: If DPIUtil.useCairoAutoScale() is true, scalingFactor is 1f
and hence the scaling emulation dropped.
    
'DPIUtil.setUseCairoAutoScale((sx[0]*100) == scaleFactor || OS.isGNOME);'
Comment 6 Sven Gothel 2020-01-15 09:27:36 CET
(In reply to Sven Gothel from comment #5)
https://jogamp.org/deployment/v2.4.0-rc-20200115/
Comment 7 Sven Gothel 2020-01-15 12:36:29 CET
Marcel, please test this issue here at well as I have no system setup to do so.
Thank you.
Comment 8 Marcel Au 2020-01-15 13:35:20 CET
Hello Sven,

tested on Windows. At first startup the view occupies 1/4.

When I click on the view the canvas get's resized to highdpi which is then correct.

When I change to fullscreen I see the same wrong fullscreen layout as on MacOSX (top-left coordinate of view).

After a fullscreen I also need to resize the view so that correct canvas size is applied.

When I close and reopen the perspective once again I need to resize the view.
Comment 9 Marcel Au 2020-01-15 15:43:56 CET
Hello Sven,

tested on Linux Ubuntu 19.10 (VirtualBox, Display set to scale factor 2). 

The scaling has no effect like on Windows only 1/4 is covered.

View resizing and clicking on the canvas has no effect (which is different from Windows).

Fullscreen is working with the correct coordinates.
Comment 10 Sven Gothel 2020-01-16 01:56:27 CET
(In reply to Marcel Au from comment #9)

commit 78609202731252f0024e6330cc94c52b05c1d146

    Bug 1422: Use own deviceZoomScaleUp(..) disregarding higher-toolkit's compensation like 'DPIUtil.useCairoAutoScale()'
    
    We can't use DPIUtil's 'autoScaleUp(..)' method on non-native DPI scaling platforms
    as it uses a scale-factor of 1f if the higher toolkit compensates, i.e. 'DPIUtil.useCairoAutoScale()'.
    
    Since NEWT uses X11 and GDI directly, which are not DPI scale-aware,
    we have to drop the semnatics of 'DPIUtil.useCairoAutoScale()'
    and merely use the actual 'deviceZoom'.
    
    This was proposed by Marcel Au in the first place.
    At least I understand these semantics by now.
    
    +++
    
    Additionally NewtCanvasSWT.SWTNativeWindow needs to return the 'deviceZoomScaleUp(..)'
    values for returning its size in window- and pixel-units (surface).
Comment 11 Sven Gothel 2020-01-16 01:56:58 CET
(In reply to Sven Gothel from comment #10)

Our TestGLCanvasSWTNewtCanvasSWTPosInTabs unit test uses
a window shell size of 640x480 and the NewtCanvasSWT a window-unit size of around 628x424.

Testing:
- move window
- resize window
- for each GLWindow (gears + red-square):
-- fullscreen-on (focus in GLWindow and press 'f')
-- fullscreen-off (focus in GLWindow and press 'f')

+++

Debian 10 w/ KDE High-DPI 'natural' - No Scale (100%)

SWT: Platform: gtk, Version 4930
SWT: isX11 true, isX11GTK true (GTK Version: 3.24.5)
SWT: isOSX false, isWindows false
SWT: DeviceZoom: 100, deviceZoomScalingFactor 1.0
SWT: Display.DPI Point {158, 158}; DPIUtil: autoScalingFactor 1.0 (use-swt true), useCairoAutoScale true

After creation we have a scaled up NEWT Window:
NewtCanvasSWT(0x13e39c73).Event.FOCUS_IN, Event {type=15 NewtCanvasSWT {} time=-925993934 data=null x=0 y=0 width=0 height=0 detail=0}
NewtCanvasSWT(0x13e39c73).Event.FOCUS_IN, WindowDriver[State [visible, child, repositionable, resizable]; Window[0/0 628x413 wu, 628x413 pixel] handle 0xcc0002a, surfaceHandle 0xcc0002a, children 0; ParentWindow com.jogamp.newt.swt.NewtCanvasSWT$SWTNativeWindow@1bd4fdd (handle 0xca00014)]

++++

Virtual Machine (VMM) qemu-kvm ..

Ubuntu 19.10 with Gnome WM and Gnome Settings: Display.Scale 200%

SWT: Platform: gtk, Version 4930
SWT: isX11 true, isX11GTK true (GTK Version: 3.24.12)
SWT: isOSX false, isWindows false
SWT: DeviceZoom: 200, deviceZoomScalingFactor 2.0
SWT: Display.DPI Point {96, 96}; DPIUtil: autoScalingFactor 1.0 (use-swt true), useCairoAutoScale true

After creation we have a scaled up NEWT Window:
NewtCanvasSWT(0x13e39c73).Event.FOCUS_IN, Event {type=15 NewtCanvasSWT {} time=14604810 data=null x=0 y=0 width=0 height=0 detail=0}
NewtCanvasSWT(0x13e39c73).Event.FOCUS_IN, WindowDriver[State [visible, child, repositionable, resizable]; Window[0/0 1256x848 wu, 1256x848 pixel] handle 0x2a0002a, surfaceHandle 0x2a0002a, children 0; ParentWindow com.jogamp.newt.swt.NewtCanvasSWT$SWTNativeWindow@1bd4fdd (handle 0x2800014)]


+++

Virtual Machine (VMM) qemu-kvm ..

Ubuntu 19.10 with Gnome WM and Gnome Settings: Display.Scale 100%

SWT: Platform: gtk, Version 4930
SWT: isX11 true, isX11GTK true (GTK Version: 3.24.12)
SWT: isOSX false, isWindows false
SWT: DeviceZoom: 100, deviceZoomScalingFactor 1.0
SWT: Display.DPI Point {96, 96}; DPIUtil: autoScalingFactor 1.0 (use-swt true), useCairoAutoScale true

After creation we have a scaled up NEWT Window:
NewtCanvasSWT(0x13e39c73).Event.FOCUS_IN, Event {type=15 NewtCanvasSWT {} time=15140863 data=null x=0 y=0 width=0 height=0 detail=0}
NewtCanvasSWT(0x13e39c73).Event.FOCUS_IN, WindowDriver[State [visible, child, repositionable, resizable]; Window[0/0 628x424 wu, 628x424 pixel] handle 0x2a0002a, surfaceHandle 0x2a0002a, children 0; ParentWindow com.jogamp.newt.swt.NewtCanvasSWT$SWTNativeWindow@1bd4fdd (handle 0x2800014)]
Window.focusAction() START - main, focusAction: null - windowHandle 0x2a0002a

+++
Comment 12 Sven Gothel 2020-01-16 01:59:48 CET
(In reply to Marcel Au from comment #8)

Fullscreen confusion might be related to 
"NewtCanvasSWT.SWTNativeWindow needs to return the 'deviceZoomScaleUp(..)'
values for returning its size in window- and pixel-units (surface)."

which is now committed, but only active for !OSX.

Hence it might/should work for Windows now.
(How to test w/ virtual machine on Window?)

Need to check on OSX...

Build aggregation maybe tomorrow morning,
just sneaked this in after my other regular work :)
Comment 13 Sven Gothel 2020-01-16 07:39:19 CET
(In reply to Sven Gothel from comment #10)
commit f63b94cccc71cf154a7a6d3359ceface3a683229

    Bug 1422: NewtCanvasSWT: Handle case of !OSX && DPIUtil.getScalingFactor() > 1
    
    NewtCanvasSWT.SWTNativeWindow's surfaceSize in pixel units
    shall only return scaled-up windowUnits using SWTAccessor.deviceZoomScaleUp(..) for !OSX
    and potentially auto scaled-up pixelUnits to passthrough (OSX).
    
    See detailed API doc to NewtCanvasSWT.newtScaleUp(..)

+++

What a good night sleep can do to you ;-)

+++

Marcel, now I need to learn how to test this on Windows.
Either using a virtual machine .. or on metal, but w/o a high dpi screen.
Comment 14 Marcel Au 2020-01-16 12:13:24 CET
Hello Sven,

with built 1502 Linux works correctly (as well as fullscreen).

On Windows still at startup the wrong coordinates. It can be mended when
I remove the boolean newtChildReady here:

https://github.com/sgothel/jogl/blob/78609202731252f0024e6330cc94c52b05c1d146/src/newt/classes/com/jogamp/newt/swt/NewtCanvasSWT.java#L225

When removing the boolean variable the NewtCanvas is scaled correctly at startup.

Only when exiting fullscreen the newtCanvas returns to a 1/4 size so have to to resize it to apply the scale correctly.
Comment 15 Sven Gothel 2020-01-22 09:21:54 CET
(In reply to Sven Gothel from comment #11)
Virtual Machine (VMM) qemu-kvm ..

Windows 7 and Display Settings: Custom 200% 'Size of Text'
Mesa 19.3.2 https://github.com/pal1000/mesa-dist-win/releases
GL_VENDOR      VMware, Inc.
GL_RENDERER    llvmpipe (LLVM 9.0, 128 bits)
GL_VERSION     3.3 (Core Profile) Mesa 19.3.2


SWT: Platform: win32, Version 4930
SWT: isX11 false, isX11GTK false (GTK Version: 0.0.0)
SWT: isOSX false, isWindows true
SWT: DeviceZoom: 200, deviceZoomScalingFactor 2.0
SWT: Display.DPI Point {96, 96}; DPIUtil: autoScalingFactor 2.0 (use-swt true), useCairoAutoScale false

+++

i.e. working on Windows as well.
Comment 16 Marcel Au 2020-01-22 14:18:50 CET
Hello Sven,

I rebuilt everything on Windows. Using again the latest built (1502).

Now it seems to work on HighDPI.

For fullscreen see comment for 1423:

https://jogamp.org/bugzilla/show_bug.cgi?id=1423
Comment 17 Sven Gothel 2020-02-03 02:07:21 CET
(In reply to Marcel Au from comment #16)
Last build <https://jogamp.org/deployment/v2.4.0-rc-20200202/>
contains the MacOS child window fullscreen fix.

Hence this bug maybe closed now, IMHO.
We have Bug 1423 and Bug 1424 still left though.
Comment 18 Marcel Au 2020-02-04 08:17:30 CET
Hello Sven,

thanks. Tested and works fine on MacOSX Retina.
Comment 19 Sven Gothel 2020-02-04 14:26:44 CET
see comment 17
Comment 20 Sven Gothel 2020-02-04 14:27:01 CET
see comment 18