Bug 641 - NEWT: Distinguish keyCode (kbd layout independent) and keySym (kbd layout dependent) (was: Invalid keyCodes returned with com.jogamp.newt.event.KeyEvent)
Summary: NEWT: Distinguish keyCode (kbd layout independent) and keySym (kbd layout de...
Status: RESOLVED FIXED
Alias: None
Product: Jogl
Classification: JogAmp
Component: core (show other bugs)
Version: 2
Hardware: All all
: P3 normal
Assignee: Sven Gothel
URL:
Depends on:
Blocks:
 
Reported: 2012-11-16 05:46 CET by olamedia
Modified: 2013-04-11 10:00 CEST (History)
2 users (show)

See Also:
Type: ---
SCM Refs:
85338858f5c58694fa88e77df1386d0556887944 ed596d9a329f1788979e148a4d09df7815ada527 3052de236eb08a27286f20d52671bb40dbdd8875 6755fc707672a77025bcde81a47a5e4d93b37fb1 d2fc229b844942646fd9a32fc943923a5770a4be 09ebc2cd6fdd317134f0afb38174418a6d067d65 b13868b612689307ebf4e46ee4ede2fd0560e504
Workaround: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description olamedia 2012-11-16 05:46:52 CET
Left: native keycode, right: keySyms (to print: #xmodmap -pke)
keycode  25 = w W Cyrillic_tse Cyrillic_TSE
keycode  38 = a A Cyrillic_ef Cyrillic_EF
keycode  39 = s S Cyrillic_yeru Cyrillic_YERU
keycode  40 = d D Cyrillic_ve Cyrillic_VE

KeyCodes returned in english layout by com.jogamp.newt.event.KeyEvent.getKeyCode():
* Key pressed: 87
* Key pressed: 65
* Key pressed: 83
* Key pressed: 68
In russian layout:
* Key pressed: 1731
* Key pressed: 1734
* Key pressed: 1753
* Key pressed: 1751

Expected: same keycodes as in english layout

Sample output of "xev" utility for "w" key
#xev
* KeyPress event, serial 36, synthetic NO, window 0x4600001,
* root 0xb7, subw 0x0, time 326572351, (602,679), root:(669,730),
* state 0x10, keycode 25 (keysym 0x77, w), same_screen YES,
* XLookupString gives 1 bytes: (77) "w"
* XmbLookupString gives 1 bytes: (77) "w"
* XFilterEvent returns: False

* KeyPress event, serial 36, synthetic NO, window 0x4600001,
* root 0xb7, subw 0x0, time 326588119, (118,-7), root:(185,44),
* state 0x2010, keycode 25 (keysym 0x6c3, Cyrillic_tse), same_screen YES,
* XLookupString gives 2 bytes: (d1 86) "ц"
* XmbLookupString gives 2 bytes: (d1 86) "ц"
* XFilterEvent returns: False

Can't confirm, but it seems that event comes from parent component, ignoring /newt/native/ stuff.
Comment 3 Sven Gothel 2012-11-16 09:14:01 CET
As we discussed in the forum, I like to discuss terminology first, e.g.:
  <http://berrange.com/posts/2010/07/04/more-than-you-or-i-ever-wanted-to-know-about-virtual-keyboard-handling/>

Note: This text is X11 focused

Extracting the following from the above:
  scanCode - keymap independent code per key, same across same keyboard type
  keyCode   - keymap independent code per key, same across all keyboards - OS defined value
  keySym     - keymap *dependent* code per key, same across all keyboards - [same across OS ?]

- scanCode and keyCode are always reversible
- keySym and keyCode are reversible _if_ the keymap is known.

+++

Compared w/ NEWT's current state, and maybe even glancing at at AWT,
NEWT and AWT 'keyCode' matches the above 'keySym'
and hence depends on the keymap.

So from here on, let's name the current NEWT keyCode: keySym.

+++

If we desire to match the keymap independent keyCode definition,
we have to validate:
  A - how is this supported on the various platforms (X11, Windows, OSX, ..)
  B - how can we map keySym -> keyCode for AWT translated events
 
+++
Comment 4 Sven Gothel 2012-11-16 09:22:21 CET
Further more (if we acknowledge to impl. the orig keyCode definition), 
shall we replace and hence change the current keyCode semantics
- or - shall we introduce a new value, e.g. scanCode or iKeyCode (i == independent) ?

It is also unsure how keyCode - keySym events are mapped in case of 
key combinations like 'shift-ctrl-A'. As it is, I don't see a problem since we 
deliver an 'upper case A' semantic w/ keySym.

Question: Would a replacement of keySym/keyCode break applications ?
This would be true if users would already expect the symbolic key, 
e.g. a Cyrillic .. etc number.

Further more, the keySym delivered at up and down is the only available information
due to limitations of the Windows platform, only pressed deliveres the keyChar.
Comment 5 olamedia 2012-11-16 21:45:09 CET
One of good solutions is to get all keymaps (at init stage) and search for keySym less than 255 for each keyCode. Then use result as keymap for keyCode->keySym translation for AWT events. Reverse translation is possible using c++.

> Question: Would a replacement of keySym/keyCode break applications ?
It will repair some of applications. keyCode is a code for key since first keyboard :) getKeyCode() is expected to get keyCode, getChar()/getCharCode() - to get character.

> It is also unsure how keyCode - keySym events are mapped in case of 
key combinations like 'shift-ctrl-A'. As it is, I don't see a problem since we 
deliver an 'upper case A' semantic w/ keySym.
upper/lower doesn't matter since we must choose ONE NUMBER for both : it's some magic number, not a character. Shift and Ctrl are modifier keys, keySym will be given for A.

>shall we replace and hence change the current keyCode semantics
Yes, we should repair/replace current incorrect behavior for linux platform.

>shall we introduce a new value
We CAN introduce 
getScanCode() - for keyboard scan codes
getRawCode() - for OS keyCodes
getKeySym() - for linux keySyms
but that's not required.

>Windows platform
Windows platform looks OK

(Newt-AWT-Window KeyEvent[EVENT_KEY_TYPED, code 87(0x57), char 'w' (0x77), isActionKey false, InputEvent[modifiers: 0x0, NEWTEvent[sys:false, source:jogamp.newt.driver.windows.WindowsWindow, when:1353039647580 d 1ms]]]

Newt-Standalone KeyEvent[EVENT_KEY_TYPED, code 87(0x57), char 'w' (0x77), isActionKey false, InputEvent[modifiers: 0x0, NEWTEvent[sys:false, source:jogamp.newt.driver.windows.WindowsWindow, when:1353039646094 d 0ms]]]

But when typing in a text field the actual key is 'z')
Comment 6 olamedia 2012-11-16 21:47:56 CET
Another solution is to use first keySym (almost always latin).

Another is to use fixed keymap. (jinput approach)
Comment 7 olamedia 2012-11-16 23:10:29 CET
Confirmed on windows, both for translated and newt events.

newt is using character instead of scanCode:
static int WmKeyDown(JNIEnv *env, jobject window, UINT wkey, WORD repCnt, BYTE scanCode, BYTE flags, BOOL system) {
...
    jkey = WindowsKeyToJavaKey(wkey);
Comment 8 Sven Gothel 2012-11-16 23:32:45 CET
(In reply to comment #5)
> One of good solutions is to get all keymaps (at init stage) and search for
> keySym less than 255 for each keyCode. Then use result as keymap for
> keyCode->keySym translation for AWT events. Reverse translation is possible
> using c++.

Well, we have access to the OS keyCode AFAIK.

> 
> > Question: Would a replacement of keySym/keyCode break applications ?
> It will repair some of applications. keyCode is a code for key since first
> keyboard :) getKeyCode() is expected to get keyCode, getChar()/getCharCode() -
> to get character.

I can live w/ that, and you want OS keyCode - this was more a question for other users -> open question.

> 
> > It is also unsure how keyCode - keySym events are mapped in case of 
> key combinations like 'shift-ctrl-A'. As it is, I don't see a problem since we 
> deliver an 'upper case A' semantic w/ keySym.
> upper/lower doesn't matter since we must choose ONE NUMBER for both : it's some
> magic number, not a character. Shift and Ctrl are modifier keys, keySym will be
> given for A.
Ok.

> 
> >shall we replace and hence change the current keyCode semantics
> Yes, we should repair/replace current incorrect behavior for linux platform.
> 
> >shall we introduce a new value
> We CAN introduce 
> getScanCode() - for keyboard scan codes
> getRawCode() - for OS keyCodes
> getKeySym() - for linux keySyms
> but that's not required.

Well, it seems we change NEWT-keyCode to be the real keyCode as describe above - good.

> 
> >Windows platform
> Windows platform looks OK
> 
> (Newt-AWT-Window KeyEvent[EVENT_KEY_TYPED, code 87(0x57), char 'w' (0x77),
> isActionKey false, InputEvent[modifiers: 0x0, NEWTEvent[sys:false,
> source:jogamp.newt.driver.windows.WindowsWindow, when:1353039647580 d 1ms]]]
> 
> Newt-Standalone KeyEvent[EVENT_KEY_TYPED, code 87(0x57), char 'w' (0x77),
> isActionKey false, InputEvent[modifiers: 0x0, NEWTEvent[sys:false,
> source:jogamp.newt.driver.windows.WindowsWindow, when:1353039646094 d 0ms]]]
> 
> But when typing in a text field the actual key is 'z')

Oh - great news. So you are saying that on Windows, NEWT keyCode
is already keymap independent ?
Then thats great, and removes many above questions!
Comment 9 olamedia 2012-11-16 23:40:30 CET
Newt-Standalone

EN -> Newt-Standalone KeyEvent[EVENT_KEY_TYPED, code 87(0x57), char 'w' (0x77), isActionKey false, InputEvent[modifiers: 0x0, NEWTEvent[sys:false, source:jogamp.newt.driver.windows.WindowsWindow, when:1353102465898 d 0ms]]]

FR -> Newt-Standalone KeyEvent[EVENT_KEY_TYPED, code 90(0x5a), char 'z' (0x7a), isActionKey false, InputEvent[modifiers: 0x0, NEWTEvent[sys:false, source:jogamp.newt.driver.windows.WindowsWindow, when:1353102484011 d 0ms]]]

Newt-AWT-Window

EN -> Newt-AWT-Window KeyEvent[EVENT_KEY_TYPED, code 87(0x57), char 'w' (0x77), isActionKey false, InputEvent[modifiers: 0x0, NEWTEvent[sys:false, source:jogamp.newt.driver.windows.WindowsWindow, when:1353102690781 d 0ms]]]

FR -> Newt-AWT-Window KeyEvent[EVENT_KEY_TYPED, code 90(0x5a), char 'z' (0x7a), isActionKey false, InputEvent[modifiers: 0x0, NEWTEvent[sys:false, source:jogamp.newt.driver.windows.WindowsWindow, when:1353102707692 d 0ms]]]

Canvas-Window

Canvas-Window java.awt.event.KeyEvent[KEY_PRESSED,keyCode=87,keyText=W,keyChar='w',keyLocation=KEY_LOCATION_STANDARD,rawCode=87,primaryLevelUnicode=119,scancode=17] on canvas0

Canvas-Window java.awt.event.KeyEvent[KEY_PRESSED,keyCode=90,keyText=Z,keyChar='z',keyLocation=KEY_LOCATION_STANDARD,rawCode=90,primaryLevelUnicode=122,scancode=17] on canvas0
Comment 10 olamedia 2012-11-17 02:57:21 CET
TEST:
Pressing "W" key ("Z" in FR layout) on different platforms with different jre

java-6-sun-1.6.0 / LINUX
java.awt.event.KeyEvent[KEY_PRESSED,keyCode=87,keyText=W,keyChar='z',keyLocation=KEY_LOCATION_STANDARD,rawCode=25,primaryLevelUnicode=119,scancode=0] on frame0
java.awt.event.KeyEvent[KEY_TYPED,keyCode=0,keyText=Unknown keyCode: 0x0,keyChar='z',keyLocation=KEY_LOCATION_UNKNOWN,rawCode=0,primaryLevelUnicode=119,scancode=0] on frame0
java.awt.event.KeyEvent[KEY_RELEASED,keyCode=87,keyText=W,keyChar='w',keyLocation=KEY_LOCATION_STANDARD,rawCode=25,primaryLevelUnicode=119,scancode=0] on frame0

java-6-openjdk / LINUX
java.awt.event.KeyEvent[KEY_PRESSED,keyCode=87,keyText=W,keyChar='w',keyLocation=KEY_LOCATION_STANDARD] on frame0
java.awt.event.KeyEvent[KEY_TYPED,keyCode=0,keyText=Unknown keyCode: 0x0,keyChar='w',keyLocation=KEY_LOCATION_UNKNOWN] on frame0
java.awt.event.KeyEvent[KEY_RELEASED,keyCode=87,keyText=W,keyChar='w',keyLocation=KEY_LOCATION_STANDARD] on frame0

java-7-openjdk-i386 / LINUX
java.awt.event.KeyEvent[KEY_PRESSED,keyCode=87,keyText=W,keyChar='z',extModifiers=Button7,keyLocation=KEY_LOCATION_STANDARD,rawCode=25,primaryLevelUnicode=122,scancode=0,extendedKeyCode=0x5a] on frame0
java.awt.event.KeyEvent[KEY_TYPED,keyCode=0,keyText=Unknown keyCode: 0x0,keyChar='z',extModifiers=Button7,keyLocation=KEY_LOCATION_UNKNOWN,rawCode=0,primaryLevelUnicode=122,scancode=0,extendedKeyCode=0x0] on frame0
java.awt.event.KeyEvent[KEY_RELEASED,keyCode=87,keyText=W,keyChar='z',extModifiers=Button7,keyLocation=KEY_LOCATION_STANDARD,rawCode=25,primaryLevelUnicode=122,scancode=0,extendedKeyCode=0x5a] on frame0

JRE 1.7 / WINDOWS
JFrame-Window java.awt.event.KeyEvent[KEY_PRESSED,keyCode=90,keyText=Z,keyChar='z',keyLocation=KEY_LOCATION_STANDARD,rawCode=90,primaryLevelUnicode=122,scancode=17,extendedKeyCode=0x5a] on frame0 Ex: 90
JFrame-Window java.awt.event.KeyEvent[KEY_TYPED,keyCode=0,keyText=Unknown keyCode: 0x0,keyChar='z',keyLocation=KEY_LOCATION_UNKNOWN,rawCode=0,primaryLevelUnicode=0,scancode=0,extendedKeyCode=0x0] on frame0
JFrame-Window java.awt.event.KeyEvent[KEY_RELEASED,keyCode=90,keyText=Z,keyChar='z',keyLocation=KEY_LOCATION_STANDARD,rawCode=90,primaryLevelUnicode=122,scancode=17,extendedKeyCode=0x5a] on frame0

JDK 1.6 / WINDOWS
JFrame-Window java.awt.event.KeyEvent[KEY_PRESSED,keyCode=90,keyText=Z,keyChar='z',keyLocation=KEY_LOCATION_STANDARD,rawCode=90,primaryLevelUnicode=122,scancode=17] on frame0
JFrame-Window java.awt.event.KeyEvent[KEY_TYPED,keyCode=0,keyText=Unknown keyCode: 0x0,keyChar='z',keyLocation=KEY_LOCATION_UNKNOWN,rawCode=0,primaryLevelUnicode=0,scancode=0] on frame0
JFrame-Window java.awt.event.KeyEvent[KEY_RELEASED,keyCode=90,keyText=Z,keyChar='z',keyLocation=KEY_LOCATION_STANDARD,rawCode=90,primaryLevelUnicode=122,scancode=17] on frame0
Comment 11 Sven Gothel 2013-02-19 08:07:58 CET
http://jogamp.org/git/?p=jogl.git;a=commit;h=85338858f5c58694fa88e77df1386d0556887944

TODO:
  X11 keyCode: i.e. keySym for US Keyboard
  OSX  keyCode: i.e. keySym for US Keyboard
Comment 12 Sven Gothel 2013-04-08 23:13:28 CEST
Done: X11 and OSX .. for each a 'best practice' approach is being used.

OSX has no layout dependent keyCode to generate keySym, 
hence we create our own mapping of 1st 128 chars ..

Refine isPrintable()

Refined keySym definition, hinting to unshifted UTF16 keyCode value
for printable chars.
Comment 13 Sven Gothel 2013-04-11 10:00:07 CEST
Use UTF-16 UniCode for key-code and key-symbol exposing well defined key encoding and simplify usage.

This concludes keyCode/keySym handling, i.e. allows a more simple implementation
for platform drivers.