Bug 814 - Enhance PointerEvent / Deprecate MouseEvent
: Enhance PointerEvent / Deprecate MouseEvent
Status: CONFIRMED
Product: Newt
Classification: JogAmp_Core
Component: core
: 1
: All all
: --- enhancement
Assigned To: Sven Gothel
Depends on: 807 813
Blocks: 595
  Show dependency treegraph
 
Reported: 2013-08-09 14:49 CEST by Sven Gothel
Modified: 2013-11-08 15:05 CET (History)
1 user (show)

See Also:
Type: ---
SCM Refs:
Workaround: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sven Gothel 2013-08-09 14:49:22 CEST
Currently PointerEvent is still called MouseEvent, rename it!

PointerEvent currently misses:
  tiltXYZ[]
to match
   http://www.w3.org/Submission/pointer-events/#pointerevent-interface

+++

Enhance PointerEvent to satisfy 'joystick' controllers.
- Add  PointerTypes: 
   - JoystickA(PointerClass.Offscreen) (analogue)
   - JoystickD(PointerClass.Offscreen) (digital)
   - ???
   - JoystickD == JoyPad ?

- Add 3rd axis for abs coord, z.

- Rel. movement:
  'float moveXYZ[]' and 'float moveScale'
  similar to rotate* !

  Optional:
    Abs. position may be calculated via current position
    and moveScale * moveXYZ[].    
    Resulting position may exceed window coordinates.
    Requires a device context to keep track of position,
    i.e. PointerDevice (see below).

    Hence moveScale is optional.

- Send event to focused window only or
  consider auto-assignment - see Bug 813

- PointerDevice extends InputDevice:
  - Pool of static PointerDevice's accessible by user to
    query (position) and configure (origin, ..).

  - Unique by it's Pointer-ID (as already used in PointerEvent)

  - Keep track of absolute position (see above).

  - Used to fire 'added' and 'removed' events.
Comment 1 Julien Gouesse 2013-08-09 19:00:56 CEST
We have to handle the following kind of events:
- button events (pressed, released) [digital]
- axis events (float) [analog]
- sliding axis events (float, like 2 axis: abscissa, ordinate) [analog]
- hat switch or POV (point-of-view) hat events (9 directions) [analog]
- vector3 events (float, like 3 axis: abscissa, ordinate, applicate) [analog]
- plug in/out events (when a gamepad/mouse/keyboard is added or removed)

We have to handle some effects:
- rumbler
- force feedback

We have to represent all these devices:
- keyboard
- mouse
- gamepad

For example, a mouse event should indicate which mouse has just generated it, nothing forces the end user to plug in a single mouse and a single keyboard even though it is the general case.

An input device must describe its "components" which is very important for gamepads. The developer should be able to listener a specific device. It's already possible to listen to all keyboards which is the case with current key listeners.
Comment 2 Julien Gouesse 2013-08-09 19:04:39 CEST
Generating "mouse" events and moving the mouse pointer with a gamepad or any joystick can be useful when there is no mouse, for video games machines and in assistive technology, for people who can no longer use a mouse because of a muscular weakness.
Comment 3 Julien Gouesse 2013-08-09 19:31:11 CEST
A mouse can be seen as an input device with 2 axis (X & Y), one or multiple buttons and maybe a wheel. The problem is that some native APIs only provide relative data about the mouse movements and the wheel. These "raw" APIs are useful to get more accurate and resolution independent data, especially in games. It allows to decrease the screen resolution without decreasing the accuracy of the mouse. If we use these APIs, we have to compute the absolute position from the relative movement.
Comment 4 Sven Gothel 2013-08-09 23:22:32 CEST
(In reply to comment #1)
> - axis events (float) [analog]
> - sliding axis events (float, like 2 axis: abscissa, ordinate) [analog]
> - hat switch or POV (point-of-view) hat events (9 directions) [analog]
> - vector3 events (float, like 3 axis: abscissa, ordinate, applicate) [analog]

.. can you elaborate on these ? 
Best if you can add a reference/link to documentation.
Comment 5 Sven Gothel 2013-08-09 23:24:20 CEST
(In reply to comment #1)
> We have to handle some effects:
> - rumbler
> - force feedback

I assume these are not input events, but triggers the user 
sent to the InputDevice itself.
Hence InputDevice / PointerDevice may require methods like:
  doRumble(int ms, float strength/frequency)
  ..
or something ?
Comment 6 Julien Gouesse 2013-08-10 00:52:01 CEST
(In reply to comment #4)
> (In reply to comment #1)
> > - axis events (float) [analog]
> > - sliding axis events (float, like 2 axis: abscissa, ordinate) [analog]
> > - hat switch or POV (point-of-view) hat events (9 directions) [analog]
> > - vector3 events (float, like 3 axis: abscissa, ordinate, applicate) [analog]
> 
> .. can you elaborate on these ? 
> Best if you can add a reference/link to documentation.

https://en.wikipedia.org/wiki/File:Joyopis.svg
The main stick [1] can generate axis events.
The green component [7] is a POV hat.
Vector3 events concern natural input interfaces, the only examples that comes to my mind are the Wiimote and the Kinect.

My main source of inspiration is OOIS Joystick listener:
https://github.com/wgois/Object-oriented-Input-System--OIS-/blob/master/includes/OISJoyStick.h
Comment 7 Julien Gouesse 2013-08-10 00:55:30 CEST
(In reply to comment #5)
> (In reply to comment #1)
> > We have to handle some effects:
> > - rumbler
> > - force feedback
> 
> I assume these are not input events, but triggers the user 
> sent to the InputDevice itself.
> Hence InputDevice / PointerDevice may require methods like:
>   doRumble(int ms, float strength/frequency)
>   ..
> or something ?

You're right, haptic effects should be treated in the device itself.
Comment 8 Julien Gouesse 2013-08-11 13:47:10 CEST
(In reply to comment #0)
> Currently PointerEvent is still called MouseEvent, rename it!
> 
On one hand, what will we do with the interfaces dealing with MouseEvent instances? MouseListener handles the callbacks mainly useful for the mice. Should we create another listener for the callbacks used by the gamepads?

On the other hand, MouseEvent.getButton() returns a short, this isn't enough for gamepads, the object representing the device should have a method that uses a short in input and that provides some information about the button in output. Maybe some other kinds of "components" (axis, POV hat, ...) will require some complementary information.
Comment 9 Julien Gouesse 2013-08-11 14:15:48 CEST
Other kinds of devices have to be considered (like it is already the case in JInput):
- fingerstick
- headtracker
- rudder
- stick (traditional joystick, like the one designed for the Apple 2E)
- trackball
- trackpad (?)
- wheel (a steering wheel, not a mouse wheel, see net.java.games.input.Controller)

Should the port type (USB, parallel, ...) be exposed in the API? The port number will have to be exposed. The device should indicate which effects are supported (rumbler(s)?).

Look at net.java.games.input.Component to have an idea of which kind of information have to be returned to identify a "component" of a gamepad.
Comment 10 Julien Gouesse 2013-11-07 22:27:32 CET
mousePressed and mouseReleased should be renamed, why not calling them buttonPressed and buttonReleased?

Instead of creating tons of classes to handle events, PointerEvent can fit into most of our needs except when adding or removing a "pointer device".

PointerDevice can contain a mapping between short button codes and real "components". The value of a component can be relative or absolute.

If every new move can be managed as an axis move, PointerEvent.pointerMoved() is enough for point-of-view controls, sliders and axis. getTranslation() or getMove() would be enough to indicate the moves along 3 axis. A PointerEvent must contain some information about the concerned device, not just the pointer type, the pointer identifier (Pointer-ID) should allow to retrieve some complementary information. PointerEvent.PovDirection could represent the 9 possible directions of a point-of-view control.

By default, multiple mice are seen as a single one by the operating system and the data are expressed in a resolution dependent way. How should we allow the developer to switch to the resolution independent raw data? Should each mouse be exposed as 2 pointer devices?
Comment 11 Julien Gouesse 2013-11-07 22:28:27 CET
Actually, pointerPressed and pointerReleased might be better than buttonPressed and buttonReleased.
Comment 12 Julien Gouesse 2013-11-08 15:05:25 CET
At first, maybe getMove() should be replaced by getD or getDX + getDY + getDZ to avoid any confusion as those names are already used in some other APIs and it clearly indicates that the data are relative (D -> delta).

The mouse data of the poller should be considered as relative only when the cursor is invisible (as it is done in some APIs based on JInput). The poller should store the previous fetched data.

- visible cursor:
  - The new delta is the difference between the current absolute coordinates and the previous unclipped absolute coordinates.
  - The new unclipped absolute coordinates and the new (potentially clipped) absolute coordinates are equal to the current absolute coordinates.

- invisible cursor
  - The new delta is the sum between the current relative coordinates and the previous delta.
  - The new unclipped absolute coordinates and the new (potentially clipped) absolute coordinates are equal to the sum between the current relative coordinates and themselves.

The absolute clipped coordinates have to be clipped later. We have to take care of the overflow too.

Then, the mouse (PointerDevice) can expose both its relative data and its absolute data without appearing twice in the list of devices.

Should the pollers be exposed in the public API too?

N.B: The absolute data depend on the display, monitor or screen. Another conversion might be necessary.