Pinch Gesture on web platform

[JoshBashed]

Connected to: bevyengine/bevy#23421

Description

I am trying to detect pinch to zoom gestures when using winit on the web. This is problematic because pinch to zoom gestures use the same MouseWheel event scrolling uses. The only difference is that event.ctrlKey is set.

This results in a hacky solution to try to detect whether the control key is pressed by listening to two separate events.

Solution

Adding modifier state directly to the MouseWheel event and other events similar to it would allow higher ease of use for situations such as this one. Another solution could be to expose just a variable called is_pinching that represents the control key.

A new proposed solution is to send a pinch event when the ctrlKey modifier is sent during the event.

Example JS Code

// current
document.addEventListener('wheel', (e) => {
    dispatchModifiers(...)
    dispatchWheelEvent({ x: e.deltaX, y: e.deltaY });
});

// proposed
document.addEventListener('wheel', (e) => {
    dispatchModifiers(...)
    if (!e.ctrlKey) dispatchWheelEvent({ x: e.deltaX, y: e.deltaY, modifiers: ... });
    else dispatchPinchEvent({ … });
});

[madsmtm]

We've recently talked about this in #4522 too. Would that work for you?

[JoshBashed]

We've recently talked about this in #4522 too. Would that work for you?

This is a global. I think just adding it to the event is best.

[madsmtm]

For your specific issue, in terms of what Winit exposes, it would be enough for us to emit a synthetic ModifiersChanged before and after WindowEvent::MouseWheel - that would allow you to store current_modifiers elsewhere, and then use that when handling the wheel event.

Am I misinterpreting or misunderstanding that?

(Not saying this is ergonomic, just asking whether that would have the right semantics from your POV?)

[JoshBashed]

For your specific issue, in terms of what Winit exposes, it would be enough for us to emit a synthetic ModifiersChanged before and after WindowEvent::MouseWheel - that would allow you to store current_modifiers elsewhere, and then use that when handling the wheel event.

Am I misinterpreting or misunderstanding that?

(Not saying this is ergonomic, just asking whether that would have the right semantics from your POV?)

I can already capture the ModifiersChanged. It's just not a nice solution. I don't see why this can't be added to the MouseWheel event.

[madsmtm]

Hmm, I think there's a general confusion / disagreement about how these things work / should work.

In Winit, for each window, we expose effectively three things:

1. Methods to query the current state (Window::outer_position).
2. Events to subscribe to updates to that state (WindowEvent::Moved). Ideally, the window state is updated before the event is emitted.
3. Methods to (attempt to) change the state (Window::set_outer_position).

Unfortunately, we're not very good at doing this consistently. For example, I think it could make sense to provide a method for querying the list of currently pressed buttons, or an event to know when the window is minimized. IIRC Bevy doesn't really trust our events (with good reason), which leads to the design where it basically maintains a "shadow view" of the window.

I have lamented this a bit before in #3690.

I don't see why this can't be added to the MouseWheel event.

Found some prior discussion in #328 (where modifiers was on the MouseWheel event) and #1124 (comment) (where it was argued for its removal).

I think those arguments still hold, though we already have the "multiple sources of information" problem (window state vs. events). I guess having no way to query it (i.e. not having Window::modifiers) is also a way to avoid that problem, though I agree that it isn't very ergonomic.

[JoshBashed]

Hmm, I think there's a general confusion / disagreement about how these things work / should work.

In Winit, for each window, we expose effectively three things:

1. Methods to query the current state (Window::outer_position).
2. Events to subscribe to updates to that state (WindowEvent::Moved). Ideally, the window state is updated before the event is emitted.
3. Methods to (attempt to) change the state (Window::set_outer_position).

Unfortunately, we're not very good at doing this consistently. For example, I think it could make sense to provide a method for querying the list of currently pressed buttons, or an event to know when the window is minimized. IIRC Bevy doesn't really trust our events (with good reason), which leads to the design where it basically maintains a "shadow view" of the window.

I have lamented this a bit before in #3690.

I don't see why this can't be added to the MouseWheel event.

Found some prior discussion in #328 (where modifiers was on the MouseWheel event) and #1124 (comment) (where it was argued for its removal).

I think those arguments still hold, though we already have the "multiple sources of information" problem (window state vs. events). I guess having no way to query it (i.e. not having Window::modifiers) is also a way to avoid that problem, though I agree that it isn't very ergonomic.

The modifier sent during a wheel event is not meant to be used globally. I fundamentally disagree.

[alice-i-cecile]

The modifier sent during a wheel event is not meant to be used globally.

This is basically my position here, coming from Bevy. It's very weird that we might get a phantom modifier key pressed because of some touch gesture, with no corresponding release event, and which should not be used elsewhere.

Simultaneously though, I would be loathe to add this data onto MouseWheel: it's rather unrelated, and basically a leaky implementation detail.

The ideal for me would be "these gesture events are exposed as semantically distinct", and we don't muck about with either mouse wheel or modifier keys. If winit is sending me a mouse wheel event, or a modifier key event, I'd like to have some confidence that there's actually, you know, a mouse or keyboard involved. But that's inconsistent with how devs used to other stacks might think this works...

[JoshBashed]

Maybe the wheel event with a control key modifier should be treated differently because it fundamentally has a different meaning than scrolling.

[JoshBashed]

Potentially you could move it to a separate event.

[madsmtm]

This is problematic because pinch to zoom gestures use the same MouseWheel event scrolling uses. The only difference is that event.ctrlKey is set.

I think I didn't properly read this, that does sound pretty weird.

Potentially you could move it to a separate event.

That seems like the preferred solution, we already have WindowEvent::PinchGesture.

[madsmtm]

See also #2768 for the idea of emitting synthetic modifiers events on focus change.

[kchibisov]

The modifier event is like that because it can happen without any key press event received by the application in the first place. Having it in every event was done before and you can read the statement why it was like that.

The ideal for me would be "these gesture events are exposed as semantically distinct", and we don't muck about with either mouse wheel or modifier keys

I agree, we should probably put it in our gesture event if it's established semantic or at least let the user request such semantic.