2

Say I have a computer vision system capable of tracking multiple colored blobs. Further, assume I want the blobs to represent "fingers" in a multitouch system. Now, I have many examples I can look at to see how to make multitouch gestures. However, I cannot seem to find a clear explanation of how a "touch" event is defined in the first place. Best case scenario, the system can detect blob size and I can say something somewhat like "blobSize>minSize=touch". Worst case scenario. I can only detect position. Do I wait for the blob to stay still for some amount of time, or what?

1 Answer 1

1

A touch event, equivalent to a mouse click, would be a blob that appeared and then disappeared with minimal movement in between that happened on, or near enough on a clickable item.

You will have to have some tolerances for:

  • Minimum & maximum blob size
  • Minimum and Maximum duration, > maximum becoming a click and hold
  • How much movement is allowed before it becomes a drag event
  • How close to the clickable area you need to be.

It is possible that, when using a computer vision system that is tracking objects to represent finger tips, i.e. some tracking targets, they may be tracked from the edge of the field of view which is, of course, dissimilar to touch tracking, where you can lift your finger off to end the touch.

For this sort of system you will have to implement either a stateful system where there is a threshold size for the target, or even a delta size, representing the distance from the camera to the target as either a fixed distance from the camera as an invisible surface or as a motion towards it, then a motion that does not change the target size by more than a given threshold would constitute a drag and one that reduced it by more would be a button up event.

Alternatively, could also use a mechanism, similar to the eye tracking systems, where a pause on an active area of the screen is regarded as a button down and a longer pause as a change to button held then a pause at a drop target because the button release.

In either case I would strongly recommend also implementing a user training mode where the user and the system adjust to each other tailoring the thresholds for either target size or pause length that constitutes a click and hold. Remember people have different sizes of fingers, motion rates, habits, etc., and it is always better for the system to learn from the user rather than the other way round. Also remember that most people can never hold their fingers absolutely still for any amount of time so you will need to allow a threshold for how much movement doesn't count.

2
  • But thats just the thing - they dont appear or disappear spontaneously. They can enter or leave the screen via an edge, but they never just vanish.
    – jamesson
    Jan 28, 2017 at 8:04
  • @jamesson Addressed in the answer. Jan 28, 2017 at 10:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.