Dev09 – la battaglia del touchless
-
Upload
clemente-giorio -
Category
Devices & Hardware
-
view
134 -
download
2
Transcript of Dev09 – la battaglia del touchless
DEV09 – LA BATTAGLIA DEL TOUCHLESSMatteo Valoriani
@MatteoValoriani
Clemente Giorio
@Tinux80
#CDays15 – Milano 24, 25 e 26 Marzo 2015
Grazie a
PlatinumSponsor
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
Who we are…
Matteo ValorianiMicrosoft MVP Kinect for WindowsIntel Software InnovatorPhD on HCI @Politecnico di MilanoCEO of Fifth IngeniumConsultant
Clemente GiorioMicrosoft MVP Kinect for WindowsSenior Developer at Crionet srlInstructor at OvernetAuthor at Packt PublishingConsultant
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
Agenda
• HCI evolution
• Microsoft Kinect
• Intel Real Sense
• Leap Motion
• Duo3D
• Myo
• MaindWave
UI evolution
5Natural User Interface
#CDays15 – Milano 24, 25 e 26 Marzo 2015
You have to be a magician…
or at least a good illusionist
#CDays15 – Milano 24, 25 e 26 Marzo 2015
Kinect 2 - Specs
3D DEPTH SENSOR
RGB CAMERA
MULTI-ARRAY MIC
Hardware:
Depth resolution:512×424
RGB resolution:1920×1080 (16:9)
FrameRate:30 FPS
Latency:60 ms
#CDays15 – Milano 24, 25 e 26 Marzo 2015
#CDays15 – Milano 24, 25 e 26 Marzo 2015
Kinect 2 vs Kinect 1Version 1 Version 2
Depth range 0.4m → 4.0m 0.4m → 4.5m
Color stream 640×480 (VGA) 1920×1080 (Full HD)
Depth stream 320×240 @8bits 512×424 @16bits
Infrared stream None 512×424
Audio stream 4-mic array 16 kHz 4-mic array 48 kHz
USB 2.0 3.0
Hand Traking External tools Yes
Face Traking Yes Yes+Expressions
FOV 57° H 43° V 70° H 60° V
Tilt Motorized Manual
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
demo
Cons:The device is large
A dedicated power cord is required
The amount of data generated by the sensor also tends to saturate a USB controller
Kinect cannot easily distinguish individual fingers on a hand
Most of the features listed above require the Microsoft SDK, which is only supported for desktop applications on Windows 7 and 8.
Pros Vs. Cons
Pros:Skeletal trackingFace trackingMultiple sensors:Raw dataVoice control
Best for:Kiosks, installations, and digital signage projects where the user will be standing fairly far away from the display.
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
Understands 4 basic types of input
15Intel® RealSense™ Hands-On Lab - Milan
Categories of Input Capabilities Features
Hands • Hand and Finger Tracking • Gesture Recognition
• 22-point Hand and Finger Tracking• 9 static and dynamic mid-air gestures
Face • Face Detection and Tracking
• Multiple Face Detection and tracking• 78-point Landmark Detection (facial features)• Emotion Recognition (7 emotions, coming post-Beta)• Pulse Estimation• Face Recognition (Coming post-beta)
Speech • Speech Recognition • Command and Control• Dictation• Text to Speech
Environment • Segmentation• 3D Scanning• Augmented Reality
• Background Removal• 3D Object / Face / Room Scanning (Coming post-beta)• 2D/3D Object Tracking• Scene Perception (coming post-beta)
Understands Hardware Limits
16Intel® RealSense™ Hands-On Lab - Milan
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
demo
Cons:Getting some of the deeper features (like age and gender detection) to work is a bit tricky.
Device and software are in beta.
Due to the close range of the tracking system, hand gestures must be designed such that a user’s hand doesn’t occlude their own view of the display.
Pros Vs. Cons
Pros:
Smaller and less expensive
Close-range tracking
Hand posture/gesture recognition
Facial analysis
Speech (built-in support for speech synthesis powered by Nuance).
Raw data
Framework support: Processing, Unity and OpenFrameworks
Best for:Desktop/laptop applications where the user will be seated in front of the PC.Close range applications where features, apart from hand tracking and recognition, are necessary without too much precision or accuracy.
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
https://www.youtube.com/watch?v=_d6KuiuteIAhttps://airspace.leapmotion.com/
Leap Motion
Leap Motion -Field of View
150° - Long Side
120° - Short Side
Max 60 cm abovethe controller
Max 60 cm wide on each side
Leap Motion for Developers
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
demo
Cons:Sensing range is fairly limited.
Only fingers are tracked.
There is no skeleton or face tracking.
No access to the raw sensor data.
Pros Vs. Cons
Pros:
Finger tracking is fast and accurate.HMD Mountable
Smaller and less expensive
Framework support: .NET, Processing, Cinder, etc.
Compatible: Mac OS and Windows
Best for:Controlled kiosk environments with a pointing-based UI.Generally best for general audience desktop apps which can be distributed in the Airspace store. Head-mounted display video pass-through. Augmented Reality. Computer Vision.
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
https://youtu.be/XNztZ2Chmzg
Cons:SDK SamplesDevice APIDense3D APIDocumentation
Pros Vs. Cons
Pros:
Ultra Compact Design (52x25x11mm).Fast (360 FPS @320x120, 56FPS @752x480).FoV 170° with Low Distortion ( < 3% )Acceleromet, gyroscope and temperature sensor.HMD MountableOutdoor with environmental IR.Cross Platform
Best for:Augmented Reality and Virtual Reality HeadSet. Outdoor, portable applications. Mird-range depth sensing with low latency.
Leap, RealSense, Kinect
27Intel® RealSense™ Hands-On Lab - Milan
2,5 cm 60 cm 2 m 4 m
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
demo
Cons:Calibration
Dry Eyes
Near range
Pros Vs. Cons
Pros:
Accuracy
Low Latency
Lenses and glasses support
Price
Best for:EyeTraking, Marketing Analysis, Medical device option
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
Hand Gestures
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
demo
Cons:Confort One device for each user.For complex scenarios required multiple devices
Pros Vs. Cons
Pros:WearableHand gesturesMulti Platform (Windows, Mac, iOS and Android)Unity support
Best for:Outdoor, portable applications.
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
demo
Cons:Confort One device for each userFew sensors
Pros Vs. Cons
Pros:WearableCheapHand gesturesMulti Platform (Windows, Mac, iOS and Android)
Best for:Outdoor, portable applications.
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
Recap
Some selection criterias…1° 2° 3° 4°
2 or + users Kinect 2 Intel Leap
Full body interaction Kinect 2 - -
Hand Gesture Recognition Myo Intel Leap Kinect 2
Accuracy Leap Intel Kinect 2 Myo
Voice command Intel Kinect 2
FaceTraking Tobii Intel Kinect
Commecial use Kinect 2 Intel Tobii Leap Myo
Compatibility Leap MYO Tobii Intel Kinect 2
Capture Volumes
41Intel® RealSense™ Hands-On Lab - Milan
The user is performing a hand gesture outside of the capture volume. The camera will not see this gesture
Evaluate different settings and environment
42Intel® RealSense™ Hands-On Lab - Milan
Sensor with Camera use IR light and Sunlight can blind the camera!!!
• Check exposition during all day
• Verify that there isn’t direct light on the camera
Indoor/Outdoor
43Intel® RealSense™ Hands-On Lab - Milan
These devices aren’t a Rugged devices:
• Check temperatures (+3/33°)
• Check humidity
Indoor/Outdoor (2)
44Intel® RealSense™ Hands-On Lab - Milan
Comfortable positions
Your users are not GORILLAS!!!
Intel® RealSense™ Hands-On Lab - Milan 45
User posture may affect design of a gesture
Intel® RealSense™ Hands-On Lab - Milan 46
Input variability
Intel® RealSense™ Hands-On Lab - Milan 47
Feedback, feedback, feedback,…
48Intel® RealSense™ Hands-On Lab - Milan
View of user:
• User Viewport
• User Overlay
… where actions performed for some other purpose or unconscious signs are interpreted in order to influence/improve/facilitate the actors' future interaction or day-to-day life (from Alan Dix)
• The interaction is not purposeful from the person side, but it is designed “to happen”
• It “happens” in relation to signs which are not done for that (body temperature, unconscious reactions such as blink rate, or unconscious aspects of activities such as typing rate, vocabulary shifts (e.g. modal verbs), actions done for other purposes, …
• It is designed for people acting
Manage Incidental Interaction
49Intel® RealSense™ Hands-On Lab - Milan
#CDays14 – Milano 25, 26 e 27 Febbraio 2014
Q&A
Tutto il materiale di questa sessione suhttp://www.communitydays.it/
Lascia subito il feedback su questa sessione,potrai essere estratto per i nostri premi!
Seguici suTwitter @CommunityDaysITFacebook http://facebook.com/cdaysit#CDays15