Anandu Seminar

8/2/2019 Anandu Seminar

1/20

1

DEPARTMENT OF TECHNICAL EDUCATION

GOVERNMENT OF KERALA

NN..SS..SSPPOOLLYYTTEECCHHNNIICCCCOOLLLLEEGGEEPPAANNDDAALLAAMM

SSEEMMIINNAARRRREEPPOORRTT

OONN

SSIIXXTTHHSSEENNSSEE TTEECCHHNNOOLLOOGGYY

PPRREESSEENNTTEEDD BBYY::

NNaammee :: AANNAANNDDUU SS KKUUMMAARR

RReegg.. NNoo :: 8899003300224422

CCllaassss NNoo :: 1144

DDEEPPAARRTTMMEENNTT OOFF EELLEECCTTRRIICCAALL && EELLEECCTTRROONNIICCSS

EENNGGIINNEEEERRIINNGG

22001111--22001122


2/20

2

NN..SS..SS PPOOLLYYTTEECCHHNNIICC CCOOLLLLEEGGEE

PPAANNDDAALLAAMM

DDEEPPAARRTTMMEENNTT

OOFF

EELLEECCTTRRIICCAALL && EELLEECCTTRROONNIICCSS EENNGGIINNEEEERRIINNGG

CCEERRTTIIFFIICCAATTEE

Certified that this is a bonfire record of the seminar report on

SSIIXXTTHHSSEENNSSEETTEECCHHNNOOLLOOGGYY

Presented by AANNAANNDDUUSS KKUUMMAARRReg. No: 89030242Final semestereelleeccttrriiccaall

&&EElleeccttrroonniiccssEEnnggiinneeeerriinnggssttuuddeennttofN.S.SPolytechnic College, Pandalam in partial fulfillment of the requirements for the award of Diploma in EElleeccttrriiccaall && EElleeccttrroonniiccss

Engineering under theDirectorate ofTechnicalEducation, Kerala state during the year 2011-

12.

Guide: RAJESH

Radhika. P(Head of section)


3/20

3

Examiner

ACKNOWLEDGEMENT

I hereby express my gratitude to Mr. M.KISHOR KUMAR, Principal of

N.S.S POLYTECHNIC COLLEGE, PANDALAM, PATHANAMTHITTA and, Mrs. P. RADHIKA

Head of Section, Electrical & Electronics Engineering for their valuable advice and support

for this seminar.

I am immensely thankful to my seminar coordinator Mr. RAJESH INSTRUCTER

Department of Electrical & Electronics For coordinating the seminar, and giving me

valuable suggestions and guidance without which the work would not have been possible.

Finally, I thank my parents, classmates and all well-wishers who helped me in the

successful completion of my seminar. Last but not the least; I thank the Almighty without

whose blessings I would not have succeeded in my endeavors.

ANANDU S KUMAR


4/20

4

CONTENTS

y Abstract . 05y Introduction - Sixth Sense Technology . 06y The Master Mind . 07y Definition . 08y Gestures . 09y Components . 10y The Working & Basics . 11y The Related Technologies . 13y Educational Use . 15 Engagement ResearchCollaborationCost-effectiveness

y Advantages . 16y Future Implications . 17y Capabilities . 17y Applications . 18y Anticipated Time Frame . 19y Conclusion . 20y Reference . 20


5/20

5

ABSTRACT

'Sixth Sense' is a wearable gestural interface that augments the physicalworld around us with digital information and lets us use natural hand gestures to interact

with that information. All of us are aware of the five basic senses seeing,feeling, smelling, tasting and hearing. But there is also another sense calledthe Sixth Sense. It is basically a connection to something greater than what their physicalsenses are able to perceive. To a layman, it would be something supernatural. Somemight just consider it to be a superstition or something psychological. But the inventionofSixth Sense technology has completely shocked the world. Although it is not widelyknown as of now but the time is not far when this technology will change our perceptionof the world. Pranav Mistry, 28 year old, of Indian origin is the mastermindbehind the sixth sense technology. The device sees what we see but it lets outinformation that we want to know while viewing the object. It can project

information on any surface, be it a wall, table or any other object and useshand / arm movements to help us interact with the projected information. Thedevice brings us closer to reality and assists us in making right decisions byproviding the relevant information, thereby, making the entire world a computer. TheSixth Sense prototype implements several applications that demonstrate theusefulness, viability and flexibility of the system. The map application lets the usernavigate a map displayed on a nearby surface using hand gestures, similar to gesturessupported by Multi-Touch based systems, letting the user zoom in, zoom out or pan usingintuitive hand movements. The user can stop by any surface or wall and flick through thephoto she/she has taken. Sixth Sense also lets the user draw icons or symbols

in the air using the movement of the index finger and recognizes those symbols asinteraction instructions. For example, drawing a magnifying glass symbol takes theuser to the map application or drawing a @ symbol lets the user check hismail.

The Sixth Sense prototype is used to implement several applicationsthat have shown the usefulness, viability and flexibility of the system.

The current prototype costs around $350.


6/20

6

SIXTH SENSE TECHNOLOGY

Although miniaturized versions of computers help us to connect to thedigital world even while we are travelling there arent any devices as of now whichgives a direct link between the digital world and our physical interaction with the realworld. Usually the informations are stored traditionally on a paper or a digital storagedevice. Sixth sense technology helps to bridge this gap between tangible and non-tangibleworld. Sixth Sense device is basically a wearable gestural interface that connects thephysical world around us with digital information and lets us use natural hand gesturesto interact with this information .The sixth sense technology was developed by PranavMistry, a PhD student in the Fluid Interfaces Group at the MIT Media Lab. The sixthsense technology has a Web 4.0 view of human and machine interactions. Sixth Senseintegrates digital information into the physical world and its objects, making theentire world your computer. It can turn any surface into a touch-screen for computing,controlled by simple hand gestures. It is not a technology which is aimed at changinghuman habits but causing computers and other machines to adapt to human needs. Italso supports multi user and multi touch provisions. Sixth Sense device is a mini-

projectorcoupled with a camera and a cell phonewhich acts as the computer andyour connection to the Cloud, all the information stored on the web. The currentprototype costs around $350.

The Sixth Sense prototype is used to implement several applicationsThat have shown the usefulness, viability and flexibility of the system.


7/20

7

THE MASTER MIND:

PRANAV MISTRY

Pranav Mistry, Indian origin is the mastermind behind the sixth sensetechnology. Born in Palanpur, Gujarat he received his Bachelors of technology incomputer science and engineering from India and later received masters in design fromMIT Boston. He invented Sixth Sense / WUW ( Wear UR World) which is a wearable

gestural , user friendly interface which links the physical world around us with digitalinformation and uses hand gestures to interact with them. Pranavs eagerness to poppixels out of the digital world into the real world led to the birth of Sixth Sensetechnology.

Currently he is a PhD student at MIT and he won the Invention of the Year2009 - by Popular Science. The device sees what we see but it lets out information thatwe want to know while viewing the object. It can project information on any surface, beit a wall, table or any other object and uses hand / arm movements to help us interact withthe projected information. The device brings us closer to reality and assists us in making

right decisions by providing the relevant information, thereby, making the entire world acomputer.


8/20

8

DEFINITION

Sixth Sense' is a wearable gestural interface that augments the physical

world around us with digital information and lets us use natural hand gestures to interactwith that information. By using a camera and a tiny projector mounted in a pendant likewearable device, 'Sixth Sense' sees what you see and visually augments any surfaces orobjects we are interacting with. It projects information onto surfaces, walls, and physicalobjects around us, and lets us interact with the projected information through natural handgestures, arm movements, or our interaction with the object itself. 'Sixth Sense' attemptsto free information from its confines by seamlessly integrating it with reality, and thusmaking the entire world your computer. All of us are aware of the five basic senses seeing, feeling, smelling, tasting and hearing. But there is also another sense calledthe sixth sense. It is basically a connection to something greater than what their physicalsenses are able to perceive. To a layman, it would be something supernatural. Somemight just consider it to be a superstition or something psychological. But the inventionofsixth sense technology has completely shocked the world. Although it is not widelyknown as of now but the time is not far when this technology will change our perceptionof the world.

The Sixth Sense prototype is comprised of a pocket projector, a mirror anda camera. The hardware components are coupled in a pendant-like mobile wearabledevice. Both the projector and the camera are connected to the mobile computing devicein the users pocket. The device projects visual information, enabling surfaces, walls and

physical objects around the wearer to be used as interfaces; while the camera recognizesand tracks the user's hand gestures and physical objects using computer-vision basedtechniques. The software program processes the video stream data captured by thecamera and tracks the locations of the colored markers at the tip of the users fingersusing simple computer-vision techniques. The movements and arrangements of thesefiducials are interpreted into gestures that act as interaction instructions for the projectedapplication interfaces. The maximum number of tracked fingers is only constrained bythe number of unique fiducials, thus Sixth Sense also supports multi-touch and multi-userinteraction.

The device sees what we see but it lets out information that we want toknow while viewing the object. It can project information on any surface, be it a wall,table or any other object and uses hand / arm movements to help us interact with the projected information. The device brings us closer to reality and assists us in makingright decisions by providing the relevant information, thereby, making the entire world acomputer.


9/20

9

GESTURES

The software recognizes three kinds of gestures:

Multi-Touch Gestur

Like the ones we see in the iphone-where we touch the screen and make themap move by pinching and dragging.

Freehand Gestures:

Like when you take a picture or Namaste gesture to start the projection onthe wall. The below figure shows the basic components of the hand Gestures

ICONICGestures:

Drawing a icon in the air. Like, whenever we draw a star, shows us theweather details. When we draw a magnifying glass, shows us the map.


10/20

10

COMPONENTS

The devices which are used in sixth sense technology are :1. Camera

2. Coloured Marker3. Mobile component4. Projector5. Mirror

Camera:It captures the image of the object in view and tracks the users hand gesture. The camerarecognizes individuals, images, pictures, gestures that user makes with his hand. Thecamera then sends this data to a smart phone for processing. Basically the camera forms adigital eye which connects to the world of digital information.

Coloured Marker:There are color markers placed at the tip of users finger. Marking the users fingers withred, yellow green and blue colored tape helps the webcam to recognize the hand gestures.The movements and arrangement of these markers are interpreted into gestures that act asa interaction instruction for the projected application interfaces.

Mobile Component:The Sixth Sense device consists of a web enabled smart phone which process the datasend by the camera. The smart phone searches the web and interprets the hand gestures

with help of the colored markers placed at the finger tips.

Projector:The information that is interpreted through the smart phone can be projected into anysurface. The projector projects the visual information enabling surfaces and physicalobjects to be used as interfaces. The projector itself consists of a battery which have3hours of battery life.A tiny LED projector displays the data sent from the smart phone on any surface in

view- object, wall or person. The downward facing projector projects the image on to amirror.

Mirror:The usage of a mirror is important as the projector dangles pointing downwards from theneck. The mirror reflects the image on to a desire surface. Thus finally the digital imageis freed from its confines and placed in the physical world.


11/20

11

HOW IT WORKS

The Sixth Sense prototype is comprised of a pocket projector, a mirror and

a camera. The hardware components are coupled in a pendant like mobile wearabledevice. Both the projector and the camera are connected to the mobile computing devicein the users pocket. The projector projects visual information enabling surfaces, wallsand physical objects around us to be used as interfaces; while the camera recognizes andtracks user's hand gestures and physical objects using computer-vision based techniques.The softwareprogram processes the video stream data captured by the camera and tracksthe locations of the colored markers at the tip of the users fingers using simplecomputer-vision techniques. The movements and arrangements of these fiducials areinterpreted into gestures that act as interaction instructions for theprojected application interfaces. The maximum number of tracked fingers is onlyconstrained by the number of unique fiducials, thus Sixth Sense also supports multi-touchand multi-user interaction.

Both the projector and the camera are connected to the mobile computing devicein the users pocket. The projector projects visual information enabling surfaces, wallsand physical objects around us to be used as interfaces; while the camera recognizes andtracks users hand gestures and physical objects using computer-vision based techniques.The software program processes the video stream data captured by the camera and tracksthe locations of the colored markers at the tip of the users fingers using simplecomputer-vision techniques. It also supports multi touch and multi user interaction.

The technology in itself is nothing more than the combination ofsome stunning technologies. The technology is mainly based on hand gesture recognition,image capturing, processing, and manipulation, etc. The software of the technology usesthe video stream, which is captured by the camera, and also tracks the location of the tipsof the fingers to recognize the gestures. This process is done using some techniques ofcomputer vision. However, instead of requiring you to be in front of a big screen likeTom Cruise, Sixth Sense can do its magicand a lot moreeverywhere. The camerarecognizes objects around you instantly, with the micro-projector overlaying theinformation on any surface, including the object itself or your hand. Then, you can accessor manipulate the information using your fingers. The key here is that Sixth Sense

recognizes the objects around you, displaying information automatically and letting youaccess it in any way you want, in the simplest way possible. Clearly, this has the potentialof becoming the ultimate "transparent" user interface for accessing information abouteverything around us. If they can get rid of the colored finger caps and it ever goesbeyond the initial development phase, that is. But as it is now, it may change the way weinteract with the real world and truly give everyone complete awareness of theenvironment around us.


12/20

12

The Sixth Sense technology works as follows:

o It captures the image of the object in view and tracks the users hand gesture.

o There are colour markers placed at the tip of users finger. Marking the usersfingers with red, yellow green and blue coloured tape helps the webcam torecognize the hand gestures. The movements and arrangement of these markersare interpreted into gestures that act as a interaction instruction for the projectedapplication interfaces.

o The smart phone searches the web and interprets the hand gestures with help of thecoloured markers placed at the finger tips

o The information that is interpreted through the smart phone can be projected intoany surface.

o The mirror reflects the image on to a desire surface.

THE BASICS

o Sixth Sense allows people to use the internet without a screen or a keyboardo It can turn any surface into a touch screeno It is made with store-bought componentso The software program processes the video stream data captured by the camera

and tracks the locations of the colored markers [that you wear on your fingers]using simple computer-vision techniques)

o According to a number of sources, graduate students at MIT have been working onthis for the past four months day and night


13/20

13

THE RELATED TECHNOLOGIES

Augmented Reality:

The augmented Reality is a visualization technology that allows the user toexperience the virtual experience added over real world in real time. With the help ofadvanced AR technology the information about the surrounding real world of the user becomes interactive and digitally usable. Artificial information about the environmentand the objects in it can be stored and retrieved as an information layer on top of the realworld view. When we compare the spectrum between virtual reality, which createsimmersive, computer-generated environments, and the real world, augmented reality iscloser to the real world. Augmented reality adds graphics, sounds, haptic feedback andsmell to the natural world as it exists. Both video games and cell phones are driving thedevelopment of augmented reality. The augmented systems will also superimpose

graphics for every perspective available and try adjusting to every movement of theusers head and eyes. The three basic components of an augmented reality system are thehead-mounted display, tracking system and mobile computer for the hardware. The maingoal of this new technology is to merge these three components into a highly portableunit much like a combination of a high tech Walkman and an ordinary pair or eyeglasses.The head-mounted display used in augmented reality systems will enable the user to viewsuperimposed graphics and text created by the system. Another component of anaugmented reality system is its tracking and orientation system. This system pinpoints theusers location in reference to his surroundings and additionally tracks the users eye andhead movements. Augmented reality systems will need highly mobile computers. As of

now many computers arent there to satisfy to provide this option.

Gesture Recognition:

It is a technology which is aimed at interpreting human gestures with thehelp of mathematical algorithms. Gesture recognition technique basically focuses on theemotion recognition from the face and hand gesture recognition. Gender recognitiontechnique enables humans to interact with computers in a more direct way without usingany external interfacing devices. An interface which solely depends on the gesturesrequires precise hand pose tracking. In the early versions of gesture recognition processspecial type of hand gloves which provide information about hand position orientationand flux of the fingers. In the Sixth Sense devices colored bands are used for this purpose. Once hand pose has been captured the gestures can be recognized usingdifferent techniques. Neural network approaches or statistical templates are thecommonly used techniques used for the recognition purposes. These techniques have ahigh accuracy usually showing accuracy of more than 95%. Time dependent neuralnetwork will also be used for real time recognition of the gestures.


14/20

14

Computer Vision:

Computer vision is the technology in which machines are able tointerpret/extract necessary information from an image. Computer vision technologyincludes various fields like image processing, image analysis and machine vision. It

includes certain aspect of artificial intelligence techniques like pattern recognition. Themachines which implement computer vision techniques require image sensors whichdetect electromagnetic radiation which are usually in the form of ultraviolet rays or lightrays. The computer vision find itself applicable in varies field of interest. One such fieldis bio medical image processing. Its also used in autonomous vehicle like SUVs. Thecomputer vision technique basically includes four processes.

1. Recognition: One of the main task of computer vision technique is to determinewhether the particular object contain the useful data or not.

2. Motion Analysis: Motion analysis includes several tasks related to estimation ofmotion where an image sequence is processed continuously to detect the velocityat each point of the image or in the 3D scene.

3. Scene Reconstruction: Computer vision technique employs several methods torecreate a 3D image from the available images of a scene.

4. Image Restoration: The main of aim of this step is to remove noise from a givenimage. The simplest method involves using simple filters like low pass or medianfilters. In order to get better quality images more complex methods like In paintingare used.

Radio

Frequency

Identification: Radio frequency identification systems

transmit the identity of an object wirelessly, using radio magnetic waves. The mainpurpose of a radio frequency identification system is to enable the transfer of a data via aportable device. The portable device is commonly known as tag. The data send by the tagis received and processed by a reader according to the needs of the application. The datasend by the tag contains various informations identification or location of theinformation, or specifics about the product that has been tagged, for example price,colour, date of purchase, etc. This technology gained importance because of its ability totrack moving object. A typical radio frequency tag consists of a microchip attached to aradio antenna which is mounted on a substrate. To retrieve the data from the tag a readeris needed. A typical radio frequency reader consists of two antennas that emit radiowaves and at the same are capable of accepting the signals from the tag. The reader then passes the information that it has received to a computer device in digital form. Thecomputer then interprets this digital data and processes it. Radio frequency identificationtechniques are widely used in the fields like asset tracking, supply chain management,manufacturing and payment systems etc.


15/20

15

EDUCATIONAL USE

EngagementResearch Collaboration Cost-effectivenessEngagement

o Teachers may find that Sixth Sense, because of its unique ability to workwith many of Gardners multiple intelligences, will be a catalyst forstudents who might otherwise remain unengaged in learning.

o Sixth Sense uses the visual, kinesthetic, intrapersonal and interpersonalamong others.

o If this technology does nothing else, the idea that students can simply usetheir hands to draw images on walls and take pictures with their hands willbe (at least initially) quite motivating

Research

o Sixth Sense will give new meaning to looking something up.o Since we are essentially talking about the internet, students will be given

the opportunity to portably research the world.o They can all go on a virtual field trip and project the images on their own

desks, for example.

Collaborationo Students will be given an opportunity through this technology to feel more

connected to others.o Although there does seem to be the sense that individuals could walk

through life wired and lonely, I like to think the classroom would serve asthe environment in which students and teachers using Sixth Sense wouldwant to share.

o Not only could the sharing take place in the classroom, I suppose since theusers are connected online, sharing could take place around the world.

Cost-effectiveness

o As mentioned earlier, this product currently costs only $350. From aneducators stand point, this is minimum given the amount of money spenton textbooks, computer labs and LCD projectors.

o For educators, Sixth Sense would be a way to put technology(literally) into each students hands, because of its size and its cost.


16/20

16

ADVANTAGES

Portable:One of the main advantages of the sixth sense devices is its

small size and portability. It can be easily carried around without any difficulty. Theprototype of the sixth sense is designed in such a way that it gives more importance tothe portability factor. All the devices are light in weight and the smart phone caneasily fit into the users pocket.

Support multi touch and multi user interaction:Multi touch and multi user interaction is another added feature

of the sixth sense devices. Multi sensing technique allows the user to interact withsystem with more than one than one finger at a time.. Sixth sense devices also in-cooperates multi user functionality. This is typically useful for large interaction

scenarios such as interactive table tops and walls.

Cost effective:The cost incurred for the construction of the sixth sense proto

type is quiet low. It was made from parts collected together from common devices.And a typical sixth sense device cost up to $300. The sixth sense devices have not been made in large scale for commercial purpose. Once that happens its almostcertain that the device will cost much lower than the current price.

Connectedness between real world and digital world:

Forming a connection between the real world and the digitalworld was the main aim of the sixth sense technology.

Data access directly from the machines in real time:With help of a sixth sense device the user can easily access data

from any machine at real time speed. The user doesnt require any machine-humaninterface to access the data. The data access through recognition of hand gestures ismuch easier and user friendlier compared to the text user interface or graphical userinterface which requires keyboard or mouse.

Mind map the idea anywhere:With the advent of the sixth sense device, requirement of a

platform or a screen to analyze and interpret the data has become obsolete. We canproject the information into any surface and can work and manage the data as per ourconvenience...


17/20

17

Open source software:The software that is used to interpret and analysis the data

collected by the device is made open source. This enables other developers tocontribute to the development of the system

FUTUREIMPLICATIONS

o Students could become too wiredand the element of collaboration I envisionmight be completely erased by how the individual interacts

o Availability of inappropriate information (how does this differ from currentcomputer labs? this information is literally at students fingertips.)

o The technology will cost more because it is packaged and marketedand they cancharge more.What type of signal is needed...And might there be issues with getting signals inschools?

CAPABILITIES

o Recognizes hand movementso Examples from online demonstrations:o Make frame with hands: camera takes picture

Google map onto wall Draw circle on wrist: projects digital clock Movement of text book/web info onto page Airline ticket updates Draw @ symbol: check email for you Many more capabilities (its on YouTube or TED)


18/20

18

APPLICATION

The sixth sense technology finds a lot of application in the modern world.

Viewing Map: With the help of a map application the user can call upon any map

of his/her choice and navigate through them by projecting the map on to any surface. Byusing the thumb and index fingers movements the user can zoom in, zoom out or pan theselected map.

TakingPictures:Another application of sixth sense devices is the implementation

of a gestural camera. This camera takes the photo of the location user is looking at bydetecting the framing gesture. After taking the desired number of photos we can projectthem onto any surfaces and then use gestures to sort through those photos and organizeand resize them.

Drawing Application:The drawing application allows the user you to draw on any

surface by tracking the fingertip movements of the users index finger. The pictures thatare drawn by the user can be stored and replaced on any other surface. The user can alsoshuffle through various pictures and drawing by using the hand gesture movements

MakingCalls:We can make calls with help of sixth sense device .The sixth

sense device is used to project the keypad into your palm and the using that virtualkeypad we can make calls to anyone.

Interacting with physical objects:The Sixth Sense system also helps to interact with physical

objects we use in a better way. It augments physical objects by projecting moreinformation about these objects projected on them. For example, a gesture of drawing acircle on the users wrist projects an analog watch on the users hand. Similarly anewspaper can show live video news or dynamic information can be provided on a

regular piece of paper

GettingInformation:Sixth sense devices can be used for getting various information

relating to our everyday life by getting in contact with objects.


19/20

19

ProductInformation:Sixth sense technology uses marker technology or image

recognition techniques to recognize the objects we pick in our hand and then provideinformation relating to that product.

BookInformation:By holding and shuffling through the book pages, the sixth

sense provides Amazon ratings on that book, other reviews and other relevant thingsrelated to the book.

Flight Updates:With the help of the sixth sense technology it is no longer

required to log into any sites for checking the status of the flights. The system willrecognize your boarding pass and let you know whether the flight is on time or not.

Information AboutPeople:With help of face recognition techniques the sixth sense

devices are able to provide information about the people when we meet them. The sensordetects the face and checks the data base for the relevant information. The system willthen project the relevant information about a person like what they do, where they work.

ANTICIPATED TIMEFRAME

o According to a number of sources, the MIT students already have a patent onSixth Sense. Still it is unclear as to when this device might actually becomeavailable for general purchasing.

o Microsoft Surface does currently exist (which is similar in theory). ButMicrosofts product is not portable like Sixth Sense.

o Even if it came out next week, as educators we may need to wait some time inorder to learn of the dangers that are sure to exist with the product.


20/20

20

CONCLUSION

Sixth Sense recognizes the objects around us, displaying informationautomatically and letting us to access it in any way we need. This prototype implements

several applications that demonstrate the usefulness, viability and flexibility of thesystem, allowing us to interact with this information via natural hand gestures. This became the ultimate transparent user interface for accessing information abouteverything around us.

REFERENCE

www.google.com

www.wikipedia.orgwww.pranavmistry.comwww.theviewspaper.net

Anandu Seminar

Documents

Transcript of Anandu Seminar