SmartSense is a wearable holographic gesture and voice controlled intelligent system.
With the growing fascination in mobile/portable devices comes the question, what is the easiest way to access information throughout the day? Clearly, having to pull out a laptop, tablet or cell phone every time you need access to information can be frustrating. What if there was a device that could be worn everyday that put this information at your fingertips without having to dig into your bag or pockets? Our wearable computer allows users to interact with their system via simple hand gestures and view data in an easy to use graphical user interface that can be displayed virtually anywhere.
We have taken current color tracking techniques used in the Touchless software development kit (SDK) and spun it so hand gestures can be recognized by using only a webcam. The webcam is worn on an apparatus that hangs from the user’s neck that also harnesses a pico-projector. The webcam takes in gestures which manipulate the graphical user interface that is projected onto nearby surfaces by the pico-projector. This system will allow users to easily access information using nothing but their hands. Not only is this a new and exciting way of using computers, but the hand gestures used to control the system are simple, which enables this computing system to be used by almost anyone.

Human-computer interaction (HCI) is a heavily researched area in today’s technology driven world. Most people experience HCI using a mouse and keyboard as input devices. We wanted a more natural way to interact with the computer that also allows instant access to information. We are developing a real-time system that is always on and available to collect data at any moment but also is accessible “on-the-fly”. Our aim is to make our machine usable and receptive to the user’s needs in a timely manner. We do this by developing algorithms to detect hand gestures that map to controlling the graphical user interface (GUI) in an expected manner. This allows the user to access this information without the need to turn on a laptop or reach in their pocket to view their computer device We will also be looking at shrinking our apparatus as well as moving from a laptop to a smaller portable device for our processing engine.
We have taken current color tracking techniques used in the Touchless software development kit (SDK) and spun it so hand gestures can be recognized by using only a webcam. The webcam is worn on an apparatus that hangs from the user’s neck that also harnesses a pico-projector. The webcam takes in gestures which manipulate the graphical user interface that is projected onto nearby surfaces by the pico-projector. This system will allow users to easily access information using nothing but their hands. Not only is this a new and exciting way of using computers, but the hand gestures used to control the system are simple, which enables this computing system to be used by almost anyone.

With SmartSense you can also use Windows speech recognition which is powered by our intelligent virtual assistant called JARVIS. He got his name from the movie Iron Man. He is intelligent enough to search the internet for a word, say “Apple” when you ask him “What do you know about Apple?”.
JARVIS can recognize faces of people. You can turn on face recognition by telling “SCAN FACE”
Or “FACE RECOGNITION” or anything you like with almost the same meaning. If JARVIS is seeing the person for the first time, you will have the option to add the person’s name and profile to the database. If JARVIS knows the person before, the person’s profile will be projected on his body.
Modern society has quickly embraced the world of technology, and has openly accepted for the integration of computers in our everyday tasks. According to the International Telecommunication Union, close to 30% of the 2010 world population connects to the internet on a regular basis 1. With the ever increasing popularity of the internet, and the use of computers in our daily lives, developers are creating useful and manageable technology that is easily accessible whenever and wherever it is needed.
Our wearable computing system is being developed with the incorporation of a display and controls that can easily be seen and used without requiring a laptop or other cumbersome device. By wirelessly connecting a smart phone with a webcam and a pocket projector, we are creating a device that will be non-intrusive and non-restrictive. The projector will project a graphical user interface in front of the user, and the webcam will take in video of the user’s surroundings. With the webcam, the user will have the ability to control the system by using simple hand gestures. This wearable interface will allow the user to manage the different functions of the system and wirelessly communicate with the smart phone device that will never need to leave the user’s pocket. Our prototype can take many forms: a pendant worn around the neck, a clip on device that can be attached at numerous locations or even a single-strap backpack with the hardware embedded in the strap.
The practical applications of this system are many, for e.g., holding a news paper in front of the webcam followed by a click gesture will project the online version of the news (online news topics related to the news on the paper) on the news paper. This is done with the help of Optical Character Reading Technology. SmartSense can receive GPS data from your phone through Bluetooth and synchronize it using Google Map.

HARDWARE REQUIREMENTS
Laptop(Windows Vista or higher, 250 GB HDD, 2 GB RAM, Graphics card)
Pocket projector
Speaker
Microphone
Webcam
SOFTWARE
SmartSense application
Visual Studio 2008
.Net framework 3.5 (or higher)

SmartSense is currently looking for help from the codeplex community.So please contribute...

By
Fariz Rahman

Next release (3rd Feb 2013) SmartSense v2.0 for Windows 8:


Last edited Jan 26, 2013 at 5:22 AM by farizrahman4u, version 12