Physical Computing Final: ASL GLOVE
Fall 2005

HUI Premise: We are making a glove controller interface that will allow a user to control a computer with American Sign Language finger spelling. Our prototype interface will allow a deaf/mute user to sign letters that will then be spoken by the computer so that a blind person could hear what was being signed.

Observations: We are attaching sensors to a glove and having people sign the ASL alphabet. We have a chart for those who don't already know ASL.

Technical Development:

We tried using flex sensors on all 5 digits of the right hand and found some success. From the flex sensors we could gather and accurately establish better than half the alphabet with all users and up to 20 letters with some users.

We then added more sensors to help differentiate between several letters that have similar flex sensor readings. We were able to get accurate readings for 21 letters with almost all adult users. Letters P and Q will require an additional sensor. The distinction between U and V will require another sensor, for now we are just using U. J and Z create another challenge that we did not get to this term. They are the same finger positions as I and D but involve gestural motion. Detecting J and Z will require more sensors and perhaps a different approach to sensing letters all together.

Here are the Max Patches and other resources we are using:
aaSignLanguageGlove.mxb
ASL.mxb
lang.asl.voice~.mxb
voice_asl.aiff.aif

letterDataSpreadSheet.xls

Documentation and Reflection

No matter how simple an idea for Physical Computing seems at first, its actual successful implementation takes more time and energy than 4 students have to expend in an entire year. Yet we go ahead and try to do just that in 6 weeks. By the end of term we had a glove that could detect 21 ASL letters. With another week to develope and test we could probably get 23 or 24 letters. All 26 would take another term. User testing pointed out many weaknesses in the physical device and software that could be inspiration for improved versions. Some of us may continue working on aspects of this project, for example in an instrument design or assistive technology class...

This was a really fun project and a great group to work with. I would like to thank the group for making this project happen, none of us could have done this alone. I would also like to thank Andrew Milmoe for his patient guidance in the Pcomp class. He provided us with a nurturing and focused environment in which we could pursue and present our ideas.

More info on the ASL glove can be found by following these links:
Steve's Blog has more pix, background info on ASL and the results of user testing.
Terence's Blog has a quicktime clip of our power point presentation on the glove.
Adam's Blog has some ideas on how to improve this project and video documentation of user tests.