Software Developer Teaches Alexa To Read Sign Language

Software Developer Teaches Alexa To Read Sign Language

I recently purchased an Amazon Dot Echo in the recent Prime sale. Well, it was only £29.99. How could I say no? Since owning it I have learned to appreciate that it is an exceptionally clever piece of technology. I have also possibly learnt that while it might be a Smart device, very little in my house is.

One person has, however, found an amazing use for Alexa. Software developer Abhishek Singh has found a way to mod the Alexa to actually read sign language!

How Does It Work?

Through the use of an Alexa linked to a laptop with a webcam, he has given the Alexa a whole new skill. In it, through the use of the webcam, Alexa is able to read and understand sign language and respond in both written text and in audio. Admittedly, the latter isn’t much use if you are deaf, but it’s still pretty amazing.

This could provide a fantastic gateway for those who communicate through sign language. As can be seen in the video above, he is able to ask it the time and even order shopping with Alexa showing an unnerving accuracy in its response. For someone who is hearing impaired, this could provide a fantastic gateway.

Pretty Incredible Stuff!

Abhishek Singh was able to design the software using Google’s TensorFlow software. With it, he manually taught it to sign language based on his body movement in the webcam and as can be seen, it worked far better than most people would have expected.

Considering that I’m learning very basic sign language for my son, I must admit, I’m impressed. I daresay it learnt it faster than I am.

What do you think? Are you impressed with this mod? – Let us know in the comments!