“It’s very easy to get intimidated,” says Hamayal Choudhry, the robotics engineer who co-created the smartARM, a robotic hand prosthetic that uses a camera to analyse and manipulate objects. “You have this idea for a project, then think, I don’t know a thing about this.” Here’s how Choudhry and his partner Samin Khan, who programmed the smartARM’s machine learning algorithm, used code libraries, college assignments, and sponsored hackathons to find and execute a meaningful project at age 20.
Hamayal Choudhry and Samin Khan present the smartARM at Microsoft’s Imagine CupPhoto: Imagine Cup
Cross the streams
The smartARM works by integrating the two fields of machine learning and mechatronics (robotics). A camera in the palm detects objects, and an algorithm analyses the video feed (this is called computer vision) and tells the robotic hand how to manipulate the objects. The algorithm learns from each attempt, so the arm can be trained over time.
This is a huge advantage over current robotic prosthetics on the market, which rely on a direct connection to the user’s nerve endings. To control five fingers, says Choudhry, doctors have to find five different nerve endings to calibrate to those fingers. The patient needs surgery to get those nerves closer to the skin, or to train individual muscles. By bypassing some of this work, technology like the smartARM could dramatically reduce the cost of robotic prosthetics, making them affordable for far more patients. Current robotic prosthetics can cost up to $US100,000, says Choudhry. “They’re advanced, but who’s going to be able to afford them?”
To Choudhry and Khan’s knowledge, this is the first application of computer vision in a prosthetic hand. Khan feels there should be more of a push for integrating hardware projects into machine learning. If two students – neither of whom had any experience with prosthetics before last year – can make so much progress in such an advanced field, then there’s still a lot of unlocked potential in these combined technologies. By collaborating across fields, you’re much more likely to find new applications.
Get an assignment
It was a hackathon that gave Choudhry and Khan that push. While the two University of Toronto students had attended middle school together, they didn’t bump into each other until a hackathon – an event where people put together tech projects, often competing for a prize.
It was at one of these hackathons that Choudhry and Khan started building the smartARM. Like many hackathons, it was sponsored; Google and Microsoft presented their technologies and walked participants through them, in hopes that the participants would find interesting uses for them. The smartARM uses Microsoft Azure’s computer vision, machine learning, and cloud storage technologies. Choudhry and Khan won the hackathon, then won bigger and bigger events, eventually winning cash, a grant, and a mentoring session with Microsoft CEO Satya Nadella at Microsoft’s international Imagine Cup.
Choudhry and Khan entered these hackathons with a lot of knowledge. But much of it wasn’t from their undergrad courses; both had volunteered to help with professors’ research projects. With this research, says Choudhry, the professor often hands the student a goal, “and it’s pretty much up to you to reach that.” That taught him to be independent, and to go out and do his own projects.
As other tech professionals have told Lifehacker, the best way to learn a computer skill is by using it in a project. Coding and similar skills are about reaching certain goals, so find an interesting goal that could use the skills you’re trying to develop. “Find a project you really want to do,” says Choudhry. “For me, it’s asking why – why something is done the way it is right now.” If your project suits your interests and gets you excited, it can solve someone else’s need instead of yours. For the smartARM, Choudhry and Khan talked to amputees about their experiences and needs.
The kind of project that excites you might look intimidating. It might look impossible. But to really know, you have to break it into pieces.
Break down your project
Choudhry compares complex projects to building with Lego. Any complicated construction is still just tiny blocks attached to each other. And you don’t have to make those blocks yourself.
Khan points to the Pandas Python library, an open-source set of tools for analysing data with Python. Because they focus on recognising patterns and adapting, machine learning algorithms are surprisingly versatile across different applications. So all kinds of existing code might apply to your project.
Of course, first you need to learn to code. “You are definitely not limited by where you’re at in your career or education,” says Choudhry. To learn the basics, you can attend a boot camp or teach yourself. Choudhry recommends the free classes and tutorials at Coursera.
For robotics, Choudhry recommends starting with Arduino, an open-source platform and programmable electronic board that can control all kinds of robotics. You can get an Arduino-compatible kit for $US35 ($47) on Amazon or at a hobby store. Choudhry started his robotics education by learning how to control a simple motor with an Arduino.
Let’s say you wanted to build a robotic arm, he says. First you look at your material needs – you need an Arduino, motors, and a hand. That’s three smaller projects. Can you 3D print the parts of the hand? Can you program the motors with the Arduino, and make connections between the parts? Can you connect a camera to the Arduino, and plug in some machine learning algorithms that you’ve built from existing technologies? If you can’t yet, now you at least know which things you need to learn. Now you’ve got a very crude version of the smartARM, which you can tweak and tweak until those tweaks add up to a slick prototype.
And that’s where Choudhry and Khan are. They hope to develop smartARM into a market-ready product, but there’s still a long way to go. They need to do further research into whether their project really is what the market needs, or whether there’s a good reason no one has taken their approach before. “The worst thing,” says Khan, “would be if we move forward with this and find out it’s not a suitable product for people.” The history of technology is littered with promising inventions that couldn’t beat competing technologies. But every successful innovation was, at some point, a project hacked together out of the materials at hand.
Comments