Technology at play: Improving people’s lives | HP Innovation Journal

Ticky Thakkar
VP, HP Fellow, Director of the Emerging Compute Lab and Personal Systems Chief Technologist, HP

Tom Paddock
CEO, Sound Research® Corporation


In 1985, Stephen Hawking—the famous English theoretical physicist, cosmologist, and author—lost his ability to speak following an emergency tracheotomy. Since then, Professor Hawking has relied on computer hardware that turns text into speech to be able to communicate.

At a conference in 1997, Intel’s co-founder Gordon Moore met the Professor and promised him that Intel would provide computer voice assistance and wheelchair control technology in perpetuity. Intel has been designing custom computer and audio systems for Professor Hawking’s wheelchair ever since.

In 2003, Ticky Thakkar (now Head of the Emerging Compute Lab and Chief Technologist of Personal Systems at HP), was Chief Systems Architect with Intel. Early that summer Ticky was contacted by Professor Stephen Hawking’s assistant about two issues with the Professor’s voice equipment.

The first and most pressing issue was portability. The voice equipment was bulky and was powered by his wheelchair battery and various power supplies. Consequently, whenever he was away from the chair, he was without any means of voice communication. This situation was exceptionally acute when Professor Hawking had to fly for any length of time, as the wheelchair was stored in the hold of the airplane.

The second issue concerned the lack of replacement "current" technology for the hardware voice synthesizer. Professor Hawking’s distinctive synthesized voice had been created in the early 1980s by Dennis Klatt, an engineer at MIT, but in the intervening years since Professor Hawking began using his "CallText 5010" synthesizers, most of the electronic components were no longer being manufactured and Hawking’s three remaining voice synthesizers could not be repaired if they failed for any reason.

Over the next several months, Ticky’s team spent countless hours working on a variety of software solutions utilizing the latest technology. The majority of their time was spent trying to overcome the hardest challenge of all: reproducing Hawking’s signature voice.

Finally, after almost a year of effort, Ticky was ready to present his team's work to Professor Hawking in person.

Those were the first distinctive words heard by Ticky when he finally met Professor Hawking for the first time in the spring of 2004.

"I remember it as if it was yesterday," Ticky said. "It was a beautiful spring day. The sun was shining, the air crisp and clear. I was so excited to finally meet Professor Hawking face-to-face and review my team's efforts with him that I could barely drive the speed limit. I came away from that meeting deeply impressed, he was such a down-to-earth guy. He inspired me."

"Overall the meeting went well," Ticky recalled, "Professor Hawking was encouraged and commented that our efforts were 'close', however there was still an opportunity to do better."

Since then, Ticky has continued to think about Professor Hawking—and be on the watch for any emerging technology that could provide viable solutions.

In 2013, Professor Hawking asked Intel to provide him with a next-generation, portable speaker design that would enable him to give understandable lectures to larger audiences, easily talk with friends and family at social gatherings, and provide (as Hawking later said to Tom Paddock, CEO of Sound Research) "…sufficient decibels to be heard at a cocktail party—although perhaps not Glastonbury!" In response, David Rittenhouse and Travis Bonifield of Intel contacted Sound Research with a challenge: collaborate with Intel to create a far-field voice speaker that would: mount unobtrusively on Hawking’s existing wheelchair; be able to amplify Hawking’s voice at sound levels approaching 100 decibels, adjust the sound field to match Hawking’s exact audio quality "ear-tuning" requirements, project Hawking’s voice with a hemispherical dispersion pattern (or "2 π") and run using the same voltage as a PC. Without hesitation, Sound Research accepted Intel’s challenge.

Paddock had been working on a next-generation portable speaker with HP’s Jon Dory, Program Manager, called "HP Roar+" which featured Sound Edge^® tuning and smart amplifier optimization software running on the HP chipset. This technology could be easily ear-tuned by Professor Hawking to his own sonic tastes in real time. The main challenge was reaching Hawking’s 100 decibel loudness level, portability requirements, and wide voice dispersion angle (or "2π") goals.

Dr. Charles Chin, Senior Audio Architect for Sound Research, performed extensive power supply and custom processor modifications to the Roar+ system. Paul Kitano and Kristin Kosak, Sound Research Audio Systems Engineers, doubled Roar’s two speakers to four to achieve Hawking’s far-field loudness requirement and Dave Rittenhouse created a speaker industrial design to achieve Professor Hawking’s 2π dispersion goals. Their efforts paid off; Hawking’s 2π voice speaker now measured 99.7 decibels at 0.5 meters with exceptional far-field vocal clarity and 2π sound field dispersion. Combining the HP Roar+ technologies with updated industrial and acoustic designs from Intel and Sound Research, Hawking’s 2π voice speaker was quite loud at a great distance and over a wide sound stage. The speaker was now ready for Professor Hawking’s ear-tuning.

Tom Paddock and Sue Stallcup, Administrator for Sound Research, traveled to Cambridge, UK to meet with Professor Hawking and his local support team which included Jonathan Wood, Teaching Assistant to Professor Hawking, and Mark Green, Technical Marketing En-gineer for Intel Corporation. Two PC computers were connected to Professor Hawking's 2π voice speaker. One computer adjusted the Sound Edge^® and smart amp optimization software to Professor Hawking's exact sonic taste. The other computer adjusted the smart amplifier with custom HP speaker drivers. Professor Hawking supervised the ear tuning session by advising our team as to his perception of the 2π voice speaker's tonal balance and overall loudness, although Hawking's high frequency perception was challenged by an ear infection during our tuning session.

Sound Research achieves Hawking voice loudness goal with excellent clarity at 99.7dB

Later on,when Professor Hawking recovered from his ear infection, his aide requested that Sound Research set up a second ear-tuning session in Cambridge. Dr. Chin traveled from Taipei City, Taiwan to Cambridge, UK to readjust the tonal balance for the 2π voice speaker.

The tuned 2π voice speaker was installed onto Professor Hawking’s wheelchair. Unfortunately, an electrical hum from the wheelchair system was amplified by the voice system. After Paddock added a Jensen isolation transformer and Canare Star-Quad audio cabling to decouple the noisy audio system components, the hum completely disappeared from the voice speaker. Paddock had previously designed similar circuits for the Grateful Dead’s Jerry Garcia and Bob Weir as well as for Mickey Hart’s percussion instrument known as "The Beam."

In the summer of 2016, Ticky and Tom met for the first time. As they swapped stories and learned of each other’s efforts over the years to help Professor Hawking, a bond formed. "Ticky and I have had a unique set of experiences," said Paddock. Both men agreed that giving Professor Stephen Hawking a clear, portable voice was one of the most challenging and rewarding tasks they had ever undertaken.

Professor Hawking’s 2π voice speaker algorithm running in legacy voice synthesizers since the 1980s was transferred to a pure software algorithm running on an Intel processor within a PC. This upgrade, along with the ultra-tunable and portable Reality Amplifier system, designed by Sound Research's Dr. Chin, Paul Kitano and Kristin Kosak, will eventually provide Professor Hawking with his distinctive voice in a fully-portable system that will allow him to communicate without requiring his wheelchair.

Intel is now making Hawking's voice technology available to the public. This technology is called "Assistive Context-Aware Toolkit" or ACAT.

Stephen Hawking and Tom Paddock at University of Cambridge during their first 2π voice speaker mastering session

Intel decided to make the source code freely available so that anyone with a Windows PC and a webcam for tracking facial movements can modify the code in an effort to help Professor Hawking and others with similar disabilities. Ticky and Tom have shared important contributions toward the evolution of ACAT technology from the early days of "walled garden", one-off hardware solutions of the 1980s to the PC-enabled, Windows-based programmable solutions of the 1990's and early 2000's, and finally to today’s "open source" development environments—making possible rapid, differentiated, and powerful solutions to address disabilities in the future.

"Our hope is that, by open sourcing this configurable platform, developers will continue to expand on this system by adding new user interfaces, new sensing modalities, word prediction, and many other features" said Sai Prasad, Intel’s Program Manager for ACAT.