Technology Review: Cell Phones That Listen and Learn

Description

Researchers are increasingly using cell phones to better understand users' behavior and social interactions. The data collected from a phone's GPS chip or accelerometer, for example, can reveal trends that are relevant to modeling the spread of disease, determining personal health-care needs, improving time management, and even updating social-networks. The approach, known as reality mining, has also been suggested as a way to improve targeted advertising or make cell phones smarter: a device that knows its owner is in a meeting could automatically switch its ringer off, for example.

Now a group at Dartmouth College, in Hanover, NH, has created software that uses the microphone on a cell phone to track and interpret a user's activity. The software, called SoundSense, picks up sounds and tries to classify them into certain categories. In contrast to similar software developed previously, SoundSense can recognize completely unfamiliar sounds, and it also runs entirely on the device. SoundSense automatically classifies sounds as "voice," "music," or "ambient noise." If a sound is repeated often enough or for long enough, SoundSense gives it a high "sound rank" and asks the user to confirm that it is significant and offers the option to label the sound.

The Dartmouth team focused on monitoring sound because every phone has a microphone and because accelerometers provide only limited information. "When we think about sounds, we don't typically think that they can also represent a location that has a unique signature," says Andrew Campbell, a professor of computer science at Dartmouth and a lead researcher on the project. The researchers made sure the program is small, so that it doesn't use too much power. To address privacy concerns, they designed SoundSense so that information is not removed from the device for processing. Additionally, the program itself doesn't store raw audio clips. A user can also tell the software to ignore any sounds deemed off limits.

Original URL

http://beta.technologyreview.com/communications/22907/

http://beta.technologyreview.com/communications/22907/