Mobile Computing: emotional

Wednesday, 19 December 2012

Now the mobile phone goes emotional

Oct. 25, 2012 — ForcePhone is a mobile synchronous haptic communication system. During phone calls, users can squeeze the side of the device and the pressure level is mapped to vibrations on the recipient's device. Computer scientists from University of Helsinki indicate that an additional haptic channel of communication can be integrated into mobile phone calls using a pressure to vibrotactile mapping with local and remote feedback. The pressure/vibrotactile messages supported by ForcePhone are called pressages.

Mobile devices include an increasing number of input and output techniques that are currently not used for communication. Recent research results by Dr Eve Hoggan from HIIT / University of Helsinki, Finland, however, indicate that a synchronous haptic communication system has value as a communication channel in real-world settings with users that express greetings, presence and emotions through presages.

-Pressure and tactile techniques have been explored in tangible interfaces for remote communication on dedicated devices but until now, these techniques have not been implemented on mobile devices or been used during live phone calls, says Eve Hoggan.

Using a lab based study and a small field study, Doctor Hoggan and her co-workers show that haptic interpersonal communication can be integrated into a standard mobile device. The new non-verbal design was also appreciated.

-When asked about the non-verbal cues that could be represented by pressages, the participants in our study highlighted three different approaches: to emphasize speech, express affection and presence, and to playfully surprise each other, she says.

When asked about the specific ways in which they adapted their communication style to accommodate the tactile modality, all of the participants stated that they tended to pause briefly after sending a pressage to 'make space for it in the conversation'.

According to the longitudinal study results the participants' phone calls lasted on average 4 minutes and 43 seconds with an average of 15.56 pressages sent during each call. All phone calls involved the use of pressages.

The prototype developed in this research, ForcePhone, is an augmented, commercially available mobile device with pressure input and vibrotactile output. ForcePhone was built at the Helsinki Institute of Information Technology and Nokia Research Center, Finland.

The research paper Pressages: Augmenting Phone Calls with Non-Verbal Messages by Eve Hoggan, Craig Stewart, Laura Haverinen, Giulio Jacucci and Vuokko Lantz was presented at the ACM Symposium on User Interface Software and Technology UIST'12 in Boston, MA, USA, October, 2012.

Share this story on Facebook, Twitter, and Google:

Other social bookmarking and sharing tools:

Story Source:

The above story is reprinted from materials provided by University of Helsinki, via AlphaGalileo.
Note: Materials may be edited for content and length. For further information, please contact the source cited above.

Note: If no author is given, the source is cited instead.

Disclaimer: Views expressed in this article do not necessarily reflect those of ScienceDaily or its staff.

View the original article here

Smartphones might soon develop emotional intelligence: Algorithm for speech-based emotion classification developed

Dec. 4, 2012 — If you think having your phone identify the nearest bus stop is cool, wait until it identifies your mood. New research by a team of engineers at the University of Rochester may soon make that possible. At the IEEE Workshop on Spoken Language Technology on Dec. 5, the researchers will describe a new computer program that gauges human feelings through speech, with substantially greater accuracy than existing approaches.

Surprisingly, the program doesn't look at the meaning of the words. "We actually used recordings of actors reading out the date of the month -- it really doesn't matter what they say, it's how they're saying it that we're interested in," said Wendi Heinzelman, professor of electrical and computer engineering.

Heinzelman explained that the program analyzes 12 features of speech, such as pitch and volume, to identify one of six emotions from a sound recording. And it achieves 81 percent accuracy -- a significant improvement on earlier studies that achieved only about 55 percent accuracy.

The research has already been used to develop a prototype of an app. The app displays either a happy or sad face after it records and analyzes the user's voice. It was built by one of Heinzelman's graduate students, Na Yang, during a summer internship at Microsoft Research. "The research is still in its early days," Heinzelman added, "but it is easy to envision a more complex app that could use this technology for everything from adjusting the colors displayed on your mobile to playing music fitting to how you're feeling after recording your voice."

Heinzelman and her team are collaborating with Rochester psychologists Melissa Sturge-Apple and Patrick Davies, who are currently studying the interactions between teenagers and their parents. "A reliable way of categorizing emotions could be very useful in our research,." Sturge-Apple said. "It would mean that a researcher doesn't have to listen to the conversations and manually input the emotion of different people at different stages."

Teaching a computer to understand emotions begins with recognizing how humans do so.

"You might hear someone speak and think 'oh, he sounds angry!' But what is it that makes you think that?" asks Sturge-Apple. She explained that emotion affects the way people speak by altering the volume, pitch and even the harmonics of their speech. "We don't pay attention to these features individually, we have just come to learn what angry sounds like -- particularly for people we know," she adds.

But for a computer to categorize emotion it needs to work with measurable quantities. So the researchers established 12 specific features in speech that were measured in each recording at short intervals. The researchers then categorized each of the recordings and used them to teach the computer program what "sad," "happy," "fearful," "disgusted," or "neutral" sound like.

The system then analyzed new recordings and tried to determine whether the voice in the recording portrayed any of the known emotions. If the computer program was unable to decide between two or more emotions, it just left that recording unclassified.

"We want to be confident that when the computer thinks the recorded speech reflects a particular emotion, it is very likely it is indeed portraying this emotion," Heinzelman explained.

Previous research has shown that emotion classification systems are highly speaker dependent; they work much better if the system is trained by the same voice it will analyze. "This is not ideal for a situation where you want to be able to just run an experiment on a group of people talking and interacting, like the parents and teenagers we work with," Sturge-Apple explained.

Their new results also confirm this finding. If the speech-based emotion classification is used on a voice different from the one that trained the system, the accuracy dropped from 81 percent to about 30 percent. The researchers are now looking at ways of minimizing this effect, for example, by training the system with a voice in the same age group and of the same gender. As Heinzelman said, "there are still challenges to be resolved if we want to use this system in an environment resembling a real-life situation, but we do know that the algorithm we developed is more effective than previous attempts."