Deriving mental and emotional states from movement co-ordination patterns

A number of commercial software exists that allow deriving emotions of users based on their facial expressions (e.g. a smiley face is typically interpreted as a happy state, whereas a frowning face is often seen as representing angry or sad state). While body language is also often said to be representative of someone’s mental or emotional state, less is understood about the mapping between the two, and no automatic system exists that can annotate movement sequences with their emotional meanings in real time. At the same time, such a system would be useful in many settings such as mental health therapy sessions, sports and performing arts, interpersonal communication, and security/forensic applications.