What does the neural representation of a complex, temporally structured auditory event look like? When we listen to a symphony or a speech, what neural processes allow us to maintain a stable attentional focus? How do the requisite auditory representations form, how do they adapt to unexpected nuances, and how do they reorganize to accommodate structural change?

At DARMA, we address these questions with a special focus on music. Our theoretical goal is to develop models of music perception and performance that apply to naturally complex musical sequences, to explain how humans process the auditory complexities of the real world.The mathematical models that we develop are used to make predictions about behavioral and neural correlates of auditory representation, attention, and communication, which are then tested experimentally. Our experimental program concentrates on the perception and production of music, and simpler music-like sequences, using both behavioral and neuroimaging techniques. Behavioral experiments are currently being conducted in several areas, including 1) formation and stability of dynamic representations, 2) real-time tracking of temporally structured sequences, 3) the role of rhythm in attention, 4) the role of expectancy in auditory communication. Neuroimaging techniques (EEG, MEG and fMRI) are being used to measure temporal and spatial aspects of neural function in auditory perception and attention. This research will advance our basic understanding of auditory perception and attention by enhancing our knowledge of the role of structure in perceiving and attending to real-world events. The results have potentially wide applicability from the development of more robust computer algorithms for music processing and speech recognition, to deeper clinical understanding of recovery from neurologic trauma, such as aphasic stroke.

In one such model, the mental representation of an auditory event is captured as a self-organized, dynamic structure whose neural correlate is a spatiotemporal pattern of neural activity. This spatiotemporal structure enables anticipation of future events, and thus the targeting of perception and the coordination of action with external events. Both stability and flexibility properties of attention arise through nonlinearities in the underlying pattern-forming dynamics. Furthermore, the dynamic representation functions in auditory communication. It is known that transient stimulus fluctuations, such as intonation and rate changes observed in both speech and musical performance, communicate intention, emotion, and structural information. We hypothesize that these communicative gestures are recognized as deviations from temporal expectations embodied in the dynamic representation.