Multimodality in Karnatak music: Combining ethnographic, music-analytical and computational approaches

Music is fundamentally multimodal, with physical movement being a particularly prominent mode in musical experience. In this talk, I present an overview of my work examining sound and physical movement in Karnatak music, a style of art and devotional music performed primarily in South India. Karnatak music performance is a fruitful context for exploring movement dimensions of music due to the tendency for vocalists to gesture spontaneously as they sing. It is also a style with which I am personally familiar, having studied Karnatak violin in South India for several years before embarking on research.

The work presented encompasses sonic, kinematic and sociocultural analyses of performance practices in this tradition, using a combination of ethnographic, music-analytical and computational approaches. Through the ensuing discussion, I aim to highlight some of the epistemological and conceptual issues I have encountered when using computational methods. This includes an analysis of issues that arose while working with researchers at Pompeu Fabra University on a motif finding project using machine learning techniques. Here, my report is based on a collaboration with the Karnatak vocalist and musicologist Brindha Manickavasakan, in which we discuss ethical and conceptual issues relating to annotation and evaluation in this encounter between Karnatak music practices and MIR. In addition, I discuss approaches I have taken to combining ethnographic, music-analytical and computational methods with the goal of better understanding performers’ use of gesture in vocal music performance and pedagogy.