Research at the Social and Affective Neuroscience Group (SANE)

Emotions in Music: How Naturalistic Music Listening Encodes Acoustic and Emotional Features in the Brain and Is Shaped by Extra-Musical Context

address: IMT, Lucca

A review of theoretical and empirical work on emotion in music, predictive coding and reward, acoustic and extra-musical cues, and the neuroimaging of ... read more

A review of theoretical and empirical work on emotion in music, predictive coding and reward, acoustic and extra-musical cues, and the neuroimaging of music and emotion. Motivates the use of naturalistic, extended musical stimuli and lays out the multi-level framework (encoding, connectivity, context) that structures the thesis. show less

NaPuCcoL Non-Parametric Combination for Group-Level Inference

address: IMT, Lucca
link: GitHub Repo

Introduces and validates NaPuCco, a Non-Parametric Combination framework for one-sample, unsigned fMRI statistics that preserves idiographic voxel-wise encoding ... read more

Introduces and validates NaPuCco, a Non-Parametric Combination framework for one-sample, unsigned fMRI statistics that preserves idiographic voxel-wise encoding patterns at the group level. Simulation studies show robust control of false positives and good power for voxel- and cluster-wise inference, and an application to naturalistic music listening recovers auditory cortices and right-hemisphere regions sensitive to low-level acoustic features. show less

Music and Emotion in Functional Connectivity

address: IMT, Lucca
link: GitHub Repo

Presents BOSS (Brain Orchestra Synchronization Study), a modular toolbox that uses searchlight-based canonical correlation analysis to disentangle ... read more

Presents BOSS (Brain Orchestra Synchronization Study), a modular toolbox that uses searchlight-based canonical correlation analysis to disentangle acoustic- vs emotion-related feature contributions to functional connectivity. During naturalistic music listening, the work reveals distinct connectivity fingerprints for acoustic and emotional features, including emotion-sensitive modulation of midbrain–thalamic pathways and amygdala-centered networks linked to valuation and affect. show less

A Listening Behavioural Study With Contextual Information Manipulation

address: IMT, Lucca
link: website

Uses large language models to generate and manipulate program-note style descriptions with different affective valence before symphonic listening ... read more

Uses large language models to generate and manipulate program-note style descriptions with different affective valence before symphonic listening in an online, naturalistic paradigm. Shows that contextual information systematically and bidirectionally biases continuous emotion ratings, categorical judgments, and textual feedback, demonstrating that programme notes act as an interpretative lens on the emotional trajectory of the music. show less

Segmentation and analysis of face expressions during naturalistic affective stimuli

Audio Enhancement - real time

Audio quality enhancement

Designed low-latency deep learning pipelines for real-time audio enhancement, including preprocessing, model architecture, evaluation workflows, and deployment-oriented constraints. Relevant to AI product development where latency, reliability, and user-facing quality matter.

Real time audio denoising

Designed low-latency deep learning pipelines for real-time audio denoising, including preprocessing, model architecture, and evaluation workflows.

Urban Soundscape index modelling

Bayesian-NN for individual soundscape assessment predictions and a perceptual index design, development, evaluation

A Bayesian modelling approach has been implemented to represent and analyse the uncertainty highlighted in the psychometric revisitation of the current standard soundscape ISO.

According to some commonly agreed criteria to u

Psychometric revisitation of ISO protocol for soundscape measuring and data collection protocol design

address: University College London, London
link: Pyscometric revistation of ISO protocol
link: Data collection protocol design

Likert scales are useful for collecting data on attitudes and perceptions from large samples of people. In particular, they have become a well-established tool in soundscape studies for conducting in situ surveys to determine how people experience urban public spaces. However, it is still ... read more

Likert scales are useful for collecting data on attitudes and perceptions from large samples of people. In particular, they have become a well-established tool in soundscape studies for conducting in situ surveys to determine how people experience urban public spaces. However, it is still unclear whether the metrics of the scales are consistently interpreted during a typical assessment task. The current work aims at identifying some general trends in the interpretation of Likert scale metrics and introducing a procedure for the derivation of metric corrections by analyzing a case study dataset of 984 soundscape assessments across 11 urban locations in London. According to ISO/TS 12913-2:2018, soundscapes can be assessed through the scaling of 8 dimensions: pleasant, annoying, vibrant, monotonous, eventful, uneventful, calm, and chaotic. The hypothesis underlying this study is that a link exists between correlations across the percentage of assessments falling in each Likert scale category and a dilation/compression factor affecting the interpretation of the scales metric. The outcome of this metric correction value derivation is introduced for soundscape, and a new projection of the London soundscapes according to the corrected circumplex space is compared with the initial ISO circumplex space. The overall results show a general non-equidistant interpretation of the scales, particularly on the vibrant-monotonous direction. The implications of this correction have been demonstrated through a Linear Ridge Classifier task for predicting the London soundscape responses using objective acoustic parameters, which shows significant improvement when applied to the corrected data. The results suggest that the corrected values account for the non-equidistant interpretation of the Likert metrics, thereby allowing mathematical operations to be viable when applied to the data. - From the abstract show less

Music recommendation system

Parametric t-SNE (ANN) for music recommendation system, playlist generation and browser GUI for online music streaming providers

address: Aalborg University, Copenhagen
link: VIMEO

This project presented the development of the model and a user interface for a music-space exploration based on the t-SNE dimension reduction technique, aiming at ... read more

This project presented the development of the model and a user interface for a music-space exploration based on the t-SNE dimension reduction technique, aiming at preserving the shapes and structure of a high dimensional dataset of songs, dictated by N-dimensional features vector, to its projection onto a plane. We investigate different models obtained from using different structures of hidden layers, pre-training technique, features selection, and data pre-processing. The resulting output model has been used to build a music-space of 20000 songs, visually rendered for browser interaction, providing the user a certain degree of freedom to explore it by changing the features to highlight, offering an immersive experience for music exploration and playlist generation. show less

Deep Learning for Music

Violin Vibrato modelling (SVM) and GUI on android rendering with real time pitch detection

address: MTG Pompeu Fabra Universitat, Barcellona
link: A Machine Learning Approach to Violin Vibrato Modelling in Audio Performances and a Didactic Application for Mobile Devices
link VIMEO

We present a machine learning approach to model vibrato in classical music violin audio performances. A set of descriptors have been extracted from the music scores of the performed pieces and used to train a model for classifying notes into vibrato or non-vibrato, as well as for predicting the performed vibrato amplitude and frequency. ... read more

We present a machine learning approach to model vibrato in classical music violin audio performances. A set of descriptors have been extracted from the music scores of the performed pieces and used to train a model for classifying notes into vibrato or non-vibrato, as well as for predicting the performed vibrato amplitude and frequency. In addition to score features we have included a feature regarding the fingering used in the performance. The results show that the fingering feature affects consistently the prediction of the vibrato amplitude. Finally, an implementation of the resulting models is proposed as a didactic real-time feedback system to assist violin students in performing pieces using vibrato as an expressive resource. show less

LSTM modelling for melody and rhythmic structure extraction and generation

address: Aalborg University, Copenhagen

One of the most suggestive concept in artificial intelligence applied to music, shown in several recent studies, is the concept of style. Even if the definition of style is hard to be explicated without considering also historical and social contexts, from a basic overview we can think about it as the sequence of patterns composing the structure of a music piece. The style could be thought indeed as a particular pattern hidden ... read more

One of the most suggestive concept in artificial intelligence applied to music, shown in several recent studies, is the concept of style. Even if the definition of style is hard to be explicated without considering also historical and social contexts, from a basic overview we can think about it as the sequence of patterns composing the structure of a music piece. The style could be thought indeed as a particular pattern hidden in a sequence of symbols describing a music work, doing so it will be easy to manage the information represented by the structure containing the symbols, using them in order to manipulate the style itself or to replicate it. Making a deeper step into this subject we can separate the main goal in two different problems. One first problem is, given an audio support from which extract the style information, find an automatic way to reduce the original audio to a stream of symbols in which the style is encoded. The second problem concerns using this stream of symbols to decode the style they are carrying; in order to do that it is introduced a support structure capable to decode and memorize the piece structure that will be re-used to compose new music sequences referring to the same style of the original one. The purpose of this study is the evaluation of Long Short Term Memory network, for music generation from a percussive sequence as an example. The sequence segmented and symbolized through a single-linkage algorithm based on MFCC analysis, and the fed into the network. Then the network is trained with the analyzed data and gains the ability to generate new percussive sequences, according to the example. The results are compared with the previously implemented method of Variable Length Markov Chain models for music generation. show less

Variational Autoencoder for sounds morphing

address: Aalborg university
link: End-To-End_Dilated Variational Autoencoder with Bottleneck Discriminative Loss for Sound Morphing

This project was developed with tensorflow 1.4 where probabilistic inference was not yet available. Two strategies for end-to-end variational autoencoders (VAE) for sound morphing are compared: VAE with ... read more

This project was developed with tensorflow 1.4 where probabilistic inference was not yet available. Two strategies for end-to-end variational autoencoders (VAE) for sound morphing are compared: VAE with dilation layers (DC-VAE) and VAE only with regular convolutional layers (CC-VAE). The training strategy used a combination of the following loss functions: 1) the time-domain mean-squared error for reconstructing the input signal, 2) the Kullback-Leibler divergence to the standard normal distribution in the bottleneck layer, and 3) the classification loss calculated from the bottleneck representation. On a database of spoken digits, 1-nearest neighbor classification was used to show that the sound classes separate in the bottleneck layer. We introduce the Mel-frequency cepstrum coefficient dynamic time warping (MFCC-DTW) deviation as a measure of how well the VAE decoder projects the class center in the latent (bottleneck) layer to the center of the sounds of that class in the audio domain. In terms of MFCC-DTW deviation and 1-NN classification, DC-VAE outperforms CC-VAE. These results limited to the current parametrization and the dataset indicate that DC-VAE is more suitable for sound morphing than CC-VAE, since the DC-VAE decoder better preserves the topology when mapping from the audio domain to the latent space. show less

Artistic augmented space

An Interactive Soundscape augmented space

Address: University of Padua
link: VIMEO

A sonic augmented physical space through granular synthesis: this installation was inspired by Truax's Entrance to the Harbour, from The Vancouver Soundscape 1973 and Pacific Fanfare from The Vancouver Soundscape 1996. It reproduced a collage of sounds referring to a real soundscape. The sounds were spacialized in ... read more

... read more

A sonic augmented physical space through granular synthesis: this installation was inspired by Truax's Entrance to the Harbour, from The Vancouver Soundscape 1973 and Pacific Fanfare from The Vancouver Soundscape 1996. It reproduced a collage of sounds referring to a real soundscape. The sounds were spacialized in a room according to the users's detected position by a camera set on the top of the room. The user had the possibility to explore both the spatial structure of the soundscape and the acoustic structure of its sounds by entering in a central area of the room, where a granular decomposition of the sounds was applied. Launched in the late '70s at the Simon Fraser University, Soundscape Composition is a set of composite strategies working on sound environment. Traditionally linked with granular synthesis, it uses electronic music tools to elaborate environmental sounds, preserving their original contexts. The current state of the art technologies allowed the design and the development of an interactive environment inspired by soundscape composition, in which a user can explore a sound augmented reality referring to a real soundscape. The soundscape exploration occurs on two different layers: on a first layer the user can spatially explore the soundscape, while on a second layer an exploration of the structural composition of the sounds that build the soundscape takes place. This second kind of exploration happens through a granular analysis of the sounds. The user moving through the installation modifies the synthesis and sound dynamic parameters, building a cognitive structure of the augmented environment. The sound feedback of the environment modifies user's awareness and, consequently, his decisions on how to move within it. show less

Environmental Sonification

address: Aalborg University, Copenhagen
link: YOUTUBE

Competitive workshop for developing solutions to the new Lighting-Sound system at the AAU University bridge - winning project