Brandon Carone | Human-Centered AI, Audio, and Music Researcher

Hi, I'm Brandon Carone.

Human-centered AI and audio researcher with a PhD in Cognition and Perception from NYU. I build human-grounded benchmarks, evaluation pipelines, and research prototypes for multimodal AI, music information retrieval, and cognitively informed recommendation systems. Based in San Diego and open to full-time applied research roles.

About

I am a human-centered AI and audio researcher with a PhD in Cognition and Perception from New York University. My work combines experimental design, music cognition, machine learning, and music information retrieval to study how people and AI systems perceive, remember, and discover music. I build controlled listening studies, human-grounded benchmarks, reproducible evaluation pipelines, and research prototypes that connect model behavior with human perception.

At NYU’s Music and Audio Research Laboratory, I created the MUSE Benchmark, a human-grounded perceptual audio evaluation framework containing 10 controlled listening tasks across pitch, melody, rhythm, harmony, and timbre. I composed and recorded 200 original stimuli, collected baseline data from 200 human listeners, and evaluated four state-of-the-art audio-capable AI systems to identify perceptual failure modes. The resulting paper was presented at ICASSP 2026. I also led a LogicLM-style prompting study using a subset of the tasks and the stimuli from the MUSE benchmark. This work presented as a poster at NeurIPS in 2025, as an oral presentation at the AAAI 1st International Workshop on Emerging AI Technologies for Music, and was published in the 2026 Proceedings of Machine Learning Research (PMLR).

Previously, I developed SoundSignature, an user-facing music app that combines music information retrieval and a custom LLM to analyze listeners’ favorite tracks and provide personalized, educational feedback about their musical preferences. I evaluated the system through a mixed-methods pilot study and refined its explanations, interface, and interaction flow using participant feedback.

I currently work as a Research Consultant at LoveMind AI, where I advise research on audio and music evaluation for multimodal LLM projects. My work includes benchmark design, experimental controls, model-evaluation workflows, and methods for distinguishing audio-grounded perception from text-only inference and pre-training priors. Previously, as a Research Scientist Intern at Deezer, I analyzed large-scale music streaming data and developed cognitively informed recommendation models incorporating familiarity, novelty, cognitive effort, and repeated exposure.

I am seeking full-time applied research opportunities in human-centered AI, multimodal and audio model evaluation, music information retrieval, recommender systems, personalization, and AI behavior.

Experience

LoveMind AI

Research Consultant, Human-Centered AI & Audio Evaluation

Advise research on audio and music evaluation strategy for multimodal LLM projects, including benchmark design, study controls, and methods for separating audio-grounded perception from text-only inference and pre-training priors.
Synthesize research on multimodal AI, music emotion recognition, listener preference, music information retrieval, and affective computing into actionable recommendations for experimental design, dataset creation, and model-evaluation workflows.

May 2026 – Present | Remote

Ripollés Lab, NYU

Graduate Student Researcher

Created the MUSE Benchmark, a human-grounded perceptual audio evaluation framework with 10 controlled listening tasks spanning pitch, melody, rhythm, harmony, and timbre; composed and recorded 200 original stimuli, collected human data from N=200 listeners, and evaluated four state-of-the-art audio-capable AI systems to identify perceptual failure modes.
Built reproducible evaluation pipelines for MUSE and related experiments, including standardized audio presentation, fixed prompting, repeated model runs, deterministic parsing, metadata logging, and F1 scoring to support comparisons across models, human listeners, tasks, and experimental conditions.
Developed SoundSignature, a user-facing MIR and LLM application that extracts acoustic and perceptual features from music to generate personalized music-preference insights; evaluated the system with a mixed-methods pilot study and refined explanations, interface elements, and interaction flows based on user feedback.
Designed and analyzed behavioral and fMRI studies testing how musical reward and computational auditory surprise shape long-term memory; modeled stimulus-level perceptual predictors and linked acoustic structure with listener ratings.

Sep 2021 – May 2026 | New York, NY

Deezer, Paris

Research Scientist Intern

Investigated the cognitive mechanisms underlying music discovery by integrating large-scale streaming data analyses with frameworks from cognitive science.
Developed and formalized models linking familiarity, novelty, and cognitive effort to listener engagement and memory, framing discovery as a gradual process rather than an instantaneous event.
Implemented predictive and neural models that incorporate user embeddings, familiarity decay, and session-level cognitive load to estimate the likelihood of a song being liked.
Co-authored a paper (in preparation) proposing a cognitively informed framework for music recommender systems that operationalizes mental load, familiarity decay, and the dynamics of repeated exposure.

Jul 2025 – Jan 2026 | Paris, Île-de-France, France

NEXVOX Technologies

Consultant

Guiding improvements in software that reads signals from a consumer EEG headset and feeds them to an adaptive ML algorithm to create meditative music from real-time brain activity.
Advising on research strategy for clinical studies exploring the psychological benefits of adaptive music generation.

2023 – 2024 | Remote

Veterans Medical Research Foundation

Research Associate

Conducted multimodal clinical assessments for a 20-year longitudinal study on veterans with TBI and PTSD, characterizing overall brain health, emotional well-being, and cognitive decline.
Administered comprehensive neuropsychological and clinical assessments (e.g., Clinician-Administered PTSD Scale for DSM-5, WAIS, Delis-Kaplan Executive Function System) to measure the prevalence and intensity of chronic mental health sequelae and comorbidities.
Managed sensitive behavioral and medical data securely within the Veterans Affairs portal, working directly with the PI to assist in data analysis and manuscript preparation for clinical audiences.

2019 – 2021 | San Diego, CA

Boutelle Lab, UCSD

Neuroimaging Operator

Investigated associative conditioning and cue-reward learning in children (ages 8–10), examining developmental mechanisms through which environmental cues influence behavior over time.
Analyzed individual variability in reward sensitivity to identify neurocognitive risk factors in high-risk pediatric cohorts.
Applied developmental psychology principles to guide child participants through longitudinal fMRI paradigms while maintaining data quality and prioritizing participant well-being.

Sep 2020 – Jul 2021 | San Diego, CA

Music Mends Minds, Inc.

Manager / Executive Assistant

Led a team of five in supporting the Founder/Director with operations, fundraising, research, and legal tasks for a nonprofit creating musical support groups for patients with neurodegeneration.
Implemented the Public Education and Awareness Platform and launched the “Meet the Expert” Podcast, now with 19 episodes.
Designed the website and managed Google Ads campaigns, securing a $10,000/month in-kind ads grant.

2016 – 2021 | Los Angeles, CA

Memory and Cognition Lab, UCLA

Research Associate

Completed an honors thesis examining false memories for print advertisements with a sample of 500+ participants from the local community and Amazon Mechanical Turk.
Received grant to conduct an independent research study investigating the effects of listening to different musical genres on memory formation.
Coded the experiment and analyzed data using JASP.

2016 – 2019 | Los Angeles, CA

Projects

            
              The MUSE Benchmark
            
              A benchmark for probing music perception and abstract, relational reasoning in humans and LLMs.
            
              Accomplishments
              
            Ran large-scale evaluations on both humans and models such as Gemini, Qwen, and Audio-Flamingo using zero-shot, few-shot, and chain-of-thought prompting paradigms.
Developed analysis scripts and GLMM-based statistical workflow to compare human and model performance across tasks and prompting strategies.
Tools: Python, R (lme4), HPC, audio-LLM APIs, GitHub

              LogicLM for Audio Models
            
              Evaluating logic-style prompting strategies to improve structured reasoning in audio-capable LLMs.
            
              Accomplishments
              
            Adapted LogicLM-style prompting to music perception tasks, comparing standard, chain-of-thought, and constrained reasoning prompts on the MUSE Benchmark.
Analyzed how different prompting schemes affect error types (e.g., rhythmic vs harmonic mistakes) and alignment with human judgments.
Built logging and evaluation pipelines to track per-task accuracy, confidence, and reasoning patterns across multiple models and seeds.
Tools: Python, JSON log parsing, Matplotlib, audio-LLM APIs, GitHub

SoundSignature
              Developed an app that integrates MIR with AI to analyze users' favorite songs.
            
            Accomplishments
            Developed SoundSignature, an interactive application that integrates MIR with AI to analyze users’ favorite songs and provide
personalized insights into their musical preferences
Tools: Python, Machine Learning, MIR, Natural Language Processing

Automated Jazz Chord Annotations
              Python tool for MIDI that labels jazz chords for dataset creation.
            
            Accomplishments
            Planning to use the tool to expand the CREMA Chord Recognition dataset with extended jazz chords.
Tools: Python, MIR, Music Analysis, Chord Recognition, MIDI

LSTM Neural Networks for fMRI Data
              Developed and optimized LSTM networks to analyze fMRI data.
            
            Accomplishments
            Used in NeuroMatch Academy to explore fMRI data and its applications in understanding brain activity patterns, such as classifying social vs. nonsocial interactions.
Tools: Python, PyTorch, Keras, Machine Learning, fMRI Analysis

Chord Similarity
              Modified CREMA model to explore alignment of human perception with the model's deep features.
            
            Accomplishments
            Implemented to assess how human perception of chord similarity aligns with the CREMA model's deep feature representations.
Collected and analyzed human similarity judgments on ii-V-I chord variations.
Tools: Python, Machine Learning, Tensorflow, Keras, Audio Analysis, Chord Recognition

Publications

Peer-Reviewed Articles & Proceedings

Carone, B. J., Roman, I. R., & Ripollés, P. (2026). The MUSE Benchmark: Probing Music Perception and Auditory Relational Reasoning in Audio LLMs. Accepted to the 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://arxiv.org/abs/2510.19055

Carone, B. J., Roman, I. R., & Ripollés, P. (2026). LLMs can read music, but struggle to hear it: An evaluation of core music perception tasks Published in the Proceedings of Machine Learning Research. https://proceedings.mlr.press/v303/carone26a.html

Groves, K., Farbood, M. M., Carone, B. J., Ripollés, P., & Zuanazzi, A. (2025). Acoustic features of instrumental movie soundtracks elicit distinct and mostly non-overlapping extra-musical meanings in the mind of the listener. Scientific Reports, 15, 2327. https://doi.org/10.1038/s41598-025-86089-6

Carone, B. J., & Ripollés, P. (2024). SoundSignature: What Type of Music Do You Like? Proceedings of the IEEE 5th International Symposium on the Internet of Sounds (IS2).

Murphy, D. H., Schwartz, S. T., Alberts, K. O., Siegel, A. L. M., Carone, B. J., Castel, A. D., & Drolet, A. (2023). Clinically studied or clinically proven? Memory for claims in print advertisements. Applied Cognitive Psychology, 37(5), 1085–1093.

Workshop Papers

Carone, B. J., Roman, I. R., & Ripollés, P. (2025). Evaluating Multimodal Large Language Models on Core Music Perception Tasks. NeurIPS 2025 Workshop AI for Music: Where Creativity Meets Computation. https://arxiv.org/abs/2510.22455

Under Review

Carone, B. J., & Ripollés, P. (2025). The Effects of Novelty and Abstract Reward on Memory Performance.

In Preparation

Carone, B. J., Sguerra, B., Escobedo, G., Tamm, Y. M., & Bonnin, G. (2025). Discovery-Oriented Music Recommendation: The Role of Cognitive Effort and Familiarity.

Carone, B. J., Abrams, E. B., & Ripollés, P. (2025). The Neural Mechanisms of Novelty and Abstract Reward in Memory Performance.

Rodríguez-Vázquez, R., Carone, B. J., Groves, K., Namballa, R., Zuanazzi, A. R., & Ripollés, P. (2025). Identification of Basic Emotions Through Language Rhythms.

Education

New York University

New York, NY

Degree: Doctor of Philosophy (PhD) in Cognition and Perception
Concentration: Quantitative Psychology
Graduation: May 2026

Advisor: Professor Pablo Ripollés, Music and Audio Research Lab (MARL)
Fellowship: Dean’s Doctoral Fellowship
Relevant Coursework: Deep Learning, Computational Cognitive Modeling, Music Information Retrieval, Auditory Perception, Time Series Analysis

University of California, Los Angeles

Los Angeles, CA

Degree: Bachelor of Science (BS) in Cognitive Science
Specialization: Computing
Graduation: June 2019

Honors: Graduated with Honors
Thesis: Clinically studied or clinically proven? Memory for claims in print advertisements

Talks & Media

Conference Presentations

Carone, B. J., Roman, I. R., & Ripollés, P. (2026). LLMs can read music, but struggle to hear it: An evaluation of core music perception tasks. Oral presentation at the AAAI 1st International Workshop on Emerging AI Technologies for Music. https://openreview.net/forum?id=hKE8tQzueC

Carone, B. J., Roman, I. R., & Ripollés, P. (2025). Evaluating Multimodal Large Language Models on Core Music Perception Tasks. Poster presented at the NeurIPS 2025 Workshop AI for Music: Where Creativity Meets Computation. Poster

Carone, B. J., & Ripollés, P. (2024, October). SoundSignature: What Type of Music Do You Like? Oral presentation at the 5th annual IEEE International Symposium on the Internet of Sounds (IS2 2024), International Audio Laboratories, Erlangen, Germany.

Carone, B. J., Abrams, E. B., & Ripollés, P. (2023, November). The Effects of Novelty and Abstract Reward on Memory Performance. Poster presented at the Society for Neuroscience, Washington, D.C.

Carone, B. J., Merritt, V. C., Jurick, S. M., & Jak, A. J. (2021, February). Effects of Major Depressive Disorder on Veterans. Poster presented at the International Neuropsychological Society, San Diego, CA.

Carone, B. J., Siegel, A. L. M., Castel, A. D., & Drolet, A. (2019, May). False memory for print advertisements. Poster presented at multiple conferences.

Invited Talks

Carone, B. J. (2025, October). The MUSE Benchmark: Probing Music Perception and Auditory Relational Reasoning in Audio LLMs. Invited speaker in Marcus Pearce’s Lab Meeting at the Queen Mary University of London.

Carone, B. J. (2025, October). The MUSE Benchmark: Probing Music Perception and Auditory Relational Reasoning in Audio LLMs. Guest Lecturer for the Artificial Intelligence course at the Queen Mary University of London.

Carone, B. J. (2025, October). The MUSE Benchmark: Probing Music Perception and Auditory Relational Reasoning in Audio LLMs. Invited speaker for the Machine Listening Group at the Queen Mary University of London.

Carone, B. J. (2025, September). Modeling Successful Music Discovery. Guest Lecturer at Deezer in Paris, France.

Carone, B. J. (2025, August). Do You Hear What I Hear? Music Perception in Minds and Machines. Guest Lecturer at Deezer in Paris, France.

Carone, B. J. (2024, July). SoundSignature: What Type of Music Do You Like? Guest Lecturer at the Deep Learning for Music Information Retrieval II: State-of-the-Art Algorithms Workshop at the Center for Computer Research in Music and Acoustics at Stanford University.

Carone, B. J. (2024, October). Music and AI: Theoretical and Practical Perspectives. Guest Lecturer at Queen Mary University of London.

Media

Carone, B. J., Zatorre, R. J. (Interviewees) & Zhu, X. (Host). (2022, October 6). Cognitive Neuroscience of Music and Memory [Audio podcast]. Research Journey Initiative.

Carone, B. J. (Interviewee) & Bowes, P. (Host). (2019, April 4). Why music helps us age better [Audio podcast]. Live Long and Master Aging Podcast.

Carone, B. J., Rosenstein, C. P. (Interviewees) & Sharp, R. (Producer). (2018, October 10). Music Mends Minds [Radio show]. BBC Radio 5 Live’s Up All Night with Rhod Sharp.

Skills

Research and Technical Expertise

Perceptual Evaluation: Psychoacoustics, human-subject listening studies, experimental design, inferential statistics, survey design, mixed-methods research, and human baseline collection

AI and Machine Learning: PyTorch, TensorFlow/Keras, scikit-learn, statistical modeling, multimodal LLM evaluation, model-behavior analysis, error analysis, and reproducible evaluation pipelines

Audio and Music Information Retrieval: librosa, Essentia, madmom, audio and text pipelines, audio embeddings, retrieval, perceptual similarity, music analysis, and digital signal processing

Programming: Advanced Python; proficient R and MATLAB; working knowledge of C++, HTML, JavaScript, and SQL