This page is not as often updated as it should be —> see Google Scholar for an up-to-date list & GitHub.Current Preprints:
Modeling Sensorimotor Processing with Physics-Informed Neural Networks
A Perez Rotondo, A Marin Vargas, M Dimitriou, A Mathis
bioRxiv, 2024.09. 14.613030
End-to-End Trainable Multi-Instance Pose Estimation with Transformers
L Stoffl, M Vidal, A Mathis arXiv:2103.1211522021 (this was the first end-to-end trained pose estimation model; many followed the approach later)
Selected Peer-reviewed publications:
Decoding the brain: From neural representations to mechanistic models
MW Mathis, AP Rotondo, EF Chang, AS Tolias, A Mathis
Cell 187 (21), 5814-5832
Acquiring musculoskeletal skills with curriculum-based reinforcement learning
AS Chiappa, P Tano, N Patel, A Ingster, A Pouget, A Mathis
Neuron and bioRxiv, 2024
Elucidating the Hierarchical Nature of Behavior with Masked Autoencoders
L Stoffl, A Bonnetto, S d'Ascoli, A Mathis
ECCV (in press) - bioRxiv, 2024.08. 06.606796
Task-driven neural network models predict neural dynamics of proprioception
A Marin Vargas*, A Bisi*, AS Chiappa, C Versteeg, LE Miller, A Mathis
Cell 2024 and bioRxiv, 2023
SuperAnimal models pretrained for plug-and-play analysis of animal behavior
S Ye, A Filippova, J Lauer, M Vidal, St Schneider, T Qiu, A Mathis, MW Mathis
Nature Communications 2024
WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models
V Gabeff, M Russwurm, D Tuia, A Mathis
International Journal of Computer Vision (in press). bioRxiv, 2023.12. 22.572990. Earlier version: CVPR CV4animals 2023 (oral)
ODEFormer: Symbolic Regression of Dynamical Systems with Transformers
S d'Ascoli, S Becker, P Schwaller, A Mathis, N Kilbertus
ICLR (spotlight) 2024. arxiv 2023
Rethinking pose estimation in crowds: overcoming the detection information-bottleneck and ambiguity
M Zhou*, L Stoffl*, MW Mathis, A Mathis
ICCV '23 / arXiv preprint arXiv:2306.07879 / also CV4Animals@CVPR 2023
AmadeusGPT: a natural language interface for interactive animal behavioral analysis
S Ye, J Lauer, M Zhou, A Mathis, MW Mathis
NeurIPS 2023
Latent Exploration for Reinforcement Learning
Albert Silvio Chiappa, Alessandro Marin Vargas, Ann Zixian Huang, A Mathis
NeurIPS 2023
Contrasting action and posture coding with hierarchical deep neural network models of proprioception
Kai J. Sandbrink*, Pranav Mamidanna*, Claudio Michaelis, Matthias Bethge*, Mackenzie W. Mathis*, Alexander Mathis*
eLife 2023, BioRxiv 2020 and project website
DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Body
Alberto Silvio Chiappa, Alessandro Marin Vargas, Alexander Mathis. NeurIPS, 2022.
Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction
Korleki Akiti, Iku Tsutsui-Kimura, Yudi Xie, Alexander Mathis, Jeffrey Markowitz, Rockwell Anyoha, Sandeep Robert Datta, Mackenzie Weygandt Mathis, Naoshige Uchida, Mitsuko Watabe-Uchida, Neuron, 2022. BioRxiv 2021
Multi-animal pose estimation, identification and tracking with DeepLabCut
Jessy Lauer, Mu Zhou, Shaokai Ye, William Menegas, Tanmay Nath, Mohammed Mostafizur Rahman, Valentina Di Santo, Daniel Soberanes, Guoping Feng, Venkatesh N Murthy, George Lauder, Catherine Dulac, Mackenzie W Mathis*, Alexander Mathis*. Nature Methods, 2022. Preprint: biorxiv
Seeing biodiversity: perspectives in machine learning for wildlife conservation
Devis Tuia, Benjamin Kellenberger, Sara Beery, Blair R Costelloe, Silvia Zuffi, Benjamin Risse, Alexander Mathis, Mackenzie W Mathis, Frank van Langevelde, Tilo Burghardt, Roland Kays, Holger Klinck, Martin Wikelski, Iain D Couzin, Grant van Horn, Margaret C Crofoot, Charles V Stewart, Tanya Berger-Wolf
Nature Communications, 2022 Preprint: arXiv:2103.1177512021
Measuring and modeling the motor system with machine learning
SB Hausmann, AM Vargas, A Mathis, MW Mathis
Current Opinion in Neurobiology Volume 70, October 2021, Pages 11-23 Preprint: arXiv:2103.1177512021
Perspectives on individual animal identification from biology and computer vision
Maxime Vidal, Nathan Wolf, Beth Rosenberg, Bradley P. Harris, Alexander Mathis
Integrative and Comparative Biology, Volume 61, Issue 3, September 2021, Pages 900–916 Preprint: arXiv:2103.00560 2021
AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild
D Joska, L Clark, N Muramatsu, R Jericevich, F Nicolls, A Mathis, MW Mathis, A Patel
2021 IEEE International Conference on Robotics and Automation (ICRA) Preprint: arXiv:2103.1328212021
Out-of-distribution generalization of internal models is correlated with reward
Mackenzie W. Mathis Khushdeep S.* Mann, Steffen* Schneider, Alberto Chiappa, Jin H. Lee, Matthias Bethge, Alexander Mathis, Mackenzie W. Mathis
SSL-RL Workshop at ICLR 2021
Tumor-specific cytolytic CD4 T cells mediate immunity against human cancer
Amélie Cachot, Mariia Bilous, Yen-Cheng Liu, Xiaokang Li, Margaux Saillard, Mara Cenerenti, Georg Alexander Rockinger, Tania Wyss, Philippe Guillaume, Julien Schmidt, Raphaël Genolet, Giuseppe Ercolano, Maria Pia Protti, Walter Reith, Kalliopi Ioannidou, Laurence de Leval, Joseph A. Trapani, George Coukos, Alexandre Harari, Daniel E. Speiser, Alexander Mathis, David Gfeller, Hatice Altug, Pedro Romero and Camilla Jandus
Science Advances 26 Feb 2021: Vol. 7, no. 9, eabe3348
Pretraining boosts out-of-domain robustness for pose estimation
Alexander Mathis, Tom Biasi*, Steffen Schneider, Mert Yüksekgönül, Byron Rogers, Matthias Bethge, Mackenzie W. Mathis
IEEE's Winter Conference on Applications of Computer Vision 2021
preprint: https://arxiv.org/abs/1909.11229 Also, see: horse-10 benchmark website! A shorter & updated version of this preprint was accepted at the Uncertainty & Robustness in Deep Learning Workshop at ICML '20!
Real-time, low-latency closed-loop feedback using markerless posture tracking
Gary A Kane, Gonçalo Lopes, Jonny L. Saunders, Alexander Mathis, Mackenzie Weygandt Mathis
eLife 2020, preprint: bioRxiv 2020.08.04.236422; code and benchmarking website
A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
Alexander Mathis, Steffen Schneider, Jessy Lauer & Mackenzie W. Mathis
Neuron, primer Volume 108, Issue 1, P44-65, October 14, 2020, also: arXiv:2009.00564
ImageNet performance correlates with pose estimation robustness and generalization on out-of-domain data
Alexander Mathis, Tom Biasi, Mert Yüksekgönül, Byron Rogers, Matthias Bethge, Mackenzie W. Mathis International Conference in Machine Learning: Workshop on Uncertainty and Robustness in Deep Learning, 2020
Deep learning tools for the measurement of animal behavior in neuroscience
Mackenzie W. Mathis & Alexander Mathis
Current Opinion in Neurobiology Volume 60, February 2020, Pages 1-11 also: https://arxiv.org/abs/1909.13868
Using DeepLabCut for 3D markerless pose estimation across species and behaviors
Tanmay Nath* Alexander Mathis* An Chi Chen Amir Patel Matthias Bethge Mackenzie W. Mathis
Nature Protocols: Volume 14, 2152-2176, June 2019
Noninvasive behavioral tracking of animals during experiments is critical to many scientific pursuits. Extracting the poses of animals without using markers is often essential to measuring behavioral effects in biomechanics, genetics, ethology, and neuroscience. However, extracting detailed poses without markers in dynamically changing backgrounds has been challenging. We recently introduced an open-source toolbox called DeepLabCut that builds on a state-of-the-art human pose-estimation algorithm to allow a user to train a deep neural network with limited training data to precisely track user-defined features that match human labeling accuracy. Here, we provide an updated toolbox, developed as a Python package, that includes new features such as graphical user interfaces (GUIs), performance improvements, and active-learning-based network refinement. We provide a step-by-step procedure for using DeepLabCut that guides the user in creating a tailored, reusable analysis pipeline with a graphical processing unit (GPU) in 1–12 h (depending on frame size). Additionally, we provide Docker environments and Jupyter Notebooks that can be run on cloud resources such as Google Colaboratory.
DeepLabCut: Markerless tracking of user-defined features with deep learning
Alexander Mathis Pranav Mamidanna Kevin N. Cury Taiga Abe Venkatesh N. Murthy Mackenzie W. Mathis* Matthias Bethge*
Nature neuroscience: Volume 21, 1281-1289, September 2018
Quantifying behavior is crucial for many applications in neuroscience. Videography provides easy methods for the observation and recording of animal behavior in diverse settings, yet extracting particular aspects of a behavior for further analysis can be highly time consuming. In motor control studies, humans or other animals are often marked with reflective markers to assist with computer-based tracking, but markers are intrusive, and the number and location of the markers must be determined a priori. Here we present an efficient method for markerless pose estimation based on transfer learning with deep neural networks that achieves excellent results with minimal training data. We demonstrate the versatility of this framework by tracking various body parts in multiple species across a broad collection of behaviors. Remarkably, even when only a small number of frames are labeled (~200), the algorithm achieves excellent tracking performance on test frames that is comparable to human accuracy. News & Views at Nature Neuroscience, NVIDIA AI developer blog, The Atlantic, and Harvard Gazette article
Neuronal Representation of Social Information in the Medial Amygdala of Awake Behaving Mice
Ying Li Alexander Mathis Benjamin Grewe Jessica A. Osterhout Biafra Ahanonu Mark J. Schnitzer Venkatesh N. Murthy Catherine Dulac
Cell: Volume 171, Issue 5, p1176–1190.e17, 16 November 2017
The amygdala (MeA) plays a critical role in processing species- and sex-specific signals that trigger social and defensive behaviors. We used a miniature microscope to image the Ca-dynamics of large neural ensembles in behaving mice and tracked the responses of MeA neurons over several months. By performing long-term imaging across different social contexts, we found that sexual experience triggers lasting and sex-specific changes in MeA activity, which, in males, involve signaling by oxytocin. News piece on HHMI website
Somatosensory Cortex Plays an Essential Role in Forelimb Motor Adaptation in Mice
Mackenzie W. Mathis Alexander Mathis Naoshige Uchida
March 2017, Neuron: Volume 93, Issue 6, p1493-1503.
We developed a force field-based motor adaptation task for mice and found that photoinhibition of forelimb S1 abolished motor adaptation while sparing basic kinematics and their ability to learn a new reward location that required similar motor outputs. News piece at MCB department website
Periodic population codes: From a single circular variable to higher dimensions, multiple nested scales, and conceptual spaces Andreas V.M. Herz Alexander Mathis Martin Stemmler
October 2017, Current Opinion in Neurobiology Volume 46, Pages 99–108
Across the nervous system, neurons often encode circular stimuli using tuning curves that are not sine or cosine functions, but that belong to the richer class of von Mises functions, which are periodic variants of Gaussians. For a population of neurons encoding a single circular variable with such canonical tuning curves, computing a simple population vector is the optimal read-out of the most likely stimulus. We argue that the advantages of population vector read-outs are so compelling that even the neural representation of the outside world's flat Euclidean geometry is curled up into a torus (a circle times a circle), creating the hexagonal activity patterns of mammalian grid cells. Here, the circular scale is not set a priori, so the nervous system can use multiple scales and gain fields to overcome the ambiguity inherent in periodic representations of linear variables. We review the experimental evidence for this framework and discuss its testable predictions and generalizations to more abstract grid-like neural representations.
Reading out olfactory receptors: feedforward circuits detect odors in mixtures without demixing
Alexander Mathis* Dan Rokni* Vikrant Kapoor Matthias Bethge Venkatesh N. Murthy
September 2016, Neuron: Volume 91, Issue 5, p1110-1123.
We showed that a linear readout can directly from receptor input match the behavioral performance of mice when detecting target odors in mixtures. This model also predicted how mice perform when trained with single odors alone. This experimental test is also presented in the paper. Summary in Harvard Gazette | News piece at MCB department website | (German) news piece at U Tübingen
Connecting multiple spatial scales to decode the population activity of grid cells
Martin Stemmler* Alexander Mathis* Andreas V.M. Herz
December 2015, Science Advances Vol. 1, no. 11, e1500816
We derived a biologically plausible readout scheme for grid cells. This readout generalizes population vectors while employing gain fields and implies many known properties of grid cells. For instance, that the spatial periods should follow a geometric series with ratio 3/2. This model also made a number of novel predictions, including that such readout neurons have vectorial tuning fields (consistent with recently found hippocampal neurons Sarel et al. Science 2017 ). EurekAlert! news piece | Recommended at Faculty of 1000 by Edvard Moser and Dave Rowland.
Probable nature of higher-dimensional symmetries underlying mammalian grid-cell activity patterns
Alexander Mathis Martin Stemmler Andreas V.M. Herz
April 2015, eLife 2015;10.7554/eLife.05979
By using the Fisher information we demonstrated for an arbitrary dimensional stimulus space that the optimal periodic code is always given by the densest lattice. As a corollarly it follows that grid cells in 2D should be hexagonally tuned (which is indeed the case) and that in 3D the firing fields should arranged in a hcp or fcc scheme. The latter case makes a prediction for grid cells in bats, which is currently being investigated. Beyond grid cells, such codes might be of more general use, as aptly discussed in a recent review by Kriegeskorte and Storrs. Ludwig-Maximilians-University press release | also in German.
Multiscale codes in the nervous system: The problem of noise correlations and the ambiguity of periodic scales
Alexander Mathis Andreas V.M. Herz Martin Stemmler
August 2013, Physical Review E 88, 022713.
Based on recordings from the Moser lab we showed that there are significant noise correlations for grid cells with similar spatial preference. Although such correlations hamper the resolution of any single module, we demonstrated that the resolution of the population remains exponential in the number of neurons.
Resolution of nested neuronal representations can be exponential in the number of neurons.
Alexander Mathis Andreas V.M. Herz Martin Stemmler
August 2012, Physical Review Letters: 109,018103.
Collective computation is typically polynomial in the number of computational elements, such as transistors or neurons, whether one considers the storage capacity of a memory device or the number of floating-point operations per second of a CPU. However, we show here that the capacity of a computational network to resolve real-valued signals of arbitrary dimensions can be exponential in N, even if the individual elements are noisy and unreliable. Nested, modular codes that achieve such high resolutions mirror the properties of grid cells in vertebrates, which underlie spatial navigation. Ludwig-Maximilians-University press release
Optimal Population Codes for Space: Grid Cells Outperform Place Cells
Alexander Mathis Andreas V.M. Herz Martin Stemmler
September 2012, Neural Computation: Vol. 24, No. 9: 2280-2317.
We study how to best construct place and grid codes, taking the probabilistic nature of neural spiking into account. Optimizing the spatial resolution predicts two grid cell properties that have been experimentally observed. First, short lattice spacings should outnumber long lattice spacings. In particular, the lattice spacings should follow a geometric series, as it was later observed Stensola et al. Nature 2012. Second, the grid code should be self-similar across different lattice spacings, so that the grid field always covers a fixed fraction of the lattice period. Recommended at Faculty of 1000 by Neil Burgesss and Caswell Barry.