Author response:
The following is the authors’ response to the original reviews.
Public Reviews:
Reviewer #1 (Public review):
Summary:
In this study, the authors elegantly combined latent variable models (i.e., HMM, GPFA and dynamical system models) with a calcium imaging observation model (i.e., latent Poisson spiking and autoregressive calcium dynamics (AR)).
Strengths:
Integrating a calcium observation model into existing latent variable models improves significantly the inference of latent neural states compared to existing approaches such as spike deconvolution or Gaussian assumptions.
The authors also provide an open-source access to their method for direct application to calcium imaging data analysis.
Weaknesses:
As acknowledged by the authors, their method is dependent on the quality of calcium trace extraction from fluorescence videos. It should be noted that this limitation applies to alternative strategies.
While the contribution of this study should prove useful for researchers using calcium imaging, the novelty is limited, as it consists of an integration of the calcium imaging model from Ganmor et al. 2016 with existing LVM frameworks.
Reviewer #2 (Public review):
Summary:
This compelling study proposes a framework to implement latent variable models using population level calcium imaging data. The study incorporates autoregressive dynamics and latent Poisson spiking to improve inference of latent states across different model classes including HMMs, Gaussian Process Factor Analysis and nonlinear dynamical systems models. This approach allows for a more seamless integration of existing methods typically used with spiking data to apply on calcium imaging data. The authors test the model on piriform cortex recordings as well as a biophysical simulator to validate their methods. This approach promises to have wide usability for neuroscientists using large population level calcium imaging.
Strengths:
The strengths of this study are the flexibility in the choice of models and relatively easy adaptation to user-specific use cases.
Weaknesses:
The weakness of the study lies in its limited validation of biological calcium imaging data. Calcium dynamics in a task-specific context in a sensory brain region might be very different from slower dynamics in a region of integration. The biophysical properties of the data would also be dependent on the SNR of the imaging platform and the generation of calcium indicator being used.
Reviewers 1 and 2 correctly point out that our method depends on the quality of the upstream calcium trace extraction. As they noted, these traces can vary based on the specific indicator used, the signal-to-noise ratio (SNR) of the recording, and the region of the brain being imaged (such as sensory vs. integrative areas). As Reviewer 1 rightly mentions, this is a universal challenge that applies to alternative strategies as well, rather than a limitation unique to our framework. To address these points, we have added a paragraph to our Discussion section.
Reviewer #3 (Public review):
Summary:
S. Keeley & collaborators propose a computational approach to infer time-varying latent variables directly from calcium traces (for instance, obtained with 2p imaging) without the need for deconvolving the traces into spike trains in a preliminary, independent step. Their approach rests on 1 of 3 families of latent models: GPFA, HMM and dynamical systems - which they augment with an observation model that maps latent variables to fluorescence traces. They validate their approach on simulated and real data, showing that the approach improves latent variable inference and model fitting, compared to more traditional approaches (although not directly compared with the 2-step one; see below). They provide a GitHub repository with code to fit their models (which I have not tested).
Strengths:
The approach is sound and well-motivated. The authors are specialists in latent variable models. The manuscript is succinct, well-written, and the figures are clear. I particularly liked the diversity of latent models considered, in particular latent models with continuous (GPFA) vs.
discrete (HMM) dynamics, which are useful for characterizing different types of neural computations. The validation on both simulated and real data is convincing.
Weaknesses:
One advantage … is that one can inspect the quality of the deconvolution step independently from the latent variable inference step. For instance, if the inferred latent variables are not interpretable, how can one determine whether this is due to a poor choice of latent model (e.g., HMM with too few states), or a poor fit of the observation model (e.g., wrong parameters for the calcium dynamics)?
We agree with the reviewer that integrating the calcium likelihood introduces additional parameters that require careful diagnostics. However, for the vast majority of imaging datasets, there is no simultaneous electrophysiology to verify the deconvolution step. If the final latent states are not interpretable, it remains impossible to determine whether the error originated in the initial spike inference from deconvolution or the subsequent model fitting.
Our framework addresses this by maintaining the raw fluorescence as the fixed observation. We suggest for those using this model to employ cross-validation using this data to select model parameters. We outline how to do this below, but because the data itself does not change with each model fit, you can compare P(data | λ) across any model configuration. In contrast, different deconvolution methods change the data itself (the spiketimes) making comparison across models impossible.
Could the authors comment on whether their approach allows for instance to compare different forms of latent models (e.g., HMM vs. GPFA) in terms of model evidence, cross-validated log-likelihood or other model comparison metrics?
We thank the reviewer for highlighting this. In short: yes. Because our framework integrates the calcium observation likelihood with various latent variable models, we can assess held-out prediction P(data | λ) irrespective of the specific latent structure.
However, because fitting the LVM requires inferring the latent state z to determine the firing rate λ, proper cross-validation across models involves holding out both neurons and timepoints. A principled approach—which our framework supports—is as follows:
(1) Train both the latent states z and the model parameters (e.g., the mapping from latent space to observations) on a training portion of the recording.
(2) On a held-out test segment, withhold a subset of "test" neurons and infer the latent states using only the "held-in" neurons.
(3) Calculate the likelihood of the observed fluorescence for the test neurons given the inferred rates.
We clarify this procedure in the revised manuscript. While a comprehensive benchmarking across all possible LVM architectures is beyond the scope of this study, we provide the statistical infrastructure for users to perform such comparisons. Furthermore, we would like to emphasize that while predictive likelihood is a rigorous metric for model selection, the primary utility of these LVMs often lies in the interpretability of the latent states themselves, which can remain biologically informative even if cross-validated performance is not the sole optimization target.
While it certainly makes sense that models accounting for the full transformation of latent => spikes => fluorescence data should outperform the two-step (1) deconvolution => (2) latent variance inference approach, the amount of improvement is not clear. A direct comparison … would be useful
We thank the reviewer for this point. Figure 4 was designed to address this comparison directly. By using a biophysical simulator, we generated a pseudo-realistic spiking network with ground-truth latent trajectories governed by a Gaussian Process. This allowed us to explicitly compare our unified approach against the traditional deconvolution-then-Poisson-GPFA pipeline. While a first-order (AR1) calcium likelihood did not show improvement over the two-step deconvolution method in recovering the ground-truth latents, the second-order (AR2) process demonstrated an improvement. Because there are no ground-truth parameters in the model, we use the reconstruction error of the latent values as our primary metric for recovery. These results suggest that when the observation model sufficiently captures the underlying calcium kinetics, the unified approach offers a more accurate estimation of the neural state.
It would be useful to discuss the possible extension of the approach to other types of data that … have different observation models.
We thank the reviewer for this helpful comment. We agree that the general framing of the likelihood has potential use in a wider range of data modalities.
Specifically, all sensors (aside from some voltage sensors) have a rise and decay time in line with our model. Thus the autoregressive (AR) nature of the calcium likelihood we utilize makes the current implementation particularly well-suited for a broad range of fluorescence-based sensors with similar temporal profiles. The specific use and extension would depend heavily on the biological target of the sensor. For example Glutamate, dopamine, and similar indicators can be thought of as having a similar underlying Poisson firing model, as the release of these products is tied to neural firing. Other sensors that might relate to other biological processes, such as hemodynamics (via imaging or ultrasound) or broader neuromodulation (Norepinephrine imaging with nLight) might be more continually varying and therefore would require changing the Poisson with an appropriate alternative, for example a Gaussian Process or similar.
Voltage imaging is the one exception that may require more complex observation models. However, the challenge in voltage imaging is not the ability to identify individual spikes, but more that the speed and scope of imaging is inherently limited by the speed of the voltage process and signal-to-noise ratios induced by the low quantum efficiency and membrane-bound nature of these indicators. If imaged well, single spikes would be clearly visible and the two-stage likelihood would not be necessary—one could simply use the spike times in a Poisson model just as with electrophysiology. We have added a paragraph in the discussion highlighting these points.
