Multiple Output Variables

TemporalMixtureModels.jl supports fitting mixture models to multivariate time series data with multiple output variables. Each output variable is modeled independently, within the same mixture component. This means that for each component in the mixture model, there is a separate set of parameters for each output variable, allowing the model to capture different temporal patterns for each variable, but also to capture different combinations of patterns across variables in separate mixture components.

using TemporalMixtureModels, Random
Random.seed!(27052023)  # For reproducibility

Random.TaskLocalRNG()

Data Preparation

When preparing your data for multivariate time series modeling, the format is similar to that of univariate time series, with the key difference being that the observed values Y should be a matrix where each column corresponds to a different output variable. Each row still corresponds to a time point in t, and the ids vector indicates which time series each observation belongs to.

The example dataset provided by the example_bp_data() function includes two output variables: systolic and diastolic blood pressure measurements. You can use this function to generate a multivariate dataset as follows:

t, y, ids, class_labels = example_bp_data(;n_subjects_drug=21, n_subjects_placebo=22, n_timepoints=5)

([0.0, 1.25, 2.5, 3.75, 5.0, 0.0, 1.25, 2.5, 3.75, 5.0  …  0.0, 1.25, 2.5, 3.75, 5.0, 0.0, 1.25, 2.5, 3.75, 5.0], [125.48572170434635 82.70734137033764; 103.06459556000243 71.3956312347923; … ; 109.19769689115391 71.89320129586659; 108.0637238450581 77.0854240014355], [1, 1, 1, 1, 1, 2, 2, 2, 2, 2  …  42, 42, 42, 42, 42, 43, 43, 43, 43, 43], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1  …  0, 0, 0, 0, 0, 0, 0, 0, 0, 0])

Defining a Composite Component Model

As the PolynomialRegression component only supports a single output variable, we need to define a composite component model that combines multiple PolynomialRegression models, one for each output variable. Creation of a composite component model is very straightforward using the @component macro.

component_model = @component begin
    y[1] ~ PolynomialRegression(2)
    y[2] ~ PolynomialRegression(2)
end

CompositeComponent(Component[PolynomialRegression(2), PolynomialRegression(2)], UnitRange{Int64}[1:3, 4:6], UnitRange{Int64}[1:1, 2:2])

This composite model specifies that the first output variable (y[1], systolic pressure) is modeled using a polynomial regression of degree 2, and the second output variable (y[2], diastolic pressure) is also modeled using a polynomial regression of degree 2. The @component macro is sufficiently flexible to allow for different types of component models for each output variable if desired.

TemporalMixtureModels.@component — Macro

@component begin ... end

The @component macro allows defining a CompositeComponent in a concise way. We use composite components to connect component models to specific measurements in multivariate data. Each line inside the begin ... end block should specify a mapping from a measurement (or range of measurements) to a component model using the syntax

y[index] ~ ComponentModel(args...)

All indices refer to the columns of the observation matrix y. The macro collects all specified components and their corresponding measurement indices, and constructs a CompositeComponent instance. The resulting CompositeComponent can then be used in mixture model fitting, in the same way as regular components.

Example

Simple Composite Component

using TemporalMixtureModels

component = @component begin
    y[1] ~ PolynomialRegression(2)  # Quadratic for measurement 1
    y[2] ~ PolynomialRegression(3)  # Cubic for measurement 2
end

Composite Component with Ranges

While the PolynomialRegression component only models a single measurement, you can also use components that model multiple measurements at once. For example, if you have a bivariate measurement that you want to model with a single component (here abstracted as BivariateComponent), you can specify a range of indices:

using TemporalMixtureModels

component = @component begin
    y[1] ~ PolynomialRegression(2)  # Quadratic for measurement 1
    y[2:3] ~ BivariateComponent()  # A bivariate component for measurements 2 and 3
end

source

Fitting the Multivariate Temporal Mixture Model

Once you have defined your composite component model, you can fit the multivariate temporal mixture model using the fit_mixture function, just like in the univariate case. You need to specify the number of components (clusters) you want to fit, as well as the composite component model to use.

num_components = 2  # For example, we want to fit 2 clusters
model = fit_mixture(component_model, num_components, t, y, ids)

MixtureResult(CompositeComponent(Component[PolynomialRegression(2), PolynomialRegression(2)], UnitRange{Int64}[1:3, 4:6], UnitRange{Int64}[1:1, 2:2]), 2, [[121.94443228135796, 0.3448095076792161, -0.0904306737635472, 78.76238884028707, -0.473053295564207, 0.09185031783869191], [119.97366434856283, -7.911874318141266, 0.6336783991887212, 80.62956871083868, -7.602305525382226, 0.8123489480159889]], [[62.46925627070032, 18.311570008681848], [95.33020222464681, 25.07712577768433]], [0.5109507077757636, 0.4890492922242365], [1.532459139684492e-8 0.9999999846754086; 1.358618544187119e-13 0.9999999999998641; … ; 0.9999999995003488 4.99651185218339e-10; 0.9708879657344872 0.02911203426551288], -1435.5953080617164, true, 7, NormalError())

This will fit a temporal mixture model with 2 components, each consisting of two polynomial regression models (one for each output variable) of degree 2 to the multivariate blood pressure data.

We can now use the fitted model to make predictions for new time points, just like in the univariate case.

An example of the resulting mixture model is shown below:

Multivariate Mixture Model