Variational Streaming Flow:

Probabilistic Forecasting in Physical Time

Hans Hao-Hsun Hsu1,2 Minseon Gwak1,5 Soon Hoe Lim3,4,* Pan Li2 N. Benjamin Erichson1,5

1Lawrence Berkeley National Laboratory 2Georgia Institute of Technology 3KTH Royal Institute of Technology 4Nordita 5International Computer Science Institute *Work done while visiting ICSI/UC Berkeley

Abstract

Probabilistic forecasting is important for predicting complex dynamical systems because intrinsic randomness and incomplete observations can cause the same observed state to evolve into multiple plausible futures. While flow matching is a flexible approach for probabilistic forecasting, it is computationally expensive. Streaming flow (SF) reformulates this approach to model temporal evolution efficiently by learning a continuous velocity field directly in physical time. However, SF learns a deterministic velocity field. Thus, it provides only a single future trajectory for a given fixed initial state and observation history. To overcome this limitation, we introduce Variational Streaming Flow (VSF). Our approach learns a latent distribution that is conditioned on the dynamics of interest. In turn, this enables probabilistic forecasting. Importantly, we retain the computational efficiency of SF by generating in physical time. Across deterministic and stochastic dynamical systems, VSF demonstrates superior predictive accuracy and distributional fidelity. We demonstrate the advantage for both long-horizon rollouts exceeding 1,000 steps, and settings with bifurcating dynamics. Moreover, VSF can be integrated into existing Joint-Embedding Predictive Architecture (JEPA)-based world models as a plug-and-play predictor to improve temporal dynamics and goal-directed success rate in navigation, motion planning, and manipulation.

Why Variational Streaming Flow (VSF)?

01

1 NFE per forecast step

VSF advances each forecast step with one velocity-field evaluation (NFE = 1), avoiding iterative denoising and achieving up to a 7× speedup over autoregressive diffusion.

02

Multiple plausible futures

Different latent codes let VSF generate multiple plausible futures from the same observed history. Holding each code fixed throughout a rollout helps preserve coherence as the dynamics branch.

03

Accurate long-horizon rollouts

VSF mitigates error accumulation over 1,000-step chaotic-system forecasts. Its accurate imagined rollouts support JEPA world model planning, improving average success by 7.34% across robotics tasks.

Probabilistic forecasting

Stochastic Rössler forecasting

AR diffusionBaseline
Animated AR-diffusion forecasts for the partially observed stochastic Rössler system.
VSFOurs
Animated VSF forecasts for the same partially observed stochastic Rössler history.

1 · Forecasting paradigm

Forecasting paradigm with diffusion/flow models

Diffusion/flow models generate samples by transforming noise into clean samples through a series of iterative steps. For forecasting temporal sequences, this process can generate an entire trajectory at once or advance sequentially. Here we introduce four paradigms for forecasting with diffusion/flow models.

Observed history Being generated Completed forecast Waiting
A

Full-sequence diffusion

View
Full-sequence diffusion denoises every requested future state together across an auxiliary diffusion-time axis.
Forecast horizon
Fixed up front
Additional denoising
Full trajectory
B

Block diffusion

View
Block diffusion denoises five future states together, appends the block, and repeats.
Forecast horizon
Open-ended
Additional denoising
One loop per block
C

Autoregressive diffusion

View
Autoregressive diffusion fully denoises one future state before advancing to the next physical time.
Forecast horizon
Open-ended
Additional denoising
One loop per state
D

Streaming Flow

View
Streaming Flow moves directly from one physical forecast time to the next without an auxiliary denoising loop.
Forecast horizon
Open-ended
Additional denoising
None — physical time only

2 · Physical-time flow

What is Streaming Flow?

Diffusion-style forecasting uses an inner generation process to create states at physical times. Streaming Flow removes that extra clock: advancing the learned dynamics advances the forecast itself.

Noisy training states narrow around the reference trajectory as physical time advances.

Noise contracts in physical time

Construct noisy states around a clean trajectory. Their spread shrinks with physical time, so later states approach the path.

Ys = ξ(s) + σ(s)ε, σ(s) = σ0e−κs
A learned physical-time velocity field aligns with the trajectory and points nearby states back toward it.

Learn one physical-time field

Fit the velocity field to local motion targets. It follows the trajectory while pulling nearby states back toward it.

LSF = E‖vθ − gξ‖²
One deterministic Streaming Flow forecast rolls forward from the latest observed state.

Roll forward one step at a time

Start from the latest observation. At step k, the field reads normalized physical time sk and the observed and rolling history h, then advances the state directly.

x̂k+1 = x̂k + Δt vθ(x̂k, sk, h)
1 velocity-field evaluation per transition
Diffusion forecasting

Physical time auxiliary denoising time

Streaming Flow

Physical time only

Streaming Flow removes the denoising loop, but standard SF still maps the same state and history to only one future.

3 · Bifurcation example

Motivation of Variational Streaming Flow

Standard non-latent Streaming Flow assigns one direction to each state and observed history, so the same input always produces one trajectory. In a bifurcation example, it must encode the future choice in tiny physical-state differences; making those differences smaller makes the rollout more sensitive to perturbations.

Limitation of SF

A deterministic velocity field cannot express several future motions from the exact same state and history.

Bifurcation example

Standard SF separates branches through small state differences, which can amplify small rollout errors in bifurcation.

How VSF solves it

One sampled code can select a continuation while both futures begin from the same physical state.

The top row shows the desired branching behavior; the bottom row shows why standard SF achieves it only by becoming sensitive to small state errors.

Bifurcation analysis showing the ground truth, standard Streaming Flow, an ideal latent construction, and the fidelity-sensitivity tradeoff.
Bifurcation example. Better physical-state fidelity makes standard SF increasingly sensitive near the separator; a latent branch code removes this particular bottleneck.

We next explain how VSF learns latent codes from observed velocity field in the next section.

See technical detailsHide technical details Matched Standard-SF and latent equations, with Proposition A.1

Setup and Notation

Bifurcation Example

Let B ∈ {−1,+1} index the two futures. Both trajectories remain at x = 0 through the bifurcation time τ; for t > τ, B = −1 and B = +1 select the negative and positive branches:

ξB(t) = Bm(t), with m(t) = 0 before τ and positive after τ
B
future branch label
m(t)
branch displacement
τ
bifurcation time
σ₀
initial tube scale; the slider control
k
tube contraction rate, with σt = σ₀e−kt
JSFτ,t
flow-map sensitivity at the separator

For both constructions, let q(t) = ṁ(t) + km(t) and σt = σ₀e−kt. The quantity Jτ,t is the derivative of the corresponding flow map with respect to its physical state at the split.

Standard Streaming Flow

Why physical-state fidelity raises sensitivity

At the population optimum, Standard SF must infer branch identity from the physical state x. For the complete assumptions and proof, see Proposition A.1 in the paper .

01Velocity fieldPopulation-optimal
v⋆SF(x,t) = −kx + q(t)tanh(xm(t)/σt²)
02Split-time fidelity errorDistance to the data marginal
ετ := W₂(PτSF, Pτdata) = στ = σ₀e−kτ
Current slider setting σ₀ = 0.340 ετ ≈ 0.194
03Flow-map sensitivityFor fixed t > τ, at x = 0
JSFτ,t = exp[−k(t−τ) + m(t)²e2k(t−τ)/(2ετ²)]
The issue

For fixed t > τ with m(t) > 0, improving split-time fidelity forces the physical-state sensitivity to diverge:

ετ ↓ 0 implies JSFτ,t → ∞

Ideal latent construction

How conditioning removes the toy bottleneck

Let Z ∼ N(0,1), with b(Z) = +1 for Z ≥ 0 and b(Z) = −1 for Z < 0. The latent code supplies the branch while both futures start from xτ = 0.

01Velocity fieldIdeal latent-conditioned
videal(x,t,Z) = −kx + b(Z)q(t)
02Split-time fidelity errorDistance to the data marginal
W₂(Pτideal, Pτdata) = 0
03Flow-map sensitivityFor fixed latent code Z
Jidealτ,t = ∂Φidealτ,t(x,Z)/∂x = e−k(t−τ)
What changes

The construction is exact at the split and contracts physical-state perturbations because branch identity is carried by Z, not by tiny differences in x.

4 · Method

Variational Streaming Flow

VSF generates future trajectories by evolving the latest observed state through a learned velocity field in physical time. Different latent values allow the model to generate multiple plausible futures from the same observed history. During training, an encoder learns which latent values explain the observed dynamics.

The forecast follows the learned latent-conditioned dynamics:

d x̂s / ds = vθ(x̂s, s, h, z)

Training

Construct a local state and physical velocity (xs, us).

z(1:M) iid∼ qφ(· | us, xs, s, h)

Evaluate the corresponding physical-time velocities.

ûs(1:M) = vθ(xs, s, h, z(1:M))

Optimize generalized energy reconstruction together with KL regularization toward p(z).

Inference

Sample z ∼ p(z) and retain it throughout the forecast.

Advance the predicted state directly in physical time.

x̂k+1 = x̂k + Δs vθ(x̂k, sk, h, z)

Append generated states to the rolling history and refresh h at each segment boundary.

Latent velocity model

VSF uses a trajectory-level latent code z so the same context cs can represent different velocities us. It learns their conditional distribution by marginalizing over the latent code:

maximize log pθ(us | cs), where pθ(us | cs) = ∫ pθ(us | cs, z)p(z) dz

Training objective

VSF evaluates M latent-conditioned velocity samples together using generalized energy reconstruction:

Why not use MSE alone?

A variational encoder infers z from the motion observed during training. If that code does not fully separate the possible futures, MSE averages the unresolved velocities toward their conditional mean, v*(c, z) = E[U | C = c, Z = z]. The result can fall between valid motions.

LES(s) = (1/M) Σi ‖ûs(i) − us‖₂α − (1/2M²) Σi,j ‖ûs(i) − ûs(j)‖₂α, 0 < α < 2

The first term rewards accuracy against the observed velocity; the pairwise term accounts for the spread among samples. The full objective adds KL regularization to keep the training-time codes close to the prior used for forecasting:

LVSF = E[LES(s) + βKL KL(qφ(z | us, xs, s, h) ‖ p(z))]

The plots below show how the training objective scores exact, symmetric, and biased predictions.

α = 1.000
Prediction
At α = 1, the loss uses unsquared distances. Selected pair {−0.7, +0.7}.

(a) Predicted distribution versus target
  1. Prediction–truth mismatch: the actual distribution error; for 0 < α < 2, 0 means an exact match.
  2. Training reconstruction loss LES: the quantity VSF minimizes; it upper-bounds the mismatch in this example.
(b) Prediction–truth mismatch landscape
The star marks an exact match. Color shows the mismatch for each prediction; the selected pair has mismatch 0.150.
See technical guaranteeHide technical guarantee Proposition C.1 and its aggregate-alignment assumption
What does aggregate posterior matching mean?

We construct a two-branch example to make q̄φ(z | c) = p(z) concrete:

  • Inference prior: p(z) = (0.50, 0.50), so either branch is sampled with equal probability.
  • Training posteriors: the encoder assigns (0.85, 0.15) for U = −1 and (0.15, 0.85) for U = +1, ordered as (P(Z = −1), P(Z = +1)).
  • Aggregate posterior: averaging over the two equally likely targets recovers the inference prior.
q̄φ(z | c) = ½(0.85, 0.15) + ½(0.15, 0.85) = (0.50, 0.50) = p(z).

Objective type. Replacing Gaussian reconstruction with LES makes the final loss a KL-regularized distributional objective.

5 · Results

Dynamical-system forecasting

We evaluate the probabilistic forecasting performance of VSF on fully observed deterministic systems and partially observed stochastic systems.

The tables summarize average errors; the animations show how each forecasting method advances along individual rollouts.

5.1

Deterministic forecasting

All state variables are observed. We report pointwise forecast accuracy and compare complete long-horizon trajectories.

Table 1, deterministic systems. Forecasting error over a 1,000-step rollout. CRPS evaluates the forecast ensemble, while RMSE and MAE evaluate its mean. VSF achieves the lowest value for all three metrics on double pendulum, Rössler, and Lorenz, with samples concentrated near the ground-truth trajectory. Runtime (RT), in seconds, is averaged across tasks; lower is better.

Complete deterministic results

Scroll horizontally to view all metrics.

Complete 1,000-step deterministic forecasting results.
Model Double pendulum Rössler Lorenz RT
CRPS ↓RMSE ↓MAE ↓ CRPS ↓RMSE ↓MAE ↓ CRPS ↓RMSE ↓MAE ↓
Block diffusion0.46 ± 0.070.91 ± 0.140.47 ± 0.070.29 ± 0.130.54 ± 0.260.29 ± 0.130.31 ± 0.120.42 ± 0.180.33 ± 0.1230.2
AR diffusion0.43 ± 0.000.96 ± 0.010.59 ± 0.010.35 ± 0.061.00 ± 0.130.53 ± 0.080.86 ± 0.041.96 ± 0.061.50 ± 0.0678.2
Rectified flow0.53 ± 0.071.08 ± 0.110.67 ± 0.070.40 ± 0.070.85 ± 0.200.48 ± 0.081.46 ± 0.892.69 ± 1.461.94 ± 1.1441.1
DeepAR0.58 ± 0.001.07 ± 0.000.68 ± 0.000.17 ± 0.040.54 ± 0.050.21 ± 0.0510.30 ± 0.2212.42 ± 0.1310.88 ± 0.133.9
SF2.12 ± 0.879.18 ± 5.172.12 ± 0.870.16 ± 0.070.34 ± 0.180.16 ± 0.071.42 ± 0.142.75 ± 0.311.42 ± 0.146.4
SF (perturb.)1.83 ± 0.868.38 ± 5.181.96 ± 0.890.11 ± 0.050.34 ± 0.180.16 ± 0.071.39 ± 0.142.75 ± 0.311.42 ± 0.146.4
Chronos-20.78 ± 0.001.50 ± 0.020.96 ± 0.011.98 ± 0.004.11 ± 0.022.62 ± 0.0113.50 ± 0.0628.19 ± 1.1425.22 ± 0.850.8
Chronos-2 (calib.)0.62 ± 0.001.16 ± 0.000.74 ± 0.001.91 ± 0.003.91 ± 0.022.52 ± 0.012.39 ± 0.026.01 ± 0.274.54 ± 0.270.4
VSF Ours0.25 ± 0.070.71 ± 0.110.33 ± 0.080.03 ± 0.000.13 ± 0.010.05 ± 0.000.16 ± 0.020.25 ± 0.020.19 ± 0.017.2

Values are mean ± population standard deviation across three seeds.

5.2

Stochastic forecasting

Rössler and Lorenz are partially observed parameter-mixture systems. We evaluate the conditional distribution of complete future trajectories.

Table 2, stochastic systems. Distributional forecasting error over a 1,000-step rollout. CME and CSE measure errors in the conditional mean and spread, while Wasserstein and energy distances compare complete trajectory distributions. VSF achieves the lowest value for every metric on both Rössler and Lorenz, showing the best recovery of the distribution of plausible futures. Runtime (RT), in seconds, is averaged across tasks; lower is better.

Complete stochastic results

Scroll horizontally to view all metrics.

Complete 1,000-step partially observed stochastic forecasting results.
Model Rössler Lorenz RT
CME ↓CSE ↓W1 ↓ED ↓ CME ↓CSE ↓W1 ↓ED ↓
Block diffusion0.16 ± 0.020.72 ± 0.088.88 ± 1.0512.67 ± 1.710.20 ± 0.010.93 ± 0.0119.88 ± 0.2813.73 ± 0.4521.8
AR diffusion0.20 ± 0.034.63 ± 0.4016.21 ± 1.1211.46 ± 2.060.22 ± 0.050.95 ± 0.1425.64 ± 3.105.60 ± 1.5442.4
Rectified flow0.22 ± 0.090.89 ± 0.3012.60 ± 4.4416.72 ± 8.720.26 ± 0.030.60 ± 0.1219.74 ± 1.6313.24 ± 2.6919.4
DeepAR0.20 ± 0.020.61 ± 0.0211.81 ± 0.8015.62 ± 1.572.14 ± 0.492.61 ± 0.27131.84 ± 15.99138.30 ± 45.711.2
SF0.06 ± 0.021.00 ± 0.004.53 ± 0.675.00 ± 1.341.04 ± 0.561.00 ± 0.0058.12 ± 24.2389.84 ± 48.457.5
SF (perturb.)0.06 ± 0.021.12 ± 0.234.92 ± 0.842.28 ± 0.820.68 ± 0.331.33 ± 0.7553.62 ± 25.6935.78 ± 19.556.6
Chronos-21.28 ± 0.0647.87 ± 1.6390.54 ± 3.8694.84 ± 0.796.84 ± 0.1737.49 ± 1.66411.32 ± 9.43342.98 ± 1.870.3
Chronos-2 (calib.)1.20 ± 0.0661.83 ± 2.09101.69 ± 4.6193.23 ± 0.201.44 ± 0.0610.53 ± 0.49111.81 ± 3.1164.97 ± 0.790.1
VSF Ours0.04 ± 0.010.36 ± 0.142.62 ± 0.381.28 ± 0.420.19 ± 0.030.50 ± 0.2115.95 ± 5.645.19 ± 0.936.7

Values are mean ± population standard deviation across three seeds.

6 · World-model planning

World-model prediction

Forecast quality matters for control when a planner uses imagined trajectories to compare actions. We next test VSF inside a JEPA-based world model. We replace its deterministic autoregressive predictor with action-conditioned VSF while keeping the pretrained JEPA representation fixed. For each candidate action sequence, the Cross-Entropy Method (CEM) evaluates the predicted physical-time trajectory and selects the plan whose terminal state best matches the encoded goal.

Action-conditioned VSF world-model pipeline, from CEM candidate actions and encoded observations through the VSF predictor to a latent goal cost.
Action-conditioned VSF plugs into the existing JEPA encoder and CEM planner while retaining one velocity-field evaluation per imagined transition.
Table 3 · World-model predictor. As a world-model predictor, VSF achieves the highest planning success on all four tasks and the lowest predictor RMSE on three of four tasks. RMSE is measured between predicted and ground-truth latent trajectories along the CEM-selected action sequence.

Scroll horizontally to see every task and metric.

Planning success and latent-trajectory RMSE for AR, SF, and VSF predictors on PushT, Cube, TwoRoom, and Reacher.
Method PushT Cube TwoRoom Reacher
Success ↑ RMSE ↓ Success ↑ RMSE ↓ Success ↑ RMSE ↓ Success ↑ RMSE ↓
AR predictor 84.7 ± 4.10.28 ± 0.01 69.3 ± 2.50.80 ± 0.05 78.7 ± 9.00.67 ± 0.03 61.3 ± 8.21.06 ± 0.01
SF predictor 88.0 ± 2.80.30 ± 0.04 76.7 ± 0.90.64 ± 0.02 76.7 ± 8.20.59 ± 0.05 49.3 ± 5.01.13 ± 0.03
VSF predictor Ours 92.7 ± 2.50.27 ± 0.03 80.0 ± 1.60.65 ± 0.08 82.7 ± 5.00.58 ± 0.01 68.0 ± 0.01.03 ± 0.02

World model visualizations

Plan 1 and Plan 2 are two successive planning rounds toward the same goal.

  1. Plan 1 (blue): initial planning. CEM uses the world-model predictor to choose an action sequence from the initial observation, then the agent executes it for 25 time steps.
  2. Plan 2 (red): replanning. If the goal remains unreached, CEM uses the same predictor to choose a new action sequence from the latest observation.

Predictions that stay close to the dashed ground-truth latent trajectory help CEM rank candidate action sequences more accurately. In the examples below, the AR prediction diverges and the rollout fails, whereas VSF tracks the ground truth more closely and succeeds.

BibTeX

@article{hsu2026variational,
  title={Variational Streaming Flow: Probabilistic Forecasting in Physical Time},
  author={Hsu, Hans Hao-Hsun and Gwak, Minseon and Lim, Soon Hoe and Li, Pan and Erichson, N Benjamin},
  journal={arXiv preprint arXiv:2610.00976},
  year={2026}
}