Image and video generation is a task, not a single algorithm. Autoregressive models, masked-token models, GANs, diffusion models, and flow-based models solve that task in different ways. Several of them use random noise somewhere; that does not mean they all learn to remove Gaussian noise, or integrate a time-dependent vector field.

This article focuses on continuous, Gaussian-noise diffusion and continuous flow matching. They have useful mathematical connections, but understanding those connections requires keeping four things separate: the distribution at each time, the trajectories that connect those distributions, the quantity a network predicts, and the numerical procedure used to generate a sample.

TL;DR — five distinctions

1. Noise-to-data transport describes a family of generators. It is not a definition of image or video generation, and even diffusion-based editing can start from a partially corrupted input.
2. Matching distributions does not identify a unique ODE. A diffusion probability-flow ODE and a flow-matching ODE can coincide under a compatible construction; shared endpoints, or even shared time marginals, are insufficient on their own.
3. Straight training interpolants do not guarantee straight, constant-speed sampling trajectories. Rectified flow learns a conditional average of endpoint displacements. Reflow changes the coupling and can make coarse integration more accurate.
4. Diffusion v-prediction and flow-matching velocity are distinct targets. They coincide up to a time factor for a particular variance-preserving path, not for arbitrary schedules.
5. Few-step results need their experimental context. Distillation, training from scratch, solver steps, network evaluations, and wall-clock latency measure different things.

Scope: several ways to generate an image or video

These categories can overlap. Autoregression is a factorization of a distribution; it does not require discrete tokens. MAR, for example, combines autoregressive modeling with diffusion losses for continuous-valued tokens. A Transformer backbone also does not tell us whether a system is autoregressive, diffusion-based, or both.

Even within diffusion, initialization depends on the task. SDEdit adds noise to an existing image and starts the reverse process at an intermediate noise level. Conditional generation can also preserve observed regions or condition on reference frames. A single noise vector becoming a clean image is therefore an incomplete description of the larger field.

Notation and conventions

The mathematical sections use data at \(t=0\) and a source distribution at \(t=1\). Generation runs toward decreasing \(t\). Some cited papers use the opposite convention.

For ideal noise-to-data interpolation, \(\alpha_0=1,\sigma_0=0\) and \(\alpha_1=0,\sigma_1=1\). Practical diffusion schedules may only approximate the noise endpoint. Identities involving densities and derivatives below apply where those quantities exist; divisions by \(\alpha_t\) or \(\sigma_t\) require nonzero denominators. Endpoint limits need separate care.

Part I — The mathematical connections

1.1 A probability path is not a sample trajectory

For independent data and Gaussian noise,

This specifies a distribution at every time. It does not, by itself, specify how a sample at one time relates to a sample at another.

For example, a DDPM forward chain uses

Its marginal conditional on the clean sample is

We can sample this marginal in one operation as \(x_k=\sqrt{\bar a_k}x_0+\sqrt{1-\bar a_k}\varepsilon\). But reusing one fixed \(\varepsilon\) across all times produces a different joint process from the Markov chain, which injects fresh noise at each step. Their one-time conditional distributions can agree while their trajectories differ. The DDPM training objective uses this efficient marginal sampling identity. DDPM.

This distinction will recur: a path of distributions, a stochastic process, and a deterministic flow are different objects.

1.2 Score-based diffusion: two associated sampling dynamics

Consider the forward Itô SDE

with scalar, state-independent diffusion coefficient \(g(t)\). Under the regularity conditions for time reversal, its reverse-time SDE is

where time runs backward and the reverse Brownian increment has variance proportional to \(\lvert dt\rvert\). The associated probability-flow ODE is

With the exact score, matching initialization, and exact evolution, these dynamics share the forward process’s one-time marginals. They do not share sample paths. Estimated scores, a mismatched terminal prior, and discretization can break that distributional agreement in actual samplers. State-dependent diffusion requires additional terms. Score-Based Generative Modeling through SDEs.

For the independent Gaussian corruption of §1.1, the optimal squared-error noise predictor is \(\mathbb E[\varepsilon\mid x_t=x]\), and

A sampled training noise is therefore a regression target, not the marginal score itself. Appendix A.2 derives the conditional expectation explicitly.

An ODE sampler is deterministic given its initial state and conditioning. It still generates a distribution because its initial state is random. Determinism alone also does not guarantee accurate few-step sampling.

1.3 Flow matching: learning a velocity without simulating the ODE

For a differentiable random interpolant \(X_t\), let \(U_t=dX_t/dt\). Squared-error regression learns

with positive time weighting \(\lambda(t)\). Under suitable integrability and smoothness, this marginal velocity satisfies

The conditional regression loss and regression against \(b_t\) differ by a parameter-independent conditional-variance term. This is the simulation-free training principle: sample an interpolant state and its known target velocity instead of integrating the learned ODE during every training example. Sampling ordinarily still requires integration. Flow Matching, Flow Matching Guide and Code.

The probability-flow drift in §1.2 satisfies this same continuity equation. That provides a connection, not a uniqueness theorem. If a field \(a_t\) satisfies \(\nabla\!\cdot(p_ta_t)=0\), then \(b_t+a_t\) generates the same density evolution. For a stationary two-dimensional standard Gaussian, both zero velocity and rotational velocity \(a(x_1,x_2)=(-x_2,x_1)\) preserve the density, despite moving individual points differently.

A stronger identification is possible for the independent Gaussian interpolant and a compatible linear SDE. Write \(f(x,t)=f_t x\) and assume

Then the conditional-mean velocity is exactly

Appendix A.3 shows the algebra. This equality requires the chosen construction; an arbitrary path need not admit such an SDE with nonnegative \(g_t^2\). Gaussian conditional paths are an important overlap between diffusion and flow matching, rather than the full scope of either framework. Stochastic Interpolants gives a broader treatment.

Figure 1. The continuity equation connects the two constructions without making their velocity fields unique. Plotting code.

1.4 Rectified flow: straight targets, potentially curved learned motion

Take the linear interpolant

Each training pair defines a constant-velocity line. The learned population field, however, is

Many endpoint pairs can contribute at the same state and time. The model averages their velocities; a generated sample follows this averaged field, not its own preassigned training pair. The resulting ODE trajectories need not be straight or constant-speed. Reflow constructs new endpoint pairs from the learned flow and trains again to improve straightness. A linear interpolation between independently paired endpoints also does not establish globally optimal transport between the two distributions. Rectified Flow.

Here is a concrete counterexample to the claim that linear interpolation makes one Euler step accurate. Let both \(x_0\) and \(\varepsilon\) be independent scalar standard Gaussians. Define

Gaussian conditioning gives

The exact ODE returns \(Z_0=Z_1\), preserving a standard Gaussian at the endpoint. One backward Euler step instead gives

Every sample collapses to zero, even with an exact velocity predictor and perfectly straight training interpolants. In this one-dimensional example, the failure comes from the time-varying speed; in higher dimensions the direction can change too.

Figure 2. An analytically solvable Gaussian example. Both panels have the same one-time distributions in the population, but different endpoint couplings. The plotted curves in the lower panel show state versus time. The shaded band in each panel spans one population standard deviation around zero. The dashed Euler update fails to follow that evolution. Plotting code: plot_rectified_gaussian.py.

1.5 Prediction targets: noise, data, diffusion v, and transport velocity

For \(x_t=\alpha_t x_0+\sigma_t\varepsilon\), predictions of noise and data can be converted where the relevant coefficient is nonzero:

In the variance-preserving case, \(\alpha_t^2+\sigma_t^2=1\), diffusion v-prediction uses

Consequently,

Writing \(\alpha_t=\cos\phi(t)\) and \(\sigma_t=\sin\phi(t)\) makes the distinction precise:

Thus v-prediction is a path derivative with respect to the angular parameter \(\phi\). The actual time velocity includes \(\dot\phi(t)\), and the marginal ODE field includes a conditional expectation. Progressive Distillation, Appendix D.

For rectified flow, the target is instead \(\varepsilon-x_0\). The linear schedule \(\alpha_t=1-t,\sigma_t=t\) is not variance-preserving. Its derivative cannot be identified with \(\alpha_t\varepsilon-\sigma_t x_0\) by simply calling both quantities “velocity.”

Algebraically convertible outputs also do not imply identical optimization. For example,

An unweighted data-prediction loss therefore weights noise levels differently from an unweighted noise-prediction loss. Time sampling, loss weighting, and preconditioning are substantive choices.

1.6 Conditioning and classifier-free guidance

Conditioning supplies information to the model; guidance changes how its predictions are used. A conditional generator works at guidance scale one. Cross-attention, joint attention, adaptive normalization, and reference-image inputs are mechanisms for conditioning, not forms of CFG by themselves.

Classifier-free guidance combines conditional and unconditional predictions. With condition dropout during training, a common convention is

Here \(\gamma=0\) is unconditional, \(\gamma=1\) is conditional, and \(\gamma>1\) extrapolates. The same affine combination can be used with score or noise predictions when their parameterizations are compatible. Larger scales can improve some alignment or fidelity measures while reducing diversity or introducing artifacts; improvement is not guaranteed for every prompt or metric. Conventional CFG needs two predictions per step, sometimes batched together. Guidance-distilled models can avoid that runtime cost. Classifier-Free Diffusion Guidance, FLUX.1-dev model card.

With exact conditional and unconditional scores, a fixed-time identity is

This identifies a score of a tilted density at that time, when normalizable. It does not prove that the full guided sampler produces \(p_0(x\mid c)^\gamma p_0(x)^{1-\gamma}\) at its endpoint: the tilted densities need not satisfy the original forward diffusion equation. Appendix A.6 gives a simple illustration of the distinction.

1.7 Fast generation: better integration or a different learned object

Numerical solvers reduce integration error per unit of computation for an existing field. DPM-Solver exploits the structure of diffusion ODEs and demonstrates useful sampling with small evaluation budgets. Its results do not establish a universal floor of 10–20 steps. Accuracy depends on the field, schedule, guidance, solver, and evaluation criterion. DPM-Solver.

Distillation transfers behavior from a pretrained generator to a cheaper student. Progressive distillation, for example, repeatedly trains a student to replace two teacher updates with one. This changes the learned sampler, rather than merely swapping its numerical solver. Progressive Distillation.

Consistency models learn to map points on a probability-flow trajectory to a common low-noise endpoint. The original work includes both consistency distillation and consistency training without a pretrained teacher. A consistency function need not take two independently variable times: mapping to a fixed endpoint is already useful. Consistency Models.

Finite-interval models can learn transitions between two times. For an ODE trajectory \(dZ_\tau/d\tau=b_\tau(Z_\tau)\) and \(r<t\), define its average velocity

The exact flow map is then

The average velocity \(u\) is a parameterization of this map, not the map itself. Holding \(r\) fixed and differentiating along a trajectory gives

This motivates MeanFlow’s training objective. The identity alone is not a complete algorithm: regression targets, stop-gradient operations, and Jacobian-vector products matter. A learned \(u_\theta\) gives an approximate finite jump, not an exact solution by definition. Mean Flows for One-step Generative Modeling.

Figure 3. An exact rotational ODE separates three objects: the trajectory, a local Euler update, and the endpoint displacement represented by average velocity. Learning the displacement is a different approximation problem from estimating a local velocity. Plotting code.

Few-step methods share an efficiency goal, but not necessarily a teacher, objective, or exact underlying trajectory. One step also need not mean one network function evaluation (NFE). A solver can evaluate a field multiple times per step, and ordinary CFG adds conditional and unconditional evaluations. Decoding, text encoding, resolution, and hardware further separate NFE from latency.

Part II — Milestones and what their results establish

2.1 From diffusion processes to practical samplers

The history predates DDPM. Sohl-Dickstein et al. (2015) developed generative modeling by learning to reverse a gradual corruption process. Song and Ermon (2019) trained scores across noise levels and sampled with annealed Langevin dynamics.

DDPM (2020) established a particularly effective denoising formulation. DDIM (2020/2021) constructed alternative, including deterministic, sampling processes compatible with the same training objective. The score-SDE framework (2020/2021) connected several constructions and derived probability-flow ODEs. These developments broadened the choices of dynamics rather than proving that all generators use one sampler.

2.2 Representation and architecture are independent choices

Latent Diffusion (2021/2022) performs diffusion in a compressed autoencoder space, reducing the cost of spatial modeling. The continuous latent is not necessarily a discrete token vocabulary, and generating a latent still requires a decoder to produce pixels. Latent diffusion is not a requirement: Imagen Video (2022) uses a cascade of pixel-space video diffusion models.

DiT (2022/2023) demonstrated Transformer backbones for latent diffusion and measured improved FID with increasing compute in its experimental setting. This is evidence about that architecture and training setup, not a universal scaling law covering every image or video generator.

Stable Diffusion 3 (2024) combined rectified-flow training with MMDiT, whose modality-specific streams interact through attention. MMDiT is a particular design, not a synonym for every diffusion Transformer. For example, Wan’s implementation includes distinct visual self-attention and text cross-attention.

2.3 Flow matching and rectification

Flow Matching and Rectified Flow appeared as preprints in 2022 and at ICLR 2023. They made simulation-free velocity regression and the choice of transport paths especially useful in generative modeling.

The distinction between a path and its coupling is operational. Two training schemes can use the same source and target distributions while pairing individual endpoints differently. That changes the conditional velocities, their regression variance, and the learned transport. Even when two parameterizations describe the same ideal velocity, finite-capacity models trained with different loss weights need not learn equally accurate approximations. The Flow Matching Guide develops these choices beyond Gaussian image generation.

2.4 Few-step methods: read the training column first

The following are selected results from the linked paper versions, not a ranking across compatible experiments. FID measures the squared 2-Wasserstein distance between Gaussian fits to real and generated image features; lower values do not establish better prompt following, editing fidelity, or video coherence. Dataset, resolution, guidance, and sample protocol matter.

“From scratch” in this comparison concerns the generative network and its training procedure; it does not imply that all representation components, such as a latent autoencoder, were learned from scratch in the same experiment. Distributional convergence results likewise have mathematical assumptions; they are not unconditional guarantees for finite neural-network training.

Later work explores several directions. AlphaFlow, Terminal Velocity Matching, and Improved Mean Flows examine finite-time generation and its training objectives. Their connections do not make every few-step method the same objective.

Drifting Models illustrates an additional distinction: its generated distribution evolves through parameter updates during training, while inference applies a single-pass generator. It should not be described as just another numerical traversal of a fixed diffusion probability-flow ODE, or automatically classified as a two-time-conditioned flow-map model.

Part III — Reading actual image and video systems

A system combines a representation, backbone, prediction objective, conditioning mechanism, and sampler. Its product name does not identify all five. The examples below retain specific, publicly documented releases to illustrate these combinations. They are not an exhaustive list of the latest models or a live leaderboard. Sources were checked for this revision on October 2, 2026.

3.1 Image models: representative combinations

The Hunyuan example is particularly instructive: the report explicitly combines next-token language modeling and diffusion-based image modeling. “LLM-based” and “diffusion-based” can both describe aspects of the same system. Official repository.

Open weights mean that parameters are available under stated terms. They do not imply that training data, the full training pipeline, or unrestricted usage rights are available. A repository’s code license is also not automatically its checkpoint license; the linked release pages identify the relevant artifacts.

3.2 Video models: time adds modeling and systems choices

A causal video autoencoder constrains how its encoder or decoder uses temporal context. It does not, by itself, imply that the generative Transformer samples frames autoregressively. Likewise, “14B active” describes computation through the active expert; storing weights, activations, attention state, and other components determines memory use.

To compare video systems, the useful unit is a complete generation setting: output length, resolution, frame rate, conditioning inputs, model variant, precision, offloading, and hardware. Model size or denoising steps alone cannot establish speed or memory requirements.

3.3 What public results can and cannot tell us

For a closed service, separate the published evaluation and supported features from the implementation details that its developer actually discloses. A convincing image, a reasoning interface, or an API model name cannot establish whether the visual generator is autoregressive, diffusion-based, or hybrid. Architectural claims require model-specific evidence.

Open-versus-closed quality claims need comparable evaluation protocols too. A class-conditional ImageNet FID, a text-rendering score, a video benchmark aggregate, and a human-preference ranking answer different questions. None yields a universal ordering of visual generators. Even a vendor’s evaluation can concern a different checkpoint from the released one: the original HunyuanVideo repository explicitly distinguishes its evaluated high-quality version from its released fast version. Evaluation details.

Outlook — questions worth keeping open

How much training should replace inference? Better solvers, distillation, consistency training, average-velocity learning, and direct generators spend compute differently. A convincing comparison should include training cost, sampling quality, controllability, and end-to-end latency. Small-benchmark success is evidence to investigate scaling, rather than proof that one objective will replace all others.

Which path and coupling make learning easiest? Short conditional paths alone do not settle regression difficulty or numerical error. The Gaussian example in §1.4 shows why the actual learned field must be examined.

How should conditioning strength be controlled? Guidance changes generated distributions and costs computation unless incorporated into training or another mechanism. The challenge is to characterize that trade-off for the particular sampler, rather than assume a fixed-time score identity solves it.

What would demonstrate persistent video understanding? Attractive short clips do not establish stable object identity, coherent scene state, or physical reasoning over long horizons. Those capabilities need their own evaluation, beyond per-frame appearance and aggregate video scores.

Cheat sheet

Appendix A — Worked derivations

A.1 The DDPM ELBO and the simplified loss

With the forward chain as a variational distribution, the negative log-likelihood has the bound

Factoring the chain backward gives

For fixed reverse variance \(\rho_k^2 I\) and the noise parameterization

the parameter-dependent part of each intermediate KL is

Variance-dependent terms are constant only when the variances are fixed. The endpoint reconstruction and prior terms remain in the bound. Dropping the time-dependent weights produces the simplified denoising objective; it is not exactly the original ELBO. DDPM, Sections 3.2–3.4.

A.2 Why a noise predictor estimates the marginal score

Differentiate the Gaussian mixture from §1.1:

The second equality uses Bayes’ rule and differentiation under the integral. Squared-error regression onto sampled noise has precisely this conditional mean as its optimum. Rearranging \(x=\alpha_t\mathbb E[x_0\mid x]+\sigma_t\mathbb E[\varepsilon\mid x]\) also yields

Both identities concern optimal conditional means, not recovery of the particular training sample from every corrupted observation.

A.3 When the Gaussian interpolant velocity equals the PF-ODE drift

From the independent interpolant,

Substitute A.2 to obtain

If \(f_t=\dot\alpha_t/\alpha_t\) and \(g_t^2=2\sigma_t\dot\sigma_t-2f_t\sigma_t^2\geq0\) define a compatible linear SDE on the interval of interest, this becomes \(b_t=f_tx-\tfrac12g_t^2s_t\).

Independently, the Fokker–Planck equation for that SDE is

This verifies that the identified velocity generates the required marginals, assuming suitable boundary behavior and well-posed dynamics. It does not establish uniqueness among all velocities with those marginals. A finite-time pure-noise endpoint may also make the linear SDE coefficients singular even when the interpolant itself is well-defined.

A.4 Conditional flow matching as conditional-mean regression

Let \(b_t(X_t)=\mathbb E[U_t\mid X_t]\). For each time, expanding a square and conditioning gives

The cross term vanishes because \(\mathbb E[U_t-b_t(X_t)\mid X_t]=0\). The last term has no dependence on \(\theta\) when the interpolant and coupling are fixed. Integrating with the same time weighting proves equality of the conditional and marginal loss gradients.

For a smooth, compactly supported test function \(\psi\),

Integration by parts gives the continuity equation in weak form. This explains both why conditional velocity targets suffice and why their conditional average, rather than each individual target, governs the marginal flow.

A.5 Average velocity and a finite-time transition

For a well-defined ODE flow, \(Z_t-Z_r=(t-r)u(Z_t,r,t)\). Holding \(r\) fixed, differentiate along the same trajectory:

Rearrangement gives the identity in §1.7. The directional derivative is a Jacobian-vector product, and the diagonal limit is \(u(x,t,t)=b_t(x)\) when continuous. A trained network approximates this object; satisfying a noisy empirical objective does not make every finite jump exact.

A.6 Why a fixed-time guidance identity is not a sampling theorem

Consider a one-dimensional additive-noise process with noise variance \(\nu\). Let the unconditional and conditional data distributions be Gaussian with zero mean and variances two and one. Their noised variances are \(2+\nu\) and \(1+\nu\).

At CFG scale \(\gamma=2\), the tilted density from §1.6 is Gaussian with variance

At zero noise it has variance \(2/3\). Forward noising that tilted endpoint would instead produce variance \(\nu+2/3\). The two paths disagree for \(\nu>0\). Thus a score that equals the tilted score at every time need not be the score path obtained by noising the tilted endpoint under the original process. One must analyze the guided dynamics to identify their output distribution.

References and further reading

Sources are linked beside the claims they support. For a longer derivation of the core frameworks:

• Song et al., Score-Based Generative Modeling through Stochastic Differential Equations.
• Lipman et al., Flow Matching for Generative Modeling and Flow Matching Guide and Code.
• Liu, Gong, and Liu, Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow.
• Albergo, Boffi, and Vanden-Eijnden, Stochastic Interpolants: A Unifying Framework for Flows and Diffusions.

Cite this post

@article{zheng2026noise,
  title   = "Diffusion and Flow Matching: A Field Guide",
  author  = "Zheng, Amber Yijia",
  journal = "amberyzheng.com",
  year    = "2026",
  month   = "Jun",
  url     = "https://www.amberyzheng.com/blog/from-noise-to-data/"
}