Accelerating diffusion models with sparse consistency models

Diffusion models have achieved state-of-the-art performance in image generation, but their practical deployment remains limited by two key computational bottlenecks: iterative denoising and the quadratic complexity of Transformer-based attention. This project proposes and evaluates a unified inference pipeline that integrates temporal acceleration (via a distilled flow-matching variant, FLUX.1-Schnell) and spatial acceleration (via a hardware-aware sparse attention mechanism, SpargeAttn). While both techniques have been studied independently, their combined effects in practical diffusion inference remain underexplored. Experiments are conducted on a controlled 512-sample subset of the MS-COCO 2017 validation set at 1024×1024 resolution. We compare a high-quality baseline (FLUX.1-Dev, 50 steps, guidance 3.5) against progressively optimized configurations. Results show that the majority of end-to-end latency reduction is achieved through the distilled model, reducing generation time from 10.81s to 0.94s (11.53×). Incorporating sparse attention provides a further modest improvement to 0.91s (11.88×), with more pronounced benefits observed at higher resolutions. Quality evaluation using FID, rFID, and CLIP Score indicates that the accelerated configurations maintain comparable semantic alignment, though metric differences should be interpreted cautiously due to the limited sample size. Additional scaling experiments reveal that attention optimization becomes increasingly beneficial with growing sequence length, while VAE decoding emerges as a dominant bottleneck at extreme resolutions. Overall, this work provides an empirical study of practical acceleration trade-offs in diffusion inference pipelines, highlighting that temporal acceleration dominates latency reduction at standard resolutions, while spatial optimizations contribute primarily to scalability