Publications

Unlike UNet denoisers, DiTs don't develop geometry-adaptive harmonic bases — attention locality in early layers drives generalization instead. Detailed analysis on the important question of generalization behavior of DiTs is available at dit-generalization.github.io.

Technical report on the multilingual, multimodal foundation models behind Apple Intelligence: a ~3B on-device model optimized for Apple silicon, and a Parallel-Track MoE server model for Private Cloud Compute — covering architecture, training, inference optimizations, and the Foundation Models framework for developers.

A small fraction of LLM parameters — "super weights" — are disproportionately critical to model quality; pruning even one can collapse generation. We introduce a single-forward-pass method to find them and use them to improve low-bit quantization.

Introduced the Argoverse datasets for 3D object tracking and motion forecasting, pairing LiDAR and camera sequences with rich HD maps — now a standard benchmark in autonomous driving research. Learn more on argoverse.org.