DeltaFlow: An Efficient Multi-frame Scene Flow Estimation Method

TL;DR: We tackle a key challenge in 3D scene flow: leveraging more temporal data traditionally leads to an exploding computational cost. Our ΔFlow efficiently captures temporal motion cues, keeping the computational cost minimal: regardless of the number of frames. 🚀

To easy understand the core method, I copy the core file: sparse_encoder.py and unet.py. Feel free to have a quick look at the function to understand the core method. The old source code branch is also available here.

Quick Run

To train the full dataset, please refer to the OpenSceneFlow for raw data download and h5py files preparation.

Training

1. Prepare the demo train and val data for a quick run:

# around 1.3G
wget https://huggingface.co/kin-zhang/OpenSceneFlow/resolve/main/demo-data-v2.zip
unzip demo-data-v2.zip -d /home/kin/data/av2/h5py # to your data path

2. Follow the OpenSceneFlow to setup the environment or use docker.

3. Run the training with the following command (modify the data path accordingly):

python train.py model=deltaflow loss_fn=deltaflowLoss batch_size=4 num_frames=5 train_aug=True voxel_size="[0.15,0.15,0.15]" point_cloud_range="[-38.4,-38.4,-3,38.4,38.4,3]" optimizer.lr=2e-4 train_data=${demo_train_data_path} val_data=${demo_val_data_path}

Evaluation

Here is the pretrained weights link table for different training datasets (Note that these models are only used for research and reproducibility purposes only please follow the dataset license and privacy rules to use them):

Train Dataset	Pretrained ckpt Link
Argoverse 2	huggingface
Waymo Open Dataset	huggingface
nuScenes	huggingface

Please check the local evaluation result (raw terminal output screenshot) in this discussion thread. You can also run the evaluation by yourself with the following command with trained weights:

python eval.py checkpoint=${path_to_pretrained_weights} dataset_path=${demo_data_path}

Visualization

To make your own visualizations, please refer to the OpenSceneFlow for visualization instructions.

Cite & Acknowledgements

@inproceedings{zhang2025deltaflow,
title={{DeltaFlow}: An Efficient Multi-frame Scene Flow Estimation Method},
author={Zhang, Qingwen and Zhu, Xiaomeng and Zhang, Yushan and Cai, Yixi and Andersson, Olov and Jensfelt, Patric},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=T9qNDtvAJX}
}

This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation and Prosense (2020-02963) funded by Vinnova.