Training code and mode release?

[pulkitkumar95]

Hey!

Do you have any plans on releasing the training code and the final model?

Pulkit

[natsunoshion]

Thank you for your interest in our work!

We used Qwen2-VL-Finetune for fine-tuning. Our public dataset includes the fine-tuning config we used for Qwen2.5-VL. You can follow the instructions at Qwen2-VL-Finetune to configure it accordingly.

As for the model weights after SFT, we will be uploading them soon—stay tuned!

[pulkitkumar95]

Oh perfect, thank you.

And how about for InternVL3?

[natsunoshion]

Oh perfect, thank you.

And how about for InternVL3?

In our MotionVid-QA dataset fine-tuning experiments, we have not yet included InternVL3. However, as we plan to continue updating this project, training InternVL3 may be included in future work.

[pulkitkumar95]

In Tables 2 and 3 the last row is Internvl3 + MotionSight, I was wondering how that was computed?

[natsunoshion]

In Tables 2 and 3 the last row is Internvl3 + MotionSight, I was wondering how that was computed?

Our MotionSight is the pipeline instead of MotionVid-QA fine-tuning. Our fine-tuned model is called MotionChat.

[pulkitkumar95]

Got it! I was also able to reproduce the Qwen results, so thank you for that information.

I have one final question, how did you swerve internvl3-78B? I am trying to do it on an H100 GPU, but I am facing out-of-memory issues. Is there anything specific you did to be able to work with it?

[natsunoshion]

InternVL3-78B is a large model. It is recommended to use at least 2 H100 GPUs (providing a total of around 160GB GPU memory), and to set --tp 2 when deploying with LMDeploy.

[natsunoshion]

Sorry for replying late. We have uploaded MotionChat to ModelScope and expect to migrate it to Hugging Face soon. The more detailed training config has also been uploaded under ft_config.