size mismatch for base model

[Redtides0]

Hi, when I use the provided base_model to make an inference, during initialization, it report an error

size mismatch for blocks.xx.action_model.keyboard_embed.0.weight: copying a param with shape torch.Size([128, 6]) from checkpoint, the shape in current model is torch.Size([128, 4]).

What do these two extra dimensions represent?

Could you please guide me on how to resolve this issue?
I truly appreciate your help.

[LeeKeyu]

Hi, did anyone figure out how to infer using base model instead of the distilled ones? thanks in advance

[LeeKeyu]

Hi, when I use the provided base_model to make an inference, during initialization, it report an error

size mismatch for blocks.xx.action_model.keyboard_embed.0.weight: copying a param with shape torch.Size([128, 6]) from checkpoint, the shape in current model is torch.Size([128, 4]).

What do these two extra dimensions represent?

Could you please guide me on how to resolve this issue? I truly appreciate your help.

@Redtides0 I found this may be "keyboard_dim_in" in the model config. However, after changing it from 4 to 6, I got an error RuntimeError: mat1 and mat2 shapes cannot be multiplied (21x4 and 6x128). Cannot fix it...

[Redtides0]

Hi, when I use the provided base_model to make an inference, during initialization, it report an error

size mismatch for blocks.xx.action_model.keyboard_embed.0.weight: copying a param with shape torch.Size([128, 6]) from checkpoint, the shape in current model is torch.Size([128, 4]).

What do these two extra dimensions represent?
Could you please guide me on how to resolve this issue? I truly appreciate your help.

@Redtides0 I found this may be "keyboard_dim_in" in the model config. However, after changing it from 4 to 6, I got an error RuntimeError: mat1 and mat2 shapes cannot be multiplied (21x4 and 6x128). Cannot fix it...

@LeeKeyu Hi, I tried to fix that by changing the combine_data in Matrix-Game-2/utils/conditions.py, however, I found that the provided inference.py can not be directly used for the base model, as it just outputs pure noise.