Change `serde` dependency to `serde_core` where applicable

[crepererum]

Is your feature request related to a problem or challenge?

Currently some datafusion crates depend on serde, which may take a long time to compile if -- due to feature unification -- datafusion users opted into the derive features. See serde-rs/serde#2608 . However we often don't need derive and hence we could depend on the actual core part of serde called serde_derive.

Describe the solution you'd like

Check our dependencies and where we don't use derive, change the dependency from serde to serde_derive.

Describe alternatives you've considered

-

Additional context

• feat: Extract serde_core out of serde crate serde-rs/serde#2608
• Change serde dependency to serde_core where applicable arrow-rs#8451

[alamb]

Possibly related to

• Building project takes a *long* time (esp compilation time for datafusion core crate) #13814

[AdamGS]

take

[AdamGS]

The underlying issue in arrow-rs should be resolved first, I don't think we'll get much benefit if arrow-schema keeps pulling it in, see this comment.