[draft] i8 quantization experiment

[Vegetable26]

Changes to the core algorithm:

• Implements quantization from f32 -> i8 for centroid assignment
• It still uses f32 for centroid update
• Adds an optional comparison for how frequently the f32 and i8 assignments end up being the same
• Also adds an option where we compute KNN with i8 quantized vectors. We then perform assignment using the full fidelity f32 vectors for the K candidates.
• This PR uses XNNPack for the i8 matmul implementation

I attached the experimental results for a cohere 2M benchmark (my Mac does not have enough memory for the full benchmark). Some interesting results:

• E2E recall rates are roughly the same across all implementations
• xnnpack f32 = 260 seconds, cblas_sgemm = 111 seconds, xnnpack i8 = 65 seconds.
• Note xnnpack to cblas is not a fully fair comparison due to cblas_sgemm using the AMX coprocessor and xnnpack only using sdot. But extrapolating from this we do see a ~4x speedup from f32 -> i8 quantization within xnnpack which is roughly expected (as the we are quantizing to 25% of the original size: 32 -> 8 bits).
• The original std::partial_sort implementation for top-K was quite inefficient. I re-implemented a new version that is lower overhead (more aggressively filters invalid candidates)
• Finally we see that i8 assignments with top-10 candidates (then rescoring with f32) provides almost the exact assignments as the f32 implementation. And doing i8 top-10 (as opposed to i8 top-1) candidate assignment adds very little overhead. We do see that if we only consider the top-1 candidate, i8 and f32 assignments do disagree a bit more

[Vegetable26]

nit: mainly for my own setup

[Vegetable26]

nit: mainly for my own setup

[Vegetable26]

nit: most of the changes here are for my own setup (quickly running a few different experiments)