[diskann-garnet] Allow configurable metric types#824
Merged
tiagonapoli merged 13 commits intomainfrom Mar 12, 2026
Merged
Conversation
Ported from fork branch tiagonapoli/diskann-garnet/metric-types. Adds metric type support to the diskann-garnet FFI layer, including tests for L2, cosine, and inner-product metrics, and invalid metric type handling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tiagonapoli
commented
Mar 11, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR makes the Garnet DiskANN FFI/index creation support selectable distance metrics (instead of always using cosine), and updates tests/docs accordingly.
Changes:
- Add a
metric_typeparameter to thecreate_indexFFI and thread it throughGarnetProviderso distance/query computers use the requested metric. - Fix test store multi-read parsing (
posadvancement) and update affected expectations. - Expand FFI test coverage for valid/invalid metric values and add a grid search test using variable-length (string) external IDs; update FFI design docs.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| diskann-garnet/src/test_utils.rs | Fixes multi-read buffer parsing and adjusts tests to match correct behavior; updates provider construction for new metric parameter. |
| diskann-garnet/src/provider.rs | Stores selected metric on GarnetProvider and uses it when building distance/query computers. |
| diskann-garnet/src/lib.rs | Extends create_index FFI to accept a raw metric_type and passes it into provider construction. |
| diskann-garnet/src/ffi_tests.rs | Adds tests for metric validation and adds a string-ID grid search test; updates existing calls for new FFI signature. |
| diskann-garnet/docs/ffi-design.rs | Documents the new metric_type argument and adds the repository license header. |
Comments suppressed due to low confidence (2)
diskann-garnet/src/lib.rs:223
create_indexnow acceptsmetric_type, but the config is still built withconfig::PruneKind::TriangleInequality. ForMetric::InnerProduct, DiskANN expects occluding-style pruning; using the triangle-inequality prune kind can produce incorrect/low-quality graphs for that metric. Consider usingconfig::PruneKind::from_metric(metric_type)(or equivalent match) when building the config.
target_degree,
config::MaxDegree::Value(max_degree as usize),
l_build as usize,
config::PruneKind::TriangleInequality,
)
diskann-garnet/docs/ffi-design.rs:37
- The docs signature for
create_indexis still missing thermw_callbackparameter that exists in the actual exported function (diskann-garnet/src/lib.rs). Since this doc block was updated formetric_type, it should be brought back in sync with the real FFI signature.
extern "C" fn create_index(
context: u64,
dimensions: u32,
reduce_dims: u32,
quant_type: SomeCStyleEnumeration,
metric_type: i32,
build_exploration_factor: u32,
num_links: u32,
read_callback: unsafe extern "C" fn(u64, *const u8, usize, *mut u8, usize) -> i32,
write_callback: unsafe extern "C" fn(u64, *const u8, usize, *const u8, usize) -> bool,
delete_callback: unsafe extern "C" fn(u64, *const u8, usize) -> bool,
) -> *mut c_void;
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
metajack
requested changes
Mar 11, 2026
added 4 commits
March 11, 2026 17:16
Signed-off-by: Tiago Napoli <tiagonapoli@microsoft.com>
2c7819d to
ff888e3
Compare
metajack
approved these changes
Mar 12, 2026
added 2 commits
March 12, 2026 10:31
Signed-off-by: Tiago Napoli <tiagonapoli@microsoft.com>
metajack
approved these changes
Mar 12, 2026
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #824 +/- ##
==========================================
+ Coverage 88.96% 89.11% +0.14%
==========================================
Files 442 443 +1
Lines 81906 83354 +1448
==========================================
+ Hits 72868 74281 +1413
- Misses 9038 9073 +35
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Changed insert_f32_vector, insert_f32_vector_str, and do_search from unsafe fn to safe fn with internal unsafe blocks. Removed unnecessary unsafe wrappers at all call sites, keeping only drop_index in unsafe. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
8d9b25a to
15ea3c8
Compare
Remove coordinate suffix from grid vector IDs, changing from 'grid_vector_00000001_dim3_at_0_0_0' to 'grid_vector_00000001_dim3'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
hildebrandmw
approved these changes
Mar 12, 2026
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move grid/circle recall helpers and 7 test functions from ffi_tests.rs into a dedicated ffi_recall_tests.rs module for better organization. Includes prior uncommitted refactoring: - Epsilon-based distance bucketing for float-safe comparisons - Unified brute_force_knn with distance_fn parameter - Common run_recall function for grid and circle tests - Named variables for search and index creation parameters - parse_string_ids buffer consumption assertion - Imperative loops throughout Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace EPSILON division (/ 0.001) with BUCKET_SCALE multiplication (* 1000.0) for more precise and faster floating-point bucketing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use diskann_vector::distance::{SquaredL2, Cosine} via PureDistanceFunction
trait instead of custom squared_l2 and cosine_distance helper functions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
hildebrandmw
approved these changes
Mar 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this implement/fix? Briefly explain your changes.
Allow Garnet to configure the Metric type to be used by DiskANN
Any other comments?