rust_test: shard by stable name hash

[bolinfest]

Why

The Rust test sharding wrapper was assigning tests to shards by numeric position in the libtest --list --format terse output. That makes shard assignment depend on list order: changing the order, or inserting a test before existing tests, can move unrelated tests to different shards.

For CI workflows that shard expensive Rust test targets, an individual test name should map to a stable shard bucket. Some downstream users also need separate Bazel test rule labels per shard so systems like BuildBuddy can report timing and flakiness by shard label rather than only by the aggregate test target.

What

• Assigns tests to shards by a stable FNV-1a hash of the test name modulo the total shard count.
• Sorts the discovered libtest test names before executing a shard so in-shard execution order is deterministic.
• Keeps support for Bazel's native TEST_TOTAL_SHARDS / TEST_SHARD_INDEX env.
• Adds explicit RULES_RUST_TEST_TOTAL_SHARDS / RULES_RUST_TEST_SHARD_INDEX env support for downstream macros that generate separate shard targets.
• Adds a Windows manifest lookup fallback so nested launcher rules can still find the real test binary through the active Bazel runfiles manifest.
• Uses explicit decimal UInt64 constants in the Windows PowerShell FNV hash expression so the 32-bit mask cannot be interpreted as -1.
• Uses TEST_TMPDIR plus a per-wrapper temp directory in the Windows sharding wrapper so parallel shards do not collide on shared %TEMP%\rust_test_list_*.txt files.
• Updates both the Unix and Windows sharding wrappers.
• Updates the experimental_enable_sharding docs to describe name-hash sharding and both env modes.
• Adds a focused wrapper test that verifies shard assignments are unchanged when libtest list order changes and when a new test is inserted.

Examples

With native Bazel sharding:

rust_test(
    name = "sharded_integration_test",
    srcs = ["sharded_test.rs"],
    experimental_enable_sharding = True,
    shard_count = 3,
)

Bazel still exposes a single test target, //test/unit/test_sharding:sharded_integration_test; it does not generate separate rule names with shard numbers appended. At execution time, Bazel runs that target as three shard invocations by setting:

TEST_TOTAL_SHARDS=3 TEST_SHARD_INDEX=0
TEST_TOTAL_SHARDS=3 TEST_SHARD_INDEX=1
TEST_TOTAL_SHARDS=3 TEST_SHARD_INDEX=2

For separate shard rule labels, a downstream macro can generate one compiled rust_test binary and wrap it with lightweight test rules that set the explicit shard env:

rust_test(
    name = "core-all-test-bin",
    srcs = ["tests/all.rs"],
    experimental_enable_sharding = True,
    tags = ["manual"],
)

test_binary_test(
    name = "core-all-test-shard-1-of-8",
    test_bin = ":core-all-test-bin",
    env = {
        "RULES_RUST_TEST_TOTAL_SHARDS": "8",
        "RULES_RUST_TEST_SHARD_INDEX": "0",
    },
)

test_binary_test is a downstream wrapper rule in this example, not a new rules_rust API. This shape gives BuildBuddy and GitHub reruns concrete labels like //codex-rs/core:core-all-test-shard-1-of-8 while still compiling the Rust test crate once as //codex-rs/core:core-all-test-bin.

In both modes, the wrapper lists libtest names, computes fnv1a32(test_name) % total_shards, and runs only the tests whose bucket matches the shard index. Sorting the listed names only makes the order within each shard deterministic; the shard bucket itself depends on the test name hash, not on list position.

Verification

• pre-commit run --files rust/private/rust.bzl rust/private/test_sharding_wrapper.bat rust/private/test_sharding_wrapper.sh test/unit/test_sharding/fake_libtest_binary.sh test/unit/test_sharding/test_sharding.bzl test/unit/test_sharding/test_sharding_wrapper_hashes_sorted_names.sh
• bazel test //test/unit/test_sharding:test_sharding_test_suite