Tracking issue for migrating from Glacier to compiletest ICE/crash tracking

[fmease]

Tracks the necessary steps to "fully"¹ transition from rust-lang/glacier to compiletest ICE/crash tracking inside tests/crashes/.
See also this Zulip topic.

Steps

• Implement rustc ICE/crash tracking: compiletest ice tracking #122997
• Implement rustdoc ICE/crash tracking
• Implement rustfmt ICE/crash tracking (questionable)
• Implement ICE/crash tracking via run-make/rmake (to supersede Glacier's shell scripts)
• Archive rust-lang/glacier: 🧊 Put glacier on ice 🧊 team#1459
• Sunset triagebot support for Glacier
  • Disable triagebot's glacier handler #141427
  • Remove glacier integration triagebot#2008
• Update the rustc-dev-guide: Flesh out sections about crashes tests and update mentions of glacier rustc-dev-guide#2413
  • Remove mentions of Glacier
  • Add small section about tests/crashes/: Overview, rough procedure (adding //@ known-bug, S-bug-has-test Status: This bug is tracked inside the repo by a `known-bug` test. )
  • Rename "crashes tests" to "crash tests" rustc-dev-guide#2588
  • Mention tests/crashes/ in places where we reference E-needs-test Call for participation: An issue has been fixed and does not reproduce, but no test has been added. (EDIT(fmease): What did I mean by this?)
• Update the Rust Forge
  • Remove mentions of Glacier (namely the triagebot section): Remove mention of now removed triagebot glacier integration rust-forge#856
  • Maybe mention the new system in the issue triaging chapter (EDIT(fmease): The Forge, really?)
• If technically feasible, add a CI check that a PR fixing crash tests mentions all fixed issues via GH auto-close keywords like Fixes to prevent the accumulation of "ghost issues". EDIT(fmease): triagebot nowadays posts a general message if tests/crashes/ has been modified. Doesn't that suffice?

Unresolved Questions

• Should we implement rustdoc support via subdirectories/tests suites (e.g., tests/crashes/rustdoc/) or via a new compiletest directive (e.g., //@ bin: rustdoc)
  • If we do the former, should we move rustc tests from tests/crashes/ to tests/crashes/rustc/?
• Should we define precise procedures for adding new tests to tests/crashes?

Footnotes

1. Glacier|_{rust-lang/rust}, i.e., restriction of Glacier to rust-lang/rust (excluding git submodules/subtrees), i.e., Glacier without support for rustfmt, Cargo, Clippy, Miri ICE/crash tracking ↩

[fmease]

cc @matthiaskrgr

[matthiaskrgr]

I'm not sure if it makes a lot of sense to track crashes inside the rustc repo for tools (rustfmt, clippy, miri etc) that are actually living inside their own repos because in most cases I guess you would "accidentally" fix something during a submodule/tree sync?

[fmease]

Good point. I guess that means T-rustfmt/T-cargo/... for example will need to introduce their own mechanism assuming they're interested in doing that and that "we" won't try to find a Glacier replacement for them now that it's officially archived.