rustc-dev component is broken by missing literal-escaper

[RalfJung]

Trying to load the rustc-dev packages with cargo leads to an error:

    Updating crates.io index
error: failed to get `literal-escaper` as a dependency of package `rustc_ast v0.0.0 (/home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc_ast)`
    ... which satisfies path dependency `rustc_ast` of package `rustc_codegen_ssa v0.0.0 (/home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc_codegen_ssa)`
    ... which satisfies path dependency `rustc_codegen_ssa` of package `rustc-main v0.0.0 (/home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/compiler/rustc)`

Caused by:
  failed to load source for dependency `literal-escaper`

Caused by:
  Unable to update /home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/library/literal-escaper

Caused by:
  failed to read `/home/r/.rustup/toolchains/miri/lib/rustlib/rustc-src/rust/library/literal-escaper/Cargo.toml`

Caused by:
  No such file or directory (os error 2)

This is caused by #136355: compiler crates cannot have path dependencies outside the compiler folder.

Cc @GuillaumeGomez @Amanieu

[GuillaumeGomez]

cc @rust-lang/bootstrap

[Kobzol]

Hmm, I'm not sure if we want to ship library sources in the rustc-dev rustup component (which contains only the compiler sources currently).

[Kobzol]

But I guess why not? Anyone have any concerns about doing that?

[bjorn3]

It should also be possible to add just library/literal-escaper, right?

[Amanieu]

One possible solution:

• rustc_lexer should not depend on literal-escaper via a Cargo.toml dependency, it should instead import it directly with extern crate literal_escaper to get the one from the library sysroot.
• For cfg(bootstrap) where literal-escaper is not available in the library sysroot just use #[path] to include the code directly.

[Veykril]

rustc_lexer should not depend on literal-escaper via a Cargo.toml dependency, it should instead import it directly with extern crate literal_escaper to get the one from the library sysroot.

This will break rustc_lexer from compiling on stable I believe. Not that it matters I suppose as the change done here in general makes it impossible to publish the crate to crates-io now for rust-analyzer :/

[Kobzol]

It should also be possible to add just library/literal-escaper, right?

Sure, we could selectively add only the required crates.

[bjorn3]

Another option would be to publish literal-escaper to crates.io. That would also allow proc-macro2 to use it. proc-macro2 needs to support not using libproc_macro, so it can't unconditionally use literal escaping through libproc_macro.

[GuillaumeGomez]

Publishing on crates.io seems like the simplest solution. Should I do it?

[Kobzol]

That doesn't sound like the simplest solution to me, tbh 😅 Unless the code is done and will never ever change? Otherwise we should probably have some setup for deploying it, managing the crates.io token, figuring out who will own the crate etc. (it probably shouldn't be owned by your GH account).

[GuillaumeGomez]

Very good point. Then I'll keep quiet in the back and step in if my help is needed. 😆