Skip to content

compile_examples hangs when compiling many targets due to stderr pipe buffer filling up #435

@basil

Description

@basil

compile_examples spawns cargo build --message-format=json and reads stdout for JSON messages, but does not drain stderr. When compiling many example targets, Cargo's "Compiling..." progress messages fill the 64KB OS pipe buffer, and the subprocess blocks on write() to stderr indefinitely.

Stack trace of blocked subprocess

0x00007f5b5b3c99a2 __syscall_cancel_arch+0x32                                                                                              
0x00007f5b5b3bdc3c __internal_syscall_cancel+0x5c                                                                                          
0x00007f5b5b3bdc84 __syscall_cancel+0x14                                                                                                   
0x00007f5b5b4381fe __write+0x1e                                                                                                            
0x00005648b54606ff _$LT$std..io..stdio..StderrLock$u20$as$u20$std..io..Write$GT$::write_all::h1861e61dff6c25b0+0x6f                        
0x00005648b4d4e4b0 <anstream::auto::AutoStream<std::io::stdio::Stderr> as std::io::Write>::write_all+0x1a0                                 
0x00005648b4a31e3b <cargo::core::shell::ShellOut>::message_stderr+0x19b                                                                    
0x00005648b4a66cd5 <cargo::core::compiler::job_queue::DrainState>::drain_the_queue+0x2515                                                  
0x00005648b4a4f08a <cargo::core::compiler::job_queue::JobQueue>::execute+0x1eca                                                            
0x00005648b49daa79 <cargo::core::compiler::build_runner::BuildRunner>::compile+0xd39                                                       
0x00005648b4b3eca0 cargo::ops::cargo_compile::compile_ws+0xc90                                                                             
0x00005648b4b43391 cargo::ops::cargo_compile::compile_with_exec+0x51                                                                       
0x00005648b4b4aabb cargo::ops::cargo_compile::compile+0x5b                                                                                 
0x00005648b4ed6929 cargo::commands::build::exec+0x4b9                                                                                      
0x00005648b4ead9df <cargo::cli::Exec>::exec+0x55f                                                                                          
0x00005648b4eaa04f cargo::main+0x37ef                                                                                                      
0x00005648b4e85593 std::sys::backtrace::__rust_begin_short_backtrace::<fn(), ()>+0x3                                                       
0x00005648b4e998a9 std::rt::lang_start::<()>::{closure#0}+0x9                                                                              
0x00005648b542cca6 std::rt::lang_start_internal::h31d4445a63e30e2c+0x4b6                                                                   
0x00005648b4f1971c main+0x2c                                                                                                               
0x00007f5b5b3525b5 __libc_start_call_main+0x75                                                                                             
0x00007f5b5b352668 __libc_start_main@@GLIBC_2.34+0x88                                                                                      
0x00005648b44ba4c5 _start+0x25                                                                                                             

Workaround

Pass -q to suppress stderr output:

snapbox::cmd::compile_examples(["-q"])

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions