Once we support passing ImageStacks from process -> process, we need a way to coordinate how much parallelism the individual processes kick off. Each process will see a system with N cpus and start up N threads, which won't work out well.
One quick and dirty hack is to suppress multithreading when we do multiprocessing + ImageStacks. However, if one process takes a lot longer to process than the other processes, then we waste compute at the end of the processing.
Ideally, the solution is to have a centralized semaphore that each thread needs to acquire to start. Each worker process can start up N threads, but the system as a whole can only run N threads concurrently.
Once we support passing ImageStacks from process -> process, we need a way to coordinate how much parallelism the individual processes kick off. Each process will see a system with N cpus and start up N threads, which won't work out well.
One quick and dirty hack is to suppress multithreading when we do multiprocessing + ImageStacks. However, if one process takes a lot longer to process than the other processes, then we waste compute at the end of the processing.
Ideally, the solution is to have a centralized semaphore that each thread needs to acquire to start. Each worker process can start up N threads, but the system as a whole can only run N threads concurrently.