Simplify retry extension

[Evangelink]

In future, we should be able to simplify this. We don't need a custom filter at all. Instead, we can use --filter-uid and pass all the UIDs. We need to be very careful though when the list is long, in which case we might need to create a response file for that. And in that case we should carefully decide where to write such file to not leak any info to other potential users on the same system.

Originally posted by @Youssef1313 in #7409 (comment)