I have a use-case that can be repaired, but the logic for repairing it is best done in a higher level of abstraction. Just throwing exception/failure is ugly because it's hard to pass back up through the stack the information needed for the parent-layer to know how to fix it.
To explain the the specifics, I have a program that controls multiple pieces of hardware. The 'top' layer is responsible for calculating how to task each piece of hardware to work together, how to generate 'paths' of data across hardware etc. My next layer includes threads for each individual device. each device thread generates and caches device state (fetched from hardware and config files). Devices process requests from the application, figure out how to update the hardware to make those requests happen, and then will modify the cached state to reflect the changes made. Finally I have plugins which send physical requests to hardware.
This works fine usually, requests are sequential to avoid data race issues. We only pass information down layers, we have no way of getting async updates back up the layer. However, it's very unlikely, but possible, that someone could physically change the hardware without going through our control system. We will wait until the plugin detects an error trying to handle a request, and only then try updating our state for that piece of hardware to see if a change caused the failure (constant monitoring is a later feature).
My problem is that when a change is detected on one device, that usually means a change was made to connected devices. When I detect this, I want to trace the change to connected device, update their state etc until I found all changes (this doesn't require updating all devices, just connected ones). Each device thread is only responsible for one device, so they can't trace the change through other devices without propagating the exception up a layer. However, I need to know exactly what changes were detected on the device to know which connected devices I should be refreshing the state of. If I just throw an exception, this knowledge is lost and must be recalculated, which is a small but real cost due to time it takes to communicate with hardware.
The tempting thing to do is to call a method in my application layer saying "this change was detected, go update everything else related to it". However, this makes drivers aware of the layer above them, which is not good. Another option is to simply propagate the error up and let my application layer notice the error and then go back to the same device to ask what changed and update all it's state, then re-calculate the changes to be made and make the request back down to the device level. The first is an obvious code smell, the second seems like more code, redundant logic, and small speed penalties (it takes a little time to get state from hardware). Are there some other options?