[Feature] Improve signal and shutdown handling

[kaimast]

These changes are part of #3874, but were moved to their own PR to reduce the size of the latter.

Motivation

Before, #4021, snarkOS nodes would call std::process::exit before shutting down. This was problematic "because this function never returns [and] terminates the process, no destructors on the current stack or any other thread’s stack will be run" (source).
The code worked around this by sleeping for multiple seconds during shutdown. Such a sleep may be too long in a test setting and too short in production. Additionally, the lack of a clean shutdown makes it more complicated to perform certain tasks at shutdown, such as caching the block tree on disk or flushing the log file.

#4021, as far as I understand, was intended to be a quick fix until this slightly cleaner version is merged. This PR improves on the former by encapsulating all signal handling logic in a dedicated module and avoids spawning the main node logic in a detached task, which makes catching panics easier.
For snarkos-display shutting down using Escape will now work the same as with Ctrl+C.

Proposed Changes

In the current design, the code passes around an AtomicBool as a shutdown flag. This design generally works, but does not allow waiting for the flag to be set. This PR introduces a SignalHandler struct and a Stoppable trait.

The Stoppable trait serves the same function as the AtomicBool did before, but it is a little clearer what its purpose is from its struct and function names alone.

The SignalHandler struct implements the Stoppable trait and also launches a background task that wait for Ctrl+C.

Testing

To test, I verified that shut down works correctly in multiple scenarios. "correctly" means that the process exists cleanly, and that the on-disk state is not corrupted. The latter was verified by starting the node again and ensuring it can load the stored state fine.

I went through the following scenarios

• started a production node and connected to canary
• ran the production node with and without terminal UI (display)
• ran production node sync with and without CDN

Additionally, I ran a development network and verified that nodes all nodes shut down cleanly without getting stuck or returning nonzero exit codes.

[ljedrz]

Just one small change left, and it's good to go 👍.

[ljedrz]

Left a few final nits, but basically LGTM 👌.

[vicsn]

Can you note in the PR README which manual tests were run? Are they comprehensive?

[kaimast]

Can you note in the PR README which manual tests were run? Are they comprehensive?

Done!

[vicsn]

LGTM!