Consider a scenario where two backup hosts exist:

• One for regular snapshots, situated in the same datacenter as the appliance.
• One for taking less frequent off-site backups (where copying the snapshots from the host above is not viable)

There's currently no check in ghe-backup which would prevent the simultaneous execution from occurring from different hosts, however it looks like there's a least one place where a semaphore file placed on the appliance (to suspend repository maintenance) may be prematurely removed if the backup tasks were to overlap during specific phases: https://github.com/github/backup-utils/blob/master/share/github-backup-utils/ghe-gc-enable#L35. For example if:

1. Backup A started the repository backup phase and creates a semaphore file.
2. The background maintenance queue on GitHub Enterprise is suspended.
3. Backup B commences its backup and overwrites the same semaphore file.
4. Backup A completes its repository backup phase and removes the semaphore file.
5. The background maintenance queue on GitHub Enterprise starts draining.
6. Repositories data is potentially modified by a maintenance task.
7. Backup B completes its repository backup phase, there is no semaphore file to remove.

Is this the only place where such a condition exists? If so, would it be possible to keep a count of the active backup tasks instead and remove the file only when it reaches 0? Or are there other considerations / complexities to take into account which makes the approach unpredictable or unreliable?