Since about one week we are experiencing slow responses from our Proxmox Backup Servers. After debugging, we found that the issue is in the authenticationlayer of Proxmox Backup Server. We've contacted Proxmox and a fix is on the way, we're currently waiting for a new release that contains this fix. In the meantime, there is a workround: do not let your machines login using username@pbs and a password, but let them login with a API-token. This skips the part of the authenticationlayer that causes the delay and results in the responsetimes you can expect.
The technichal background of this issue is that when you loging with username and password, PBS also issues a TFA-check. While executing this check, PBS reads a file in which the TFA-settings are configured. The reading of this file is done (and this is the bug) byte-by-byte, which causes a significant delay. As we are processing 10/15 authentication-requests per second, this causes rather much useless load. An API-token cannot be configured with TFA and thus does not cause this delay.
If you want to read more about this bug, read these threads on the pbs-devel mailinglist: