Small object is not available during put load

[alexvanin]

Expected Behavior

S3 Gateway receives small (under 10 KiB) objects with S3 credentials when it is evicted from the cache during put object load from 25 or 50 workers per node and 1MiB object size.

Current Behavior

S3 Gateway can't receive credential objects and skips some requests for a while

Steps to Reproduce (for bugs)

1. Prepare 4-node cluster
2. Create single container
3. Start uploading 1 MiB objects with 25 to 50 workers to this single container through S3
4. See some interrupts during this load.

Context

Cluster behavior seems very strange. FrostFS CLI receives object not found error while trying to HEAD the credentials objects.
On GET requests, it receives timeouts and sometimes credential object is returned

All 4 nodes behave the same way. Their shards are not degraded at that moment.

Your Environment

Hard metal cluster