I cant answer you about the EVGA power supply. I went with a Seasonic 1300watt PLatinum.  But I have the exact same UPS on my system which is a Threadripper with 32gigs of ram, 2 NVME ssd's a 3080TI Hybrid.  Point being my power draw is probably the same and maybe more than yours, and my UPS does not switch for any reason.  The big question is the problem in the power supply (Seems doubtful) or is the UPS not able to handle the current, or even the possibility that your AC plug that the UPS is plugged into is sagging due to the initial current draw and the UPS is properly switching to battery.  Id change the outlet I'm plugged in to first, and I mean go to a different wall outlet.  If that fixes it I'd take that outlet apart for a start and check it for loose wiring, after that better get an electrician.

I suppose the UPS could be failing.  I've been using that particular model since it was first introduced a few years back, I've currently got 4 of them, of the 4 I have had one fail, but it failed hard and had to be replaced. I think I'd try a new UPS then, it just seems pretty far fetched that the PS would be causing this and the system runs fine otherwise.  Not saying impossible, just seems like the odds would be pretty low. 
 
Oh I forgot to mention that I also had two different EVGA power supply's (Supernova Platinum 1000w) neither of them exhibited the behavior you describe when connected to that same model UPS.