I finally figured out why my PS5 Controller wasn't vibrating when using it for games on PC.

I had disabled the speaker on the controller via the sound options in the control panel. Apparently, this determines whether or not the blasted thing will have its vibration function.