One of the things I have no clue about recent reporting is why people imply Signal is insecure if a bad actor has access to your device.

This isn't a problem with Signal's communication, you're just totally and completely fucked if someone has your unlocked device anyway.