You’re describing lynching. No change in law is needed, juries are free to nullify the law. Governors and the President also have the power to pardon. It’s how the system was designed to work.