It’s good if you can to change them all. It’s not just about ranking, I would guess it won’t be long before it becomes a lot more than just a little warning in the address bar. I can see in the not to distant future a much more extreme warning, like the ones you get with a bad certificate.
So even if it’s not important now, I would put it on your todo list.