I see this has been discussed before. I wish RW would activate “Publish” after only changing the web icons. I work around this by making a simple change to my site. I add and then delete a text stack. That makes RW’s Publish button active. It’s a work around, and I wish RW just knew I needed to Publish. I have tested, and this does work, all of my web icons appear in my browser. Note, I do need to clear the browser cache, quit, and then reopen the browser.
I may have spoken too soon. I do see one page where the pinned tab is not appearing, so maybe it is true that a “Re-Publish All Files” is necessary.