The problem in your code is that at each back button click you add a new Panel. So if the user clicks 3 times the button you will have three panesl.
The Yes button works always because you close the app. But the No button only hides the last panel added, leaving the others on the screen making you feel that it has no effect.
Attached you find a modofied version, where the panel is initialized only once in Activity_Create, and when the back button is clicked the program checks if the panel already visible and has no effect in this case and otherwise set the panel to visible.
Best regards.