Hi funker,
If you created the panel using the designer, you can click on it to hightlight it. Next right click with the cursor on the panel and select the "Cut" item from the popup window. That will cut it out and now you can save your changes before closing the designer.
Note: If you delete the panel, all other views on that panel will also be deleted. If you want to delete the panel and keep all the views that were on the panel, I would drag them off the panel prior to deleting that panel and making sure the "Parent" property for them are on "Activity".
Hope this helps.