Isolating everything on a panel in a single action is not something I've been able to do either.
Ctrl-A selects EVERYTHING, and then you can Ctrl-Click the items you don't want, to deselect them.
MS Visual Studio allows us to drag-draw a rectangle, and the result is that it selects everything within that rectangle, but I can't seem to find a way to do such a thing in B4A.
I have found the best way for me is when the designer is open to choose WINDOWS / VIEWS TREE from the pull down menu on the designer. In this way you can work with check-boxes, rather than dealing with select/deselect of items on the panel which may overlap and further complicate selection. VIEWS TREE shows which items are subordinate to any panel, making it much easier for me.