I have been using a few B4J apps that access the serial port on both Windows and a variety of Linux Platforms (Ubuntu, Debian, etc.). Never had any issues with permissions. The default permission for the serial ports work fine. The application always runs under a sudo user. The serial ports tested are native serial ports as well as Virtual USB serial ports (/dev/ttyUSBx) using a serial/USB adapter. No issues with either.
Yes, depending on the driver, the program will crash if the USB/Serial adapter is unplugged. In some cases (driver/hardware) it will throw an exception and then automatically recover when the adapter is plugged back in. I have been unable to catch these exceptions using Try-Catch. So what I do is run the application as a systemd service and set it up so if the program crashes the service is restarted automatically. This will happen continuously until the devices is plugged in and detected again.