When a view's color property is set to "Default" (in the designer the value will appear as a gray/white checkered box with a red diagonal line across it), then it will use the device's "Theme" color for that view's color property.
By having all the colors of a view set to their default value, then your app "should" look ok on various devices because the maker of each device should have created a proper "theme" where the colors don't clash with each other.
However, if you manually specify a color in the designer or code, then you are overriding the devices theme color, and even though your chosen color looks fine on your device (using your device's theme), your manually specified color could easily clash with the color themes of other devices as you have seen.
Additionally, as you have noticed, some views don't allow you to specify all the colors of a view, like the background. So this adds to the difficulty on preventing clashing colors when run on a variety of devices.
So, you can either set all views to use the "default" color values so all the views in your app should be readable on various devices using each device's color theme (which can vary from light or dark themes)...
...Or you can create your own "theme" in your apps manifest that will specify all the default colors (including non-runtime modifiable colors) of your app so you can prevent any clashes and make your app's appearance be consistent across devices.
Here is an example of how to create a theme in your apps manifest:
https://www.b4x.com/android/forum/threads/theme-colors.87716/