I don't use the bluetooth module on my Overkill, but from what I saw in the Overkill manual password protection is only possible on the app-side and only the iPhone version at that.
If security is a concern, I'd add a switch between the BMS and the BT module to disable the BT module when you are not using it.