How about requiring a shared secret - or a way to generate an invite code.
So registration, instead of just open and closed, will have an additional option - by-invite, and the command line on the server will have an option to generate an invite code with an expiration date (or an unlimited one).
This is a very common practice in sharable self-hosted solutions - allowing it to be ‘open’ to use outside of the LAN/VPN (like on travel) - but also allow friends and family access by being selective.
Of course, any other security - like blocking brute force tactics is on the server admin to setup.