but the right of 'association'

That raises an interesting question. If the owner / manager of the family themed restaurant didn't want customers carrying in their guns and weapons, you'd think the same crowd who crows about the right of those places to serve and not serve whoever they want (wink wink i.e. African Americans), would be consistent, and abide by those wishes (don't bring in your arsenal while having your meal here)