Prevent users from accessing Site Settings application page

Just this week, I accidentally found out that users were able to access Site Settings page, with some sections open out publicly accessible, for example the Regional Settings column! This raise a security threat as we don't want users to be able to touch these settings. ...