Part of that is because the OS doesn't allow the granularity to get permissions from the user for just one thing.
A best practice for developers is to wait until the specific permission is needed (e.g., user wants to send an email) before asking for the permission (e.g., access address book). Even then, it can provide a user-friendly explanation to the user either through the OS (iOS provides this for some permissions) or display the explanation and get app-level approval before even asking the OS.
Bottom line - I wouldn't worry about it if the app is popular or from a well-known company.