When trying to access the Graph Service Client using I am receiving the error :
Code: Authorization_RequestDenied
Message: Insufficient privile
In my case, delete user was not working. I took below steps & it started working for me.
Go to Azure Active Directory > Roles and administrators > Click on 'User administrator'  > click on '+ Add assignment' to add your app. (i.e. console app using AAD Graph REST API to interact with Azure Active Directory).
Hope it helps someone.