I tried the above answers in Android Studio 3.0, but it seems AS 3.0 doesnt allow to change in the IDE screen, or not sure it was some problem with my installation. So I opened the AVD in disk and opened the config.ini, and changed the ramsize there. It worked.