I am trying to build a customized AOSP (on Android M, api 23) using Android Studio because I want to run instrumented tests on custom apps running on the hardware. This is o
Yes, it's possible to build an app APK and a test app APK with Gradle in AOSP, but the difficulty depends on their dependencies, i.e. where the Makefile may refer to a pre-built binary or another module in AOSP, there may not be an equivalent dependency in the public Maven repos so you'll need to pull in those dependencies by other means. And if your app is a system app, you'll need to provide the platform signature for the signing step.
a) What is the Gradle error you're getting? And which Gradle task gives you that error?
b) You can set up a Makefile to just build the instrumented APK, for example see how it's done here. You can also use the Trade Federation framework to handle the execution and report generation - but you won't get the integration in AS.
c) By instrumentation UI, do you mean starting the tests with the "play button", or showing the test results live as the tests are running? If you mean showing the live results, then I don't believe that's possible - as far as I know it depends on the connectedAndroidTest task. If you just want to execute tests you can create a custom Gradle task that just runs the "adb shell am instrument ..." command.