I have a java application which when in idle state before any complex executions, uses 23 MB in Heap and the java.exe process size in TaskManager was around 194 MB. After so
OS footprint of java process are composed from
-Xmx)Some garbage collection algorithms are returning free memory back to OS, other do not.
In HotSpot JVM, serial old space collector (usually enabled by default) are returning memory back to OS (so you can see shrinking of process). Though, other collector such as -XX:+UseParallelOldGC or -XX:+UseConcMarkSweepGC will never return unused heap memory to OS.
HotSpot JVM has options to manage / limit all memory areas mentioned above, you can find comprehensive list of JVM options related to memory sizing and GC tuning in my blog.