I have a server with 12G of memory. A fragment of top is shown below:
PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                              
        use quick tip using top command in linux/unix
$ top
and then hit Shift+m (i.e. write a capital M).
From man top
SORTING of task window
  For compatibility, this top supports most of the former top sort keys.
  Since this is primarily a service to former top users, these commands do
  not appear on any help screen.
    command   sorted-field                  supported
      A         start time (non-display)      No
      M         %MEM                          Yes
      N         PID                           Yes
      P         %CPU                          Yes
      T         TIME+                         Yes
Or alternatively: hit Shift + f , then choose the display to order by memory usage by hitting key n then press Enter. You will see active process ordered by memory usage
List and Sort Processes by Memory Usage:
ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS
ps aux --sort '%mem'
from procps' ps (default on Ubuntu 12.04) generates output like:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
...
tomcat7   3658  0.1  3.3 1782792 124692 ?      Sl   10:12   0:25 /usr/lib/jvm/java-7-oracle/bin/java -Djava.util.logging.config.file=/var/lib/tomcat7/conf/logging.properties -D
root      1284  1.5  3.7 452692 142796 tty7    Ssl+ 10:11   3:19 /usr/bin/X -core :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
ciro      2286  0.3  3.8 1316000 143312 ?      Sl   10:11   0:49 compiz
ciro      5150  0.0  4.4 660620 168488 pts/0   Sl+  11:01   0:08 unicorn_rails worker[1] -p 3000 -E development -c config/unicorn.rb             
ciro      5147  0.0  4.5 660556 170920 pts/0   Sl+  11:01   0:08 unicorn_rails worker[0] -p 3000 -E development -c config/unicorn.rb             
ciro      5142  0.1  6.3 2581944 239408 pts/0  Sl+  11:01   0:17 sidekiq 2.17.8 gitlab [0 of 25 busy]                                                                          
ciro      2386  3.6 16.0 1752740 605372 ?      Sl   10:11   7:38 /usr/lib/firefox/firefox
So here Firefox is the top consumer with 16% of my memory.
You may also be interested in:
ps aux --sort '%cpu'
This very second in time
ps -U $(whoami) -eom pid,pmem,pcpu,comm | head -n4
Continuously updating
watch -n 1 'ps -U $(whoami) -eom pid,pmem,pcpu,comm | head -n4'
I also added a few goodies here you might appreciate (or you might ignore)
-n 1 watch and update every second
-U $(whoami) To show only your processes. $(some command) evaluates now
| head -n4 To only show the header and 3 processes at a time bc often you just need high usage line items
${1-4} says my first argument $1 I want to default to 4, unless I provide it
If you are using a mac you may need to install watch first brew install watch
Alternatively you might use a function
psm(){
    watch -n 1 "ps -eom pid,pmem,pcpu,comm | head -n ${1-4}"
    # EXAMPLES: 
    # psm 
    # psm 10
}
you can specify which column to sort by, with following steps:
steps:
* top
* shift + F
* select a column from the list
    e.g. n means sort by memory,
* press enter
* ok
                                                                        How to total up used memory by process name:
Sometimes even looking at the biggest single processes there is still a lot of used memory unaccounted for. To check if there are a lot of the same smaller processes using the memory you can use a command like the following which uses awk to sum up the total memory used by processes of the same name:
ps -e -orss=,args= |awk '{print $1 " " $2 }'| awk '{tot[$2]+=$1;count[$2]++} END {for (i in tot) {print tot[i],i,count[i]}}' | sort -n
e.g. output
9344 docker 1
9948 nginx: 4
22500 /usr/sbin/NetworkManager 1
24704 sleep 69
26436 /usr/sbin/sshd 15
34828 -bash 19
39268 sshd: 10
58384 /bin/su 28
59876 /bin/ksh 29
73408 /usr/bin/python 2
78176 /usr/bin/dockerd 1
134396 /bin/sh 84
5407132 bin/naughty_small_proc 1432
28061916 /usr/local/jdk/bin/java 7