I have recently started multi-threaded programming with Java in case of Linux threads , i know that the kernel schedules them(as they are the unit entities that are schedule
The jvm is just a normal process which starts with one thread and can spawn as much threads it likes afterwards. Scheduling is done on two levels - between processes and between threads inside processes. All this is done by the OS (via libs) - the jvm just hooks in. Google posix threads for more details - thats whats exposed (API) to the jvm.
This goes a bit into details: http://www.ibm.com/developerworks/java/library/j-rtj3/