In the recent 10 year when discussing java and/or garbage collection, the only performance penalty that I have not been able to defend is that garbage collection algorithms more
I'm not an expert, but any kind of generational garbage collection should help somewhat. The pages which are being aggressively swapped are likely to contain older objects rather than newer ones, so the gc will naturally touch them less frequently.
I'd also ask the counter-question: are there any unix paging algorithms which are garbage-collection aware? If a given page is being hauled into memory on a regular (if not frequent) basis, then maybe it isn't such a great candidate after all to be ditched in favour of more disk cache ;-)