I want to limit the size of records inside a group, and here is my trial, how to do it right?
mysql> select * from accounts limit 5 group by type;
It appears you want to limit the number of rows returned within each group of your overall result set... this is difficult to do in a way that scales well. One technique is to perform N joins on the same table with the conditions such that the only rows that match are the top/bottom N that you want.
this page may offer some additional insight into your solution... although returning the top 5 in each group is going to get ugly fast.