it would be faster if you used a LinkedList instead of an ArrayList, as the ArrayList will need to resize when it nears capacity.
you will also want to appropriately estimate the capacity of the wrapping collection (HashMap or Multimap) you are creating to avoid repetitive rehashing.