Yes. Use a vector.
To insert, simply place at the end, and increment the size. To remove, pick an element at random, swap its contents with the end value, then pop off the end value (i.e., return the end value and decrement the vector's size).
Both operations are amortised O(1).