As I have set ItemAnimators. The ItemDecorator don't enter or exit along with the animation.
I simply ended up in having a view line in my item view layout file of each item. It solved my case. DividerItemDecoration felt to be too much of sorcery for a simple divider.