I made a simple image through Dockerfile from Fedora (initially 320 MB).
Added Nano (this tiny editor of 1MB size), and the size of the image has risen to 530 MB. I\
Docker Squash is a really nice solution to this. you can $packagemanager clean in the last step instead of in every line and then just run a docker squash to get rid of all of the layers.
$packagemanager clean
https://github.com/jwilder/docker-squash