I\'ve recently discovered that Imagick can support color profiles and thus produce images of better quality compared to GD (see this question / answer for more details), so
There are a number of reasons why your PNG images may be increasing in size, the most obvious one that you will run into is GM/IM's inability to convey transparency as a tRNS chunk (basically boolean transparency for PNG images). Unfortunately the maintainers of GraphicsMagick and ImageMagick have not implemented this feature yet. I exchanged emails with them so I know this for sure.
I know you don't want to use external tools but trust me you do. Image/GraphicsMagick are really bad at compressing PNG images. The solution I am using is, use GraphicsMagick to manipulate the image and also check if the image contains transparent pixels, if it does contain transparent pixels then run OptiPNG on the image. OptiPNG will see that transparency can be conveyed as a tRNS chunk and act accordingly. Actually you should run OptiPNG on all PNG images after using Image/GraphicsMagick because I have found that you can achieve much greater compression. You can also save space by turning dithering off and by using the YUV color space.
As for GM reducing the size of images better than IM, you should know that GM by default uses an 8 bit color space when color reducing images while ImageMagick by default uses 16 bits. This is why GM is so much faster than IM when color reducing images to a value over 255 colors. Maybe you should check the number of colors in each image after compression to confirm.