Image Segmentation overwhelming labeling background as another class

Question

I\'ve been involved in a project classifying fairly sparse details from a textured image (512x512). I\'m essentially doing image segmentation to characterise background, wat