A singleton class will have an instance which generally is one and only one per classloader. So it can have regular methods(non static) ones and they can be invoked on that particular instance.
While a Class with only static methods, there is really no need in creating an instance(for this reason most of the people/frameworks make these kind of Util classes abstract). You will just invoke the methods on class directly.
