I would like to pose this question as long as I am trying currently to dig into the use and the purpose of delegates, although it is likely to have been asked in similar formula
Yes, delegates are in many ways like single-method interfaces. However:
The last point is the most important one - consider a LINQ expression of:
var query = collection.Where(x => x > 5)
.Select(x => x * x);
Now imagine if to express the logic of x > 5 and x * x you had to write a separate class for each expression, and implement an interface: the amount of cruft vs useful code would be ridiculous. Now of course the language could have been designed to allow conversions from lambda expressions into interface implementations via separate classes, but then you'd still lose the benefit of being able to simply write a separate method and create a delegate with that as the target. You'd also still lose the multi-cast abilities.
As a similar thought exercsise, consider looping statements such as while and for. Do we really need them when we've got goto? Nope. But life is much better with them. The same is true of delegates - and indeed properties, events, etc. They all make the development simpler.