It may be better to use Apache Hadoop and streaming because Apache Hadoop is actively being developed and maintained by big giants in the Industry like Yahoo and Facebook. So it can do what you expect it to do.
If you need a solution in .NET please check Myspace implementation @ MySpace Qizmt - MySpace’s Open Source Mapreduce Framework
