Can someone suggest a way to create batches of a certain size in linq?
Ideally I want to be able to perform operations in chunks of some configurable amount.
All of the above perform terribly with large batches or low memory space. Had to write my own that will pipeline (notice no item accumulation anywhere):
public static class BatchLinq {
public static IEnumerable> Batch(this IEnumerable source, int size) {
if (size <= 0)
throw new ArgumentOutOfRangeException("size", "Must be greater than zero.");
using (IEnumerator enumerator = source.GetEnumerator())
while (enumerator.MoveNext())
yield return TakeIEnumerator(enumerator, size);
}
private static IEnumerable TakeIEnumerator(IEnumerator source, int size) {
int i = 0;
do
yield return source.Current;
while (++i < size && source.MoveNext());
}
}
Edit: Known issue with this approach is that each batch must be enumerated and enumerated fully before moving to the next batch. For example this doesn't work:
//Select first item of every 100 items
Batch(list, 100).Select(b => b.First())