tpl-dataflow

I have never worked with TPL before so I was wondering whether this can be done with it: My application creates a gif image animation file from a lot of frames. I start with a list of Bitmap which represents the frames of the gif file and need to do the following for each frame: paint a number of text/bitmaps onto the frame crop the frame resize the frame reduce the image to 256 colors Obviously this process can be done in parallel for all the frames in the list but for each frame the order of steps needs to be the same. After that, I need to write all the frames to the gif file. Therefore all

I'm processing images using TPL Dataflow. I receive a processing request, read an image from a stream, apply several transformations, then write the resulting image to another stream: Request -> Stream -> Image -> Image ... -> Stream For that I use the blocks: BufferBlock<Request> TransformBlock<Request,Stream> TransformBlock<Stream,Image> TransformBlock<Image,Image> TransformBlock<Image,Image> ... writerBlock = new ActionBlock<Image> The problem is the initial Request is what contains some data necessary to create the resulting Stream along with some additional info I need at that point. Do I

I need to do this kind of work: Get Page object from database For each page get all images and process them (IO bound, for example, upload to CDN) If all images proceeded successfully then mark Page as processed in database Since I need to control how much Pages I process in parallel I've decided to go with TPL Dataflows: ____________________________ | Data pipe | | BufferBlock<Page> | | BoundedCapacity = 1 | |____________________________| | ____________________________ | Process images | | TransformBlock<Page, Page> | | BoundedCapacity = 1 | | MaxDegreeOfParallelism = 8 | |___________________

After playing around with dataflow I encountered a new problem. I would like to limit the inputqueue of all blocks. My producingblock (ActionBlock) is creating 5000 elements really fast and posts them to an broadcastblock. So if i set the BoundedCapacity of the broadcastblock to 100 he throws a lot of data away. But I would prefer the producingblock to wait for new slots in the inputqueue of my bufferblock. Is there any way to get rid of this problem? That's exactly what BufferBlock is for. If you set its BoundedCapacity and it gets full, it will postpone receiving any messages until someone

Given the following setup in TPL dataflow. var directory = new DirectoryInfo(@"C:\dev\kortforsyningen_dsm\tiles"); var dirBroadcast=new BroadcastBlock<DirectoryInfo>(dir=>dir); var dirfinder = new TransformManyBlock<DirectoryInfo, DirectoryInfo>((dir) => { return directory.GetDirectories(); }); var tileFilder = new TransformManyBlock<DirectoryInfo, FileInfo>((dir) => { return directory.GetFiles(); }); dirBroadcast.LinkTo(dirfinder); dirBroadcast.LinkTo(tileFilder); dirfinder.LinkTo(dirBroadcast); var block = new XYZTileCombinerBlock<FileInfo>(3, (file) => { var coordinate = file.FullName.Split

I have a requirement to make a scalable process. The process has mainly I/O operations with some minor CPU operations (mainly deserializing strings). The process query the database for a list of urls, then fetches data from these urls, deserilize the downloaded data to objects, then persist some of the data into crm dynamics and also to another database. Afterwards I need to update the first database which urls were processed. Part of the requirement is to make the parallelism degree configurable. Initially I thought to implement it via a sequence of tasks with await and limit the parallelism