memory-mapped-files

I've made a simple test with a MemoryMappedFile as msdn says : 2 processes, 1 memory mapped file : the first process adds the string "1" the first process waits the second process adds the string "2" and terminates the first process now reads the whole memory mapped file process A: using (MemoryMappedFile mmf = MemoryMappedFile.CreateNew("testmap", 10000)) { bool mutexCreated; Mutex mutex = new Mutex(true, "testmapmutex", out mutexCreated); using (MemoryMappedViewStream stream = mmf.CreateViewStream()) { BinaryWriter writer = new BinaryWriter(stream, Encoding.UTF8); writer.Write("1"); } mutex

In some situations the MemoryMappedViewAccessor class just doesn't cut it for reading bytes efficiently; the best we get is the generic ReadArray<byte> which it the route for all structs and involves several unnecessary steps when you just need bytes. It's possible to use a MemoryMappedViewStream , but because it's based on a Stream you need to seek to the correct position first, and then the read operation itself has many more unnecessary steps. Is there a quick, high-performance way to read an array of bytes from a memory-mapped file in .NET, given that it should just be a particular area of

When I read from files in C++(11) I map them in to memory using: boost::interprocess::file_mapping* fm = new file_mapping(path, boost::interprocess::read_only); boost::interprocess::mapped_region* region = new mapped_region(*fm, boost::interprocess::read_only); char* bytes = static_cast<char*>(region->get_address()); Which is fine when I wish to read byte by byte extremely fast. However, I have created a csv file which I would like to map to memory, read each line and split each line on the comma. Is there a way I can do this with a few modifications of my above code? (I am mapping to memory

I saved a couple of numpy arrays with np.save(), and put together they're quite huge. Is it possible to load them all as memory-mapped files, and then concatenate and slice through all of them without ever loading anythin into memory? Using numpy.concatenate apparently load the arrays into memory. To avoid this you can easily create a thrid memmap array in a new file and read the values from the arrays you wish to concatenate. In a more efficient way, you can also append new arrays to an already existing file on disk. For any case you must choose the right order for the array (row-major or

I've used C# to solve the following requirement.. - create an app the can receive a lot of data fast - you must be able to analyse the received data while more are incoming. - use as little CPU and disk as possible My idea for an algorithm was.. SIZE = 10MB Create a mmf with the size of SIZE On data recived: if data can't fit mmf: increase mmf.size by SIZE write the data to mmf -> The size on the disc are increased in chuncks of 10MB when the previous "room/space" are used. How is the "increase mmf.size by SIZE" done in C#? I have found a lot of simple examples on creating mmfs and views but

I have an application that receives chunks of data over the network, and writes these to disk. Once all chunks have been received, they can be decoded/recombined into the single file they actually represent. I'm wondering if it's useful to use memory-mapped files or not - first for writing the single chunks to disk, second for the single file into which all of them are decoded. My own feeling is that it might be useful for the second case only, anyone got some ideas on this? Edit: It's a C# app, and I'm only planning an x64 version. (So running into the 'largest contigious free space' problem

I'm working on something that uses ByteBuffers built from memory-mapped files (via FileChannel.map() ) as well as in-memory direct ByteBuffers. I am trying to understand the concurrency and memory model constraints. I have read all of the relevant Javadoc (and source) for things like FileChannel, ByteBuffer, MappedByteBuffer, etc. It seems clear that a particular ByteBuffer (and relevant subclasses) has a bunch of fields and the state is not protected from a memory model point of view. So, you must synchronize when modifying state of a particular ByteBuffer if that buffer is used across