filestream

Asynchronous random file access with FileStream Read vs ReadAsync in a loop / parallel (ReadAsync is slow)

白昼怎懂夜的黑 提交于 2020-08-25 21:26:41
问题 As we were switching from synchronous to asynchronous code we found that reading small chunks of data across a huge file (in a loop or in parallel) was much slower than using synchronous code. In our example we need to retrieve 8000 samples of 1000 bytes uniformly across a huge file. Below are the results we are getting. Synchronous code is at least 8 times faster. Any suggestions on how to speed up the async code? Full solution is here: https://github.com/virzak/DotNetPerformance Method |

node.js modify file data stream?

蓝咒 提交于 2020-06-11 20:07:38
问题 I need to copy one large data file to another destination with some modifications. fs.readFile and fs.writeFile are very slow. I need to read line by line, modify and write to new file. I found something like this: fs.stat(sourceFile, function(err, stat){ var filesize = stat.size; var readStream = fs.createReadStream(sourceFile); // HERE I want do some modifications with bytes readStream.pipe(fs.createWriteStream(destFile)); }) But how to make modifications ? I tried to get data with data

How to get the mode of a file descriptor?

送分小仙女□ 提交于 2020-06-07 06:50:47
问题 I mean to use fdopen FILE *fdopen(int fd, const char *mode); In man pages, it is stated that "The mode of the stream (one of the values "r", "r+", "w", "w+", "a", "a+") must be compatible with the mode of the file descriptor." So I have to first know the mode of fd (which I guess is an int ) to choose an appropriate const char *mode for the stream. I understand I should use fcntl int fcntl(int fd, int cmd); to "manipulate file descriptor" (in the following, I quote from this official source).

Is there a workaround to enable FILESTREAM in the code first approach of entity framework

别来无恙 提交于 2020-04-30 07:45:28
问题 After much of googling I have come to three conclusion that until MVC 4 ( entity framework), that FILESTREAM (of Sql server) is not supported directly. Is there a work around for the same? 回答1: Unfortunatelly, there is no equivalent of FILESTREAM in EF 6, that could be used with Code First approach. There is a workaround I found today, but haven't tried myself yet, though: http://ignoringthevoices.blogspot.co.uk/2014/01/working-with-entity-framework-code.html Basicaly, you need to use

C# FileStream read set encoding

人盡茶涼 提交于 2020-03-05 09:35:10
问题 There might be something obvious I'm missing here, but I can't seem to set the encoding on my FileStream read. Here's the code: FileStream fs = new FileStream(file, FileMode.Open, FileAccess.Read); using (fs) { byte[] buffer = new byte[chunk]; fs.Seek(chunk, SeekOrigin.Begin); int bytesRead = fs.Read(buffer, 0, chunk); while (bytesRead > 0) { ProcessChunk(buffer, bytesRead, database, id); bytesRead = fs.Read(buffer, 0, chunk); } } fs.Close(); Where ProcessChunk saves the read values to

How to read large files (a single continuous string) in Java?

荒凉一梦 提交于 2020-03-04 05:05:49
问题 I am trying to read a very large file (~2GB). Content is a continuous string with sentences (I would like to split them based on a '.'). No matter how I try, I end up with an Outofmemoryerror. BufferedReader in = new BufferedReader(new FileReader("a.txt")); String read = null; int i = 0; while((read = in.readLine())!=null) { String[] splitted = read.split("\\."); for (String part: splitted) { i+=1; users.add(new User(i,part)); repository.saveAll(users); } } also, inputStream = new