binaryfiles

Fastest way to read every 30th byte of large binary file?

由 左心房为你撑大大i 提交于 2019-11-30 11:24:35
问题 What is the fastest way to read every 30th byte of a large binary file (2-3 GB)? I've read there are performance problems with fseek because of I/O buffers, but I don't want to read 2-3 GB of data into memory before grabbing every 30th byte either. 回答1: Performance test. If you want to use it yourself, note that the integrity check (printing total) only works if "step" divides BUFSZ, and MEGS is small enough that you don't read off the end of the file. This is due to (a) laziness, (b) desire

using bash: write bit representation of integer to file

由 无人久伴 提交于 2019-11-30 08:34:46
I have a file with binary data and I need to replace a few bytes in a certain position. I've come up with the following to direct bash to the offset and show me that it found the place I want: dd bs=1 if=file iseek=24 conv=block cbs=2 | hexdump Now, to use "file" as the output: echo anInteger | dd bs=1 of=hextest.txt oseek=24 conv=block cbs=2 This seems to work just fine, I can review the changes made in a hex editor. Problem is, "anInteger" will be written as the ASCII representation of that integer (which makes sense) but I need to write the binary representation. I want to use bash for this

Are Golang binaries portable?

由 依然范特西╮ 提交于 2019-11-30 07:53:11
Suppose I'm a primarily Linux user, but I'm developing an application in Go that I want to be cross platform. I've searched around, but I can't seem to find information to absolve the following: If I go install a binary on my amd64 Ubuntu system, will it also work on anyone else's 64-bit Ubuntu/Debian system? How can I use go install to build an x86_64 binary that will also run out-of-the-box on 32-bit Debianlikes? If I must use Windows to make a binary which will run on Windows, how can I also ensure that even if my Windows system is 64-bit the executable will be built for x86_64? My

are “seekp” & “seekg” interchangeable?

由 此生再无相见时 提交于 2019-11-30 07:22:17
问题 Well I just noticed that by changing the position -in microsoft visual studio- through "seekp" I implicitelly also change the read-position, when handling files. I am wondering however if this is "portable" behaviour? Can I expect the position of reading & writing to be always the same? And consequently: will tellp & tellg always return the same value? 回答1: For file positions they are the same. In other words there is only one pointer maintained. From 27.9.1.1p3 : A joint file position is

Reading 32 bit signed ieee 754 floating points from a binary file with python?

由 Deadly 提交于 2019-11-30 06:47:05
I have a binary file which is simple a list of signed 32 bit ieee754 floating point numbers. They are not separated by anything, and simply appear one after another until EOF. How would I read from this file and interpret them correctly as floating point numbers? I tried using read(4) , but it automatically converts them to a string with ascii encoding. I also tried using bytearray but that only takes it in 1 byte at a time instead of 4 bytes at a time as I need. struct.unpack('f', file.read(4)) You can also unpack several at once, which will be faster: struct.unpack('f'*n, file.read(4*n))

Evaluating HDF5: What limitations/features does HDF5 provide for modelling data?

由 拈花ヽ惹草 提交于 2019-11-30 06:20:36
问题 We are in evaluating technologies that we'll use to store data that we gather during the analysis of C/C++ code. In the case of C++, the amount of data can be relatively large, ~20Mb per TU. After reading the following SO answer it made me consider that HDF5 might be a suitable technology for us to use. I was wondering if people here could help me answer a few initial questions that I have: Performance. The general usage for the data will be write once and read "several" times, similar to the

How to reverse engineer binary file formats for compatibility purposes

由 爱⌒轻易说出口 提交于 2019-11-30 04:40:07
问题 I am working of a file preparation software to enable translators work easily and efficiently on a wide range of file formats. As far as text-based formats (xml, php, resource files,...) are concerned, my small preparation utility works fine, but a major problem for most translators is to handle all kinds of proprietary binary formats (Framemaker, Publisher, Quark...). These files are rarely requested and need to be opened in expensive applications (few freelance can afford to buy $20,000

Git mark file as binary to avoid line separator conversion

由 余生长醉 提交于 2019-11-30 01:59:37
问题 I have a text file with sample data which I want to put under Git control (on Windows). How can I tell Git to treat this file as binary, so it does not convert line separators (like for the other/text files in the repository). 回答1: Write this in .gitattributes (in the file’s directory or higher up): myfile.bin -text See the manual for more explanation. 回答2: Marking files as binary Git usually guesses correctly whether a blob contains text or binary data by examining the beginning of the

When should pdf files be tracked in a Git repository and when not

由 亡梦爱人 提交于 2019-11-30 01:24:40
问题 I am developing a LateX package (http://www.openlilylib.org/lilyglyphs) which contains a number of small PDF files. Currently there are only a few dozens of them but as the package and its user base grows there will probably hundreds of them (but unlikely more than 1000). The PDFs are typically only a few KB in size, but I don't know whether to track them in the Git repository. The files are subject to change at any time, but probably not too often. Usually one is told not to track binary

Fastest way to read every 30th byte of large binary file?

由 旧时模样 提交于 2019-11-30 00:08:10
What is the fastest way to read every 30th byte of a large binary file (2-3 GB)? I've read there are performance problems with fseek because of I/O buffers, but I don't want to read 2-3 GB of data into memory before grabbing every 30th byte either. Steve Jessop Performance test. If you want to use it yourself, note that the integrity check (printing total) only works if "step" divides BUFSZ, and MEGS is small enough that you don't read off the end of the file. This is due to (a) laziness, (b) desire not to obscure the real code. rand1.data is a few GB copied from /dev/urandom using dd .
订阅 binaryfiles