checksum

Very slow to generate MD5 for large file using Java

纵饮孤独 提交于 2019-11-28 21:41:27
问题 I am using Java to generate the MD5 hash for some files. I need to generate one MD5 for several files with a total size of about 1 gigabyte. Here's my code: private String generateMD5(SequenceInputStream inputStream){ if(inputStream==null){ return null; } MessageDigest md; try { int read =0; byte[] buf = new byte[2048]; md = MessageDigest.getInstance("MD5"); while((read = inputStream.read(buf))>0){ md.update(buf,0,read); } byte[] hashValue = md.digest(); return new String(hashValue); } catch

Amazon S3 & Checksum

喜夏-厌秋 提交于 2019-11-28 17:59:25
I try to verify the integrity of a file that was uploaded to a bucket but I don't find any information of this. In the file's headers, there is a "E-tag" but I think its not a md5 checksum. So, how can I check if the file that I uploaded on Amazon S3 is the same that I have on my computer ? Thanks. :) svetianov If you are using the REST API to upload an object (up to 5GB) in a single operation, then you can add the Content-MD5 header in your PUT request. According the the S3 documentation for PUT, the Content-MD5 header is: The base64 encoded 128-bit MD5 digest of the message (without the

Ideas to create a small (<10 digits), not (very) secure “hash”

这一生的挚爱 提交于 2019-11-28 17:06:13
I'm working on an online event ticketing system, where users will be able to self print his tickets and show up at the event where it will be scanned (barcode) and ideally the person will get in. My problem is how to create a "ticket code" that fulfills the following requirements: each "ticket code" need to be sufficiently different from each other (ie not sequentially numbered) ideally the ticket will be checked against a central DB to prevent reuse, but it NEEDS to be able to work off line too, in which case the system has to check for a "valid" ticket code and that it has not been used in

File containing its own checksum

橙三吉。 提交于 2019-11-28 16:59:53
问题 Is it possible to create a file that will contain its own checksum (MD5, SHA1, whatever)? And to upset jokers I mean checksum in plain, not function calculating it. 回答1: I created a piece of code in C, then ran bruteforce for less than 2 minutes and got this wonder: The CRC32 of this string is 4A1C449B Note the must be no characters (end of line, etc) after the sentence. You can check it here: http://www.crc-online.com.ar/index.php?d=The+CRC32+of+this+string+is+4A1C449B&en=Calcular+CRC32 This

Windows equivalent of linux cksum command

旧时模样 提交于 2019-11-28 16:58:40
I am looking for a way to compute crc checksum cross platform. cksum works on Linux, AIX, HP-UX Itanium, Solaris, is there a equivalent command of linux cksum in windows too? %cksum run.sh 1491301976 652 run.sh Note : no third party tool In Windows (command prompt) you can use CertUtil, here is the syntax: CertUtil [Options] -hashfile InFile [HashAlgorithm] for syntax explanation type in cmd: CertUtil -hashfile -? example: CertUtil -hashfile C:\myFile.txt MD5 default is SHA1 it supports: MD2, MD4, MD5, SHA1, SHA256, SHA384, SHA512. Unfortunately no CRC32 as Unix shell does. Here is a link if

What checksum algorithm should I use?

余生长醉 提交于 2019-11-28 16:36:51
问题 I'm building a system which needs to be able to find if blobs of bytes have been updated . Rather than storing the whole blob (they can be up to 5MBs), I'm thinking I should compute a checksum of it, store this and compute the same checksum a little bit later, to see whether the blog has been updated. The goal is to minimize the following (in that order) : size of the checksum time to compute likeliness of collisions (2 identical checksums happening even if the content has been modified). It

How to check if Paramiko successfully uploaded a file to an SFTP server?

我的梦境 提交于 2019-11-28 13:04:44
I use Paramiko to put a file to an SFTP server: import paramiko transport = paramiko.Transport((host, port)) transport.connect(username=username, password=password) sftp = paramiko.SFTPClient.from_transport(transport) sftp.put(local_path, remote_path) Now, I would like to check if it worked. The idea is that I compare the checksum of the local file and the remote one (that is located on the SFTP server). Does Paramiko functionality allows to do that? If it is the case, how exactly it works? With the SFTP, running over an encrypted SSH session, there's no chance the file contents could get

CRC8-Check in PHP

萝らか妹 提交于 2019-11-28 12:59:10
How can I generate a CRC-8 checksum in PHP? function crcnifull ($dato, $byte) { static $PolyFull=0x8c; for ($i=0; $i<8; $i++) { $x=$byte&1; $byte>>=1; if ($dato&1) $byte|=0x80; if ($x) $byte^=$PolyFull; $dato>>=1; } return $byte; } function crc8 (array $ar,$n=false) { if ($n===false) $n=count($ar); $crcbyte=0; for ($i=0; $i<$n; $i++) $crcbyte=crcnifull($ar[$i], $crcbyte); return $crcbyte; } To use this function for a binary string you have to convert the binary string to an array first. That can be achieved like this: function sbin2ar($sbin) { $ar=array(); $ll=strlen($sbin); for ($i=0; $i<$ll;

Oracle get checksum value for a data chunk defined by a select clause

痞子三分冷 提交于 2019-11-28 12:50:56
Is there any method in SQL (Oracle) using which I can get something like: select checksum(select * from table) from table; You can use DBMS_SQLHASH.GETHASH for this. The query results must be ordered and must not contain any LOBs, or the results won't be deterministic. select dbms_sqlhash.gethash(q'[select * from some_table order by 1,2]', digest_type => 1) from dual; Where digest_type 1 = HASH_MD4, 2 = HASH_MD5, 3 = HASH_SH1. That package is not granted to anyone by default. To use it, you'll need someone to logon as SYS and run this: SQL> grant execute on dbms_sqlhash to <your_user>; The

C Programming TCP Checksum

試著忘記壹切 提交于 2019-11-28 11:25:20
I have been having trouble doing the checksum for TCP for several days now. I have looked at many sources on the Internet but none of the examples that I have seen show you how to do the TCP checksum. I have also looked at the RFC document and still I am having trouble: Below is the code I am using to generate the checksum: unsigned short checksum(unsigned short * buffer, int bytes) { unsigned long sum = 0; unsigned short answer = 0; int i = bytes; while(i>0) { sum+=*buffer; buffer+=1; i-=2; } sum = (sum >> 16) + (sum & htonl(0x0000ffff)); sum += (sum >> 16); return ~sum; } This function works