large-files | 易学教程

Best way to process large XML in PHP [duplicate]

阅读更多关于 Best way to process large XML in PHP [duplicate]

This question already has an answer here: Parsing Huge XML Files in PHP 7 answers I have to parse large XML files in php, one of them is 6.5 MB and they could be even bigger. The SimpleXML extension as I've read, loads the entire file into an object, which may not be very efficient. In your experience, what would be the best way? For a large file, you'll want to use a SAX parser rather than a DOM parser. With a DOM parser it will read in the whole file and load it into an object tree in memory. With a SAX parser, it will read the file sequentially and call your user-defined callback functions

How to programmatically download a large file in C#

阅读更多关于 How to programmatically download a large file in C#

问题 I need to programmatically download a large file before processing it. What's the best way to do that? As the file is large, I want to specific time to wait so that I can forcefully exit. I know of WebClient.DownloadFile(). But there does not seem a way to specific an amount of time to wait so as to forcefully exit. try { WebClient client = new WebClient(); Uri uri = new Uri(inputFileUrl); client.DownloadFile(uri, outputFile); } catch (Exception ex) { throw; } Another way is to use a command

Upload 1GB files using chunking in PHP

阅读更多关于 Upload 1GB files using chunking in PHP

I have a web application that accepts file uploads of up to 4 MB. The server side script is PHP and web server is NGINX. Many users have requested to increase this limit drastically to allow upload of video etc. However there seems to be no easy solution for this problem with PHP. First, on the client side I am looking for something that would allow me to chunk files during transfer. SWFUpload does not seem to do that. I guess I can stream uploads using Java FX ( http://blogs.oracle.com/rakeshmenonp/entry/javafx_upload_file ) but I can not find any equivalent of request.getInputStream in PHP.

What is the fastest way to create a checksum for large files in C#

阅读更多关于 What is the fastest way to create a checksum for large files in C#

I have to sync large files across some machines. The files can be up to 6GB in size. The sync will be done manually every few weeks. I cant take the filename into consideration because they can change anytime. My plan is to create checksums on the destination PC and on the source PC and than copy all files with a checksum, which are not already in the destination, to the destination. My first attempt was something like this: using System.IO; using System.Security.Cryptography; private static string GetChecksum(string file) { using (FileStream stream = File.OpenRead(file)) { SHA256Managed sha =

How can I insert large files in MySQL db using PHP?

阅读更多关于 How can I insert large files in MySQL db using PHP?

问题 I want to upload a large file of maximum size 10MB to my MySQL database. Using .htaccess I changed PHP's own file upload limit to "10485760" = 10MB. I am able to upload files up to 10MB without any problem. But I can not insert the file in the database if it is more that 1 MB in size. I am using file_get_contents to read all file data and pass it to the insert query as a string to be inserted into a LONGBLOB field. But files bigger than 1 MB are not added to the database, although I can use

HTML5 - How to stream large .mp4 files?

阅读更多关于 HTML5 - How to stream large .mp4 files?

I'm trying to setup a very basic html5 page that loads a .mp4 video that is 20MB. It appears that the browser needs to download the entire thing rather than just playing the first part of the video and streaming in the rest. This post is the closest thing I've found while searching... I tried both Hand Brake and Data Go Round by neither appeared to make a difference: Any ideas on how to do this or if it's possible? Here is the code I'm using: <video controls="controls"> <source src="/video.mp4" type="video/mp4" /> Your browser does not support the video tag. </video> mark4o Ensure that the

Git lfs - “this exceeds GitHub's file size limit of 100.00 MB”

阅读更多关于 Git lfs - “this exceeds GitHub's file size limit of 100.00 MB”

问题 I have some csv files that are larger than github's file size limit of 100.00 MB. I have been trying to use the Git Large File Storage extension. https://git-lfs.github.com/ From LFS - "Large file versioning- Version large files—even those as large as a couple GB in size—with Git." I have applied the following on the folders of concern: git lfs track "*.csv" However, when I push: remote: error: File Time-Delay-ftn/Raw-count-data-minor-roads1.csv is 445.93 MB; this exceeds GitHub's file size

Searching for a string in a large text file - profiling various methods in python

阅读更多关于 Searching for a string in a large text file - profiling various methods in python

问题 This question has been asked many times. After spending some time reading the answers, I did some quick profiling to try out the various methods mentioned previously... I have a 600 MB file with 6 million lines of strings (Category paths from DMOZ project). The entry on each line is unique. I want to load the file once & keep searching for matches in the data The three methods that I tried below list the time taken to load the file, search time for a negative match & memory usage in the task

Python Random Access File

阅读更多关于 Python Random Access File

问题 Is there a Python file type for accessing random lines without traversing the whole file? I need to search within a large file, reading the whole thing into memory wouldn't be possible. Any types or methods would be appreciated. 回答1: This seems like just the sort of thing mmap was designed for. A mmap object creates a string-like interface to a file: >>> f = open("bonnie.txt", "wb") >>> f.write("My Bonnie lies over the ocean.") >>> f.close() >>> f.open("bonnie.txt", "r+b") >>> mm = mmap(f

How do I read a large CSV file with Scala Stream class?

阅读更多关于 How do I read a large CSV file with Scala Stream class?

问题 How do I read a large CSV file (> 1 Gb) with a Scala Stream? Do you have a code example? Or would you use a different way to read a large CSV file without loading it into memory first? 回答1: Just use Source.fromFile(...).getLines as you already stated. That returns an Iterator, which is already lazy (You'd use stream as a lazy collection where you wanted previously retrieved values to be memoized, so you can read them again) If you're getting memory problems, then the problem will lie in what