buffering

Can you upload to S3 using a stream rather than a local file?

…衆ロ難τιáo~ 提交于 2019-11-28 17:51:47
I need to create a CSV and upload it to an S3 bucket. Since I'm creating the file on the fly, it would be better if I could write it directly to S3 bucket as it is being created rather than writing the whole file locally, and then uploading the file at the end. Is there a way to do this? My project is in Python and I'm fairly new to the language. Here is what I tried so far: import csv import csv import io import boto from boto.s3.key import Key conn = boto.connect_s3() bucket = conn.get_bucket('dev-vs') k = Key(bucket) k.key = 'foo/foobar' fieldnames = ['first_name', 'last_name'] writer = csv

Why fprintf doesn't write directly into the file unless fflush() is used?

半腔热情 提交于 2019-11-28 11:40:32
问题 I have written a daemon that writes a value in a file. What I have observed is that when I keep writing on a file, there is nothing visible in the file. in other hand, If I use fflush() method then the characters are visible in the file. Why fflush() makes a difference? 回答1: Because it's buffered . That means all writes are stored in a buffer in memory until the buffer is flushed. For printf and friends it's when it has either a newline, or you explicitly call fflush , or of course if the

Why a script that uses threads prints extra lines occasionally?

倖福魔咒の 提交于 2019-11-28 10:15:44
If print s is replaced by print >>sys.stderr, s then the effect vanishes. import random, sys, time import threading lock = threading.Lock() def echo(s): time.sleep(1e-3*random.random()) # instead of threading.Timer() with lock: print s for c in 'abc': threading.Thread(target=echo, args=(c,)).start() Example # Run until empty line is found: $ while ! python example.py 2>&1|tee out|grep '^$';do echo -n .;done;cat out Output .................... b c a The output should not contain empty lines, but it does. I understand that print is not thread-safe, but I would've thought the lock should help.

Haskell: read input character from console immediately, not after newline

六月ゝ 毕业季﹏ 提交于 2019-11-28 08:57:30
I've tried this: main = do hSetBuffering stdin NoBuffering c <- getChar but it waits until the enter is pressed, which is not what I want. I want to read the character immediately after user presses it. I am using ghc v6.12.1 on Windows 7. EDIT: workaround for me was moving from GHC to WinHugs, which supports this correctly. Artelius Might be a bug: http://hackage.haskell.org/trac/ghc/ticket/2189 The following program repeats inputted characters until the escape key is pressed. import IO import Monad import Char main :: IO () main = do hSetBuffering stdin NoBuffering inputLoop inputLoop :: IO

Why isn't my IO executed in order?

心已入冬 提交于 2019-11-28 08:01:04
问题 I got a problem with IO not executing in order, even inside a do construct. In the following code I am just keeping track of what cards are left, where the card is a tuple of chars (one for suit and one for value) then the user is continously asked for which cards have been played. I want the putStr to be executed between each input, and not at the very end like it is now. module Main where main = doLoop cards doLoop xs = do putStr $ show xs s <- getChar n <- getChar doLoop $ remove (s,n) xs

Python sockets buffering

雨燕双飞 提交于 2019-11-28 07:30:31
Let's say I want to read a line from a socket, using the standard socket module: def read_line(s): ret = '' while True: c = s.recv(1) if c == '\n' or c == '': break else: ret += c return ret What exactly happens in s.recv(1) ? Will it issue a system call each time? I guess I should add some buffering, anyway: For best match with hardware and network realities, the value of bufsize should be a relatively small power of 2, for example, 4096. http://docs.python.org/library/socket.html#socket.socket.recv But it doesn't seem easy to write efficient and thread-safe buffering. What if I use file

Python in raw mode stdin print adds spaces

偶尔善良 提交于 2019-11-28 00:14:56
问题 I needed to switch the standard input to non-buffered mode in Python, so that I can read single characters off it. I managed to get it working, but now the standard output is broken: somehow it seems like after the newline character, some space characters are emitted, zero on the first line, 3 on the second, 6 on the third, etc, like this: ASD ASD ASD Operating system is Ubuntu Linux 12.04, 64-bit edition, Python version is 3.2.3. How can I rid myself from this behavior? Below is the code I

Statement before fork() printing twice [duplicate]

a 夏天 提交于 2019-11-27 18:32:17
问题 This question already has an answer here: printf anomaly after “fork()” 3 answers I was experimenting with fork() and re-direction to check whether the re-directions done in the parent apply to the child too. I wrote the following simple program #include<stdio.h> #include<unistd.h> #include<stdlib.h> int main () { freopen( "error.txt", "w+t", stdout ); // From now on, stdout = error.txt printf (" ERROR! WHY DONT U UNDERSTAND?\n"); if ( fork() == 0 ) { printf(" I AM CHILD\n"); exit(0); } else-

Turn off buffering

冷暖自知 提交于 2019-11-27 17:47:34
问题 Where is the buffer in this following ... and how do I turn it off? I am writing out to stdout in a python program like so: for line in sys.stdin: print line There is some buffering going on here: tail -f data.txt | grep -e APL | python -u Interpret.py I tried the following to shake off possible buffering ... with no luck: as above using the -u flag with python invocation calling sys.stdout.flush() after each sys.stdout.write() call ... all of these create a buffered stream with python

Why doesn't print output anything on each iteration of a loop when I use sleep?

我与影子孤独终老i 提交于 2019-11-27 14:41:34
Today in my college a teacher asked me a question. He wrote this code on the paper and said "What will be the output of this code?" use warnings; for (1 .. 20) { print "."; } I found it easy and said that it will loop 20 times and at each iteration it will print a dot (.) and hence total 20 dots will be the output. He said you are right and then he made some changes in the code. The code was: use warnings; for (1 .. 20) { print "."; sleep 1; } He said the what will be the output now? I didn't know about the sleep function, I guessed that at each iteration it will print the dot (.) and then it