Python shared string memory for multiprocessing

Question

I'm trying to use a shared string variable between my Python processes, but it seems that I'm doing something wrong since I'm getting coredumps and invalid memory values.

I use multiprocessing.Value to create a ctypes.c_char_p value and use the value attribute to access it. In my understanding of the Python docs the value attribute should be synchronized, as long it is an instance of Value (contrary of an instance of RawValue). Is that correct so far?

I've created a short example to demonstrate my use of Value and to show the inconsistency while executing:

from multiprocessing import Process, Value
from ctypes import c_char_p

def process(v):
    while True:
        val = v.value
        print val
        while val == v.value:
            pass

v = Value(c_char_p, None)
p = Process(target=process, args=(v,))
p.start()

for i in range(1,999):
    v.value = str(i)

p.terminate()

Answer 1

I think the problem may have been caused by using Value(c_char_p) to hold a string value. If you want a string, you should probably just use multiprocessing.Array(c_char).

Answer 2

From the Python-reference: https://docs.python.org/2/library/multiprocessing.html

your_string = Array('B', range(LENGHT))

You can take the identifier for the datatype from the table from the array module reference: https://docs.python.org/2/library/array.html

Answer 3

I ran into a similar problem when I attempted to set up multiple processes to access a shared I/O resource. It seems that Windows doesn't share a global variable space between processes, and items passed as arguments are squashed and passed by value.

This may not directly relate to your problem, but reading the discussion my help point you in the right direction:

Multiprocess with Serial Object as Parameter

Answer 4

This is very similar in function to your example, though subtly different. Notice that the child process terminates on its own when it gets the None sentinel. The polling loop could consume less CPU if it were to use a timeout.

from multiprocessing import Process, Pipe

def show_numbers(consumer):
    while True:
        if consumer.poll():
           val = consumer.recv()
           if val==None:
              break
           print(val)

(consumer,producer) = Pipe(False)
proc = Process(target=show_numbers, args=(consumer,))
proc.start()

for i in range(1,999):
    producer.send(str(i))

producer.send(None)

proc.join()