combine python list elements where value is 1 plus an offset (masking)

问题

The goal is to find a generic method to solve the following task:

I have two python lists of the same length filled with zeros and ones:

detection = [0,0,1,0]     # only examples, they can be of any length
ground_truth = [0,1,0,0]  # and the ones can be at any indizes

and a integer number

offset = 1                # this number is also variable

The goal is to combine #offset elements in detection around elements equal to 1 and then combine the same index elements of ground_truth logical or, resulting the new lists:

detection = [0,1]
ground_truth = [0,1]

graphical explanation:

Background Info: The detection / ground truth values belong to a binary classification of a time series and The idea is to have a flexible evaluation that results in a TP if the detection fits the ground_truth is within a certain range of time steps (=offset).

Additional Example:

offset = 1
detection = [1,0,0,0,1,1,0]
ground_truth = [0,0,0,1,0,0,0]

would result to:

detection = [1,0,1]
ground_truth = [0,0,1]

回答1:

My first idea is to use slice [i-offset:i+offset+1]

If lists have different lengths then you can get shorter length

shorter = min(len(detection), len(ground_truth))

To works with lists separatelly you have to first find indexes.

I use [offset:shorter-offset] because I assumed that you don't want to check if there is not enought elements on left or right (if there are less elements then offset).

indexes = [i for i, val in enumerate(detection[offset:shorter-offset], offset) if val == 1]

And now you can use indexes

for i in indexes:
    #item = detection[i-offset:i] + detection[i+1:i+1+offset]
    # or

    item = detection[i-offset:i+offset+1]
    item.pop(offset) # remove value in the middle

    print('   detection item:', item)

I don't know what you try to do with or logic - so I skip it.

Code - with offset=2

detection    = [0,0,1,0,1,1,0,1,0,1,1]   # longer
ground_truth = [0,1,0,0,0,0,1,0]

#detection    = [0,0,1,0,0,0,1,0,0]       # shorter
#ground_truth = [0,0,1,0,1,1,0,1,0,1,1] 

print('   detection:', detection)
print('ground_truth:', ground_truth)

offset = 2
shorter = min(len(detection), len(ground_truth))

indexes = [i for i, val in enumerate(detection[offset:shorter-offset], offset) if val == 1]
print('indexes:', indexes)

for i in indexes:
    #item = detection[i-offset:i] + detection[i+1:i+1+offset]
    # or

    item = detection[i-offset:i+offset+1]
    item.pop(offset) # remove value in the middle

    print('   detection item:', item)

for i in indexes:
    #item = ground_truth[i-offset:i] + ground_truth[i+1:i+1+offset]
    # or

    item = ground_truth[i-offset:i+offset+1]
    item.pop(offset) # remove value in the middle

    print('ground_truth item:', item)

Result:

   detection: [0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1]
ground_truth: [0, 1, 0, 0, 0, 0, 1, 0]
indexes: [2, 4, 5]
   detection item: [0, 0, 0, 1]
   detection item: [1, 0, 1, 0]
   detection item: [0, 1, 0, 1]
ground_truth item: [0, 1, 0, 0]
ground_truth item: [0, 0, 0, 1]
ground_truth item: [0, 0, 1, 0]

Second idea is to use shift() to move value from previous/next row to the same row but to new column. But with new information I think it creates too many new columns so I removed it.

I was wondering if it could be done with rolling(window=3) but I couldn't create solution.

Doc: shift, apply, rolling

回答2:

I found the ultimate solution. sub questions that solved it:

variable expansion of ones in python list
bitwise OR reduction of python lists
squash all consecutive ones in a python list

Code:

# Create Mask from Detection and Offset
w = offset*2 +1
mask = np.convolve(detection, np.ones(w), mode='same').clip(0,1).astype(int)

# Create Soft Detection
soft_detection = mask[~((np.diff(mask,prepend=False)==0) & mask==1)].tolist()

# Create Soft Ground Truth
idx = np.flatnonzero(np.r_[True,np.diff(mask)!=0])
soft_ground_truth = np.bitwise_or.reduceat(ground_truth, idx).tolist()

来源：https://stackoverflow.com/questions/62407188/combine-python-list-elements-where-value-is-1-plus-an-offset-masking

标签

python

list

reduce

masking