Why is the following snippet for deleting a node in a linked list not thread safe?
edit: note every node has a lock of its own
// ... lock acquisitio         
        
I assume that you are talking about a singly linked list, since you never assign 'prev' in your node-deletion. Given a singly linked list of nodes, each protected by a lock, it might be depicted as follows:
Head ==> A ==> B ==> C ==> D ==> Tail
               ^     ^
               |     |
        Thread 1     Thread 2
Let's say that Thread 1 is going to delete node B. Coincidentally, Thread 2 is going to try to delete node C at the same time. The steps that you give might execute as follows:
Thread 1                Thread 2
----------------------  ----------------------
Lock B                  Lock C
A->next = C or D; <=??  B->next = D;    <== B could be dead already
B->next = NULL;         C->next = NULL;
B->deleted = 1;         C->deleted = 1;
Unlock B                Unlock C
In this case, the result is unpredictable. If Thread 2 executed slightly ahead of Thread 1, then everything should be fine. Thread 1's second line would perform "A->next = D" since Thread 2 would have already changed B->next to D. However, if Thread 1 executes slightly ahead of Thread 2, then A->next points to dead node C, dead node B was modified, and node D is lost.
So, you might try to lock the node you're going to delete, then lock 'prev' before modifying it. The steps might execute as follows:
Thread 1                Thread 2
----------------------  ----------------------
Lock B                  Lock C
Lock A                  waiting for B
A->next = C;            waiting for B
Unlock A                waiting for B
B->next = NULL;         waiting for B
B->deleted = 1;         waiting for B
Unlock B                Lock B         <= locking dead node
                        B->next = D;   <= assigning to dead node
                        Unlock B
                        C->next = NULL;
                        C->deleted = 1;
                        Unlock C
So, this still isn't thread-safe. A->next points to dead node C, dead node B was locked and used, and D is lost. All we've done is make sure that the error case above happens reliably.
The solution here would seem to require a lock on 'prev' before locking the node to be deleted.
Thread 1                Thread 2
----------------------  ----------------------
Lock A                  Lock B
waiting for B           Lock C
waiting for B           B->next = D;
Lock B                  Unlock B  
A->next = D;            C->next = NULL;
Unlock A                C->deleted = 1;
B->next = NULL;         Unlock C
B->deleted = 1;         
Unlock B                
A->next points to D, and both B and C are now deleted.