Most efficient way to create a path in zookeeper where root elements of the path may or may not exist?

问题

Imagine a path "/root/child1/child2/child3"

Imagine in zookeeper that maybe a part of this exists, say "/root/child1"

There is no equivalent of "mkdir -p" in zookeeper; Also, ZooKeeper.multi() will fail if any one operation fails, so a "make path" couldn't really be baked into a multi call. Additionally, you could have some other client trying to make the same path...

This is what I have come up with for creating a path. I wonder if it is even worth checking to see if a part exists or not, to save the round trip of the exists() call.

//String[] pathParts new String[] { "root", "child1", "child2", "child3" };

public void savePath(String[] pathParts) {
    if (zooKeeper.exists(pathString, false) != null) return;
    StringBuilder path = new StringBuilder();
    for (String pathElement : pathParts) {
        path.append(UNIX_FILE_SEPARATOR).append(pathElement);
        String pathString = path.toString();
        try {
            //bother with the exists call or not?
            if (zooKeeper.exists(pathString, false) == null) {
                zooKeeper.create(pathString, null, ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
            }
        } catch (KeeperException e) {
            if (e.code() != KeeperException.Code.NODEEXISTS)
                throw e;
        }
    }
}

What would be the most efficient way to do this? Assuming that a) you don't know ahead of time how much of the path already exists and b) some other client might be trying to write the same path (and we want to avoid locking).

回答1:

An exist call can be made with 1 round trip from the server to the client.

A create call has the same round trip, but create is a write operation that entails a couple more round trips between the servers in the zk cluster, so a create is a little more expensive that an exist.

So the total time for your algorithm is,

Time for 1 read op * Probability node already exists + (Time for 1 write op) * (1 - Probability the node already exists).

So either of if(!exist()) create() vs create() could be faster. In the end it probably doesn't matter.

If you want to be really fast, you can use the async api so that you can create all the components of your path without waiting for the server to respond to requests 1 by 1.

final AtomicBoolean success = new AtomicBoolean(false);
final CountdownLatch latch = new CountdownLatch(1);
StringCallback cb = new StringCallback() {
    processResult(int rc, String path, Object ctx, String name) {
        if(name.equals(pathString ) {
            //wait for the last path
            success.set(rc == KeeperException.Code.NODEEXISTS ||
                        rc == KeeperException.Code.OK);
            latch.countDown();
        }
    }
};

StringBuilder path = new StringBuilder();
for (String pathElement : pathParts) {
    path.append(UNIX_FILE_SEPARATOR).append(pathElement);
    String pathString = path.toString();
    //send requests to create all parts of the path without waiting for the
    //results of previous calls to return
    zooKeeper.create(pathString, null, ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT, cb);
}
latch.await();
if(!success.get()) { 
    throw ...
}

回答2:

You can use Netflix's curator library which makes using zookeeper much simpler

client.create().withMode(CreateMode.PERSISTENT).forPath("/root/child1/child2/child3", new byte[0]).withACL(ZooDefs.Ids.OPEN_ACL_UNSAFE).creatingParentsIfNeeded();

来源：https://stackoverflow.com/questions/9898837/most-efficient-way-to-create-a-path-in-zookeeper-where-root-elements-of-the-path

标签

java

Hadoop

apache-zookeeper