HDFS thrift server returns content of local FS, not HDFS

有些话、适合烂在心里 提交于 2019-12-01 12:16:29

问题


I am accessing HDFS using thrift.

  1. This is the expected(and right) content on HDFS.

    [hadoop@hdp-namenode-01 ~]$ hadoop fs -ls /
    Found 3 items
    drwxr-xr-x   - hadoop supergroup          0 2012-04-26 14:07 /home
    drwxr-xr-x   - hadoop supergroup          0 2012-04-26 14:21 /tmp
    drwxr-xr-x   - hadoop supergroup          0 2012-04-26 14:20 /user
    
  2. And then I start an HDFSThriftServer

    [hadoop@hdp-namenode-01 ~]$ jps
    17290 JobTracker
    16980 NameNode
    27289 Jps
    17190 SecondaryNameNode
    17511 RunJar
    25270 HadoopThriftServer
    
  3. Try to access content through thrift in PHP.

    $transport = new TSocket(HDFS_HOST, HDFS_PORT);
    $transport->setRecvTimeout(60000);
    $transport->setSendTimeout(60000);
    $protocol =new TBinaryProtocol($transport);
    $client = new ThriftHadoopFileSystemClient($protocol);
    logv("connect hdfs");
    $transport->open();
    logv("testing existent of `%s'", $remote_uri);
    $remote_path = new Pathname(array('pathname' => $remote_uri));
    $remote_file = null;
    try {
            $remote_file = $client->listStatus($remote_path);
    } catch(Exception $e) { }
    if (!$remote_file)
        loge("could not open `%s'", $remote_uri);

While $remote_uri is an absolute path. For $remote_uri === '/non/existent' or '/user', etc., listStatus always fail. But if I change it to '/tmp', I find that it lists the content of '/tmp' of the thrift server's local FS.

So the returned content is local FS on the thrift server instead of HDFS! What's wrong here?

来源:https://stackoverflow.com/questions/10331992/hdfs-thrift-server-returns-content-of-local-fs-not-hdfs

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!