BlockReader is used by DFSInputStream to read a block.
There are four types of BlockReader currently in HDFS:
-
BlockReaderLocalLegacy: Enables local short circuited reads. If the DFS client is on the same machine as the datanode, then the client can read files directly from the local file system rather than going through the datanode for better performance. client must have the permission to read file directly.
-
BlockReaderLocal: Enables local short circuited reads. If the DFS client is on the same machine as the datanode, then the client can read files directly from the local file system rather than going through the datanode for better performance. While it has a difference with BlockReaderLocalLegacy: it doesn't need for client to have the permission to read the file directly, datanode will open the file and transfer the fd to client, client will use the fd to read from file, while fd must be sent via DomainSocket, this is a limit required by Unix.
-
RemoteBlockReaderFromDomain: use DomainSocket to transfer data
-
RemoteBlockReaderFromTcp: use TCP to transfer data