Why use HDFS over Lustre and / or GPFS?

Apache Hadoop: Why use HDFS over Lustre and / or GPFS?

Curious to understand what some specific use cases would be for adopting HDFS as a file system over Lustre or GPFS. i.e. what kind of storage / computing needs are more suited to one file system over the other. thanks

Allen Wittenauer, I make yellow elephants cry.

In general (and not really geared towards the specific file systems):

• Many large scale file systems do not scale as high as HDFS.  (GPFS, for example, maxes out at 4PB, IIRC).

• The price/perf ratio may not be as good, especially if licensing is involved. 
• • This can some times be attributed to POSIX support.  This isn't 'cheap' and will impact either the cost or the performance or both.  Ignoring the Amazon Web Services and other Cloud Services cases, this is probably the #1 reason why one would not use HDFS when using Hadoop.
    
• Lack of MapReduce block locality support.  (Of course, I'm assuming one would implement the ability to read specific blocks in the Hadoop FileSystem API.  But that might be problematic for some systems as well...)
  
• Lack of replication (i.e., only one copy of the file on disk so ability to recover data becomes more important).
  

• Few 'real world' large scale references.  No one wants to be the first on the block to try Hadoop+other fs outside of a lab.  This is made worse by file system vendors having a very limited view of what Hadoop actually does and why.  (The older Lustre paper at http://wiki.lustre.org/images/1/... is a funread.)