Multiple WAL in Apache HBase 1.3 and performance enhancements!!!

• 赞 Multiple WAL in Apache HBase 1.3 and performance enhancements!!!

  0
• 评论

  0
• 分享 分享Multiple WAL in Apache HBase 1.3 and performance enhancements!!!

  0

Mukesh Kumar, Machine Learning Professional and Big Data Expert

关注 Mukesh Kumar, Machine Learning Professional and Big Data Expert

Proven experience in multiple domains and BigData, a right fit for any data science project.

Apache HBase 1.3.0 was released mid-January 2017 and ships with support for date-based tiered compaction and improvements in multiple areas, like write-ahead log (WAL), and a new RPC scheduler, among others. The release includes almost 1,700 resolved issues in total.

Below are some bold points on enhancement made in HBase 1.3.0:-

1. The "date-based tiered compaction" support shipped in HBase 1.3.0 is beneficial for where data is infrequently deleted or updated and recent data is scanned more often than an older one.
2. Records time-to-live (TTL) can be easily enforced with this new compaction strategy.
3. Improved multiple WAL support in Apache HBase 1.3.0. Performance tests report 20% improvements in average latency when running on pure SATA disks, and 40% when WAL files are written to SATA-SSD disks.
4. The new RPC scheduler is based on CoDel algorithm and is used to prevent long standing call queues caused by discrepancy of request rate and available throughput, bounded by IO.
5. A throughput controller for disk flushers to avoid huge IO spikes.
6. A better performance of Apache Phoenix, OpenTSBD, and other software projects that rely on HBase engine for data persistence and fast lookup.
7. Add normalization support to hbase shell 'tools' command group for region normalizer.
8. Add configuration to skip validating HFile format when bulk loading:- A new config, hbase.loadincremental.validate.hfile , is introduced - default to true When set to false, checking hfile format is skipped during bulkloading.
9. Bulk Loaded HFile Replication enhances the HBase replication to support replication of bulk loaded data.
10. Introduces collection and reporting of various client-perceived metrics. Metrics are exposed via JMX.
11. Added 2 new pages to the region server Web UI to ease debugging and provide greater insight into the physical data layout.
12. The performance of region splitting has been improved by using a thread pool to split the store files concurrently. Prior to this change, the store files were always split sequentially in a single thread, so a region with multiple store files ended up taking several seconds. 
13. Web UI Framable Page adds protection from clickjacking using X-Frame-Options header. This will prevent use of HBase UI in frames.
14. Now flush all system tables in every 5 minute period using Memstore periodic flusher.