Spark sql Caching
The shark.cache table property no longer exists, and tables whose name end with _cached are no longer automatically cached. Instead, we provide CACHE TABLE and UNCACHE TABLE statements to let user control table caching explicitly:
CACHE TABLE logs_last_month;UNCACHE TABLE logs_last_month;
NOTE: CACHE TABLE tbl is lazy, similar to .cache on an RDD. This command only marks tbl to ensure that partitions are cached when calculated but doesn’t actually cache it until a query that touches tbl is executed. To force the table to be cached, you may simply count the table immediately after executing CACHE TABLE:
CACHE TABLE logs_last_month;SELECT COUNT(1) FROM logs_last_month;
Several caching related features are not supported yet:
· User defined partition level cache eviction policy
· RDD reloading
· In-memory cache write through policy
Spark sql cache语法:
CACHE TABLE your-table-name;
UNCACHE TABLE your-table-name;
CACHE TABLE tmp-table-name as select * from your-table-name where field=value;