RHEL 7 Technology Preview - lvm cache like zfs l2arc, bcache, flashcache-CSDN博客

zfs l2arc不回写, 不存储脏数据. LVM cache会不会回写(会, 因为默认是writeback的, 显然是可以回写的, 所以LVM和flashcache类似, 性能应该很好)? LVM cache能否达到flashcache的性能?

同时lvm还支持多个ssd设备mirror.

LVM Cache

Red Hat Enterprise Linux 7 introduces LVM cache as a Technology Preview. This feature allows users to create logical volumes with a small fast device performing as a cache to larger slower devices. Please refer to the lvm(8) manual page for information on creating cache logical volumes.

Note that the following commands are not currently allowed on cache logical volumes:

pvmove: will skip over any cache logical volume;

lvresize, lvreduce, lvextend: cache logical volumes cannot be resized currently;

vgsplit: splitting a volume group is not allowed when cache logical volumes exist in it.

Fast Block Devices Caching Slower Block Devices

LVM provides the ability to have fast block devices act as a cache for slower block devices. This feature is introduced as a Technology Preview in Red Hat Enterprise Linux 7 and allows a PCIe SSD device to act as a cache for direct-attached storage (DAS) or storage area network (SAN) storage, which improves file system performance.

For more information, refer to the LVM Cache entry in Chapter 3, Storage and the lvm(8) manual page.

man lvm

LVM(8) System Manager's Manual LVM(8)

NAME

lvm - LVM2 tools

SYNOPSIS

lvm [command | file]

DESCRIPTION

lvm provides the command-line tools for LVM2. A separate manual page describes each command in detail.

If lvm is invoked with no arguments it presents a readline prompt (assuming it was compiled with readline support). LVM

commands may be entered interactively at this prompt with readline facilities including history and command name and

option completion. Refer to readline(3) for details.

If lvm is invoked with argv[0] set to the name of a specific LVM command (for example by using a hard or soft link) it

acts as that command.

On invocation, lvm requires that only the standard file descriptors stdin, stdout and stderr are available. If others

are found, they get closed and messages are issued warning about the leak. This warning can be suppressed by setting the

environment variable LVM_SUPPRESS_FD_WARNINGS.

Where commands take VG or LV names as arguments, the full path name is optional. An LV called "lvol0" in a VG called

"vg0" can be specified as "vg0/lvol0". Where a list of VGs is required but is left empty, a list of all VGs will be sub‐

stituted. Where a list of LVs is required but a VG is given, a list of all the LVs in that VG will be substituted. So

lvdisplay vg0 will display all the LVs in "vg0". Tags can also be used - see --addtag below.

One advantage of using the built-in shell is that configuration information gets cached internally between commands.

A file containing a simple script with one command per line can also be given on the command line. The script can also

be executed directly if the first line is #! followed by the absolute path of lvm.

BUILT-IN COMMANDS

The following commands are built into lvm without links normally being created in the filesystem for them.

dumpconfig — Display the configuration information after

loading lvm.conf(5) and any other configuration files.

devtypes — Display the recognised built-in block device types.

formats — Display recognised metadata formats.

help — Display the help text.

pvdata — Not implemented in LVM2.

segtypes — Display recognised Logical Volume segment types.

tags — Display any tags defined on this host.

version — Display version information.

COMMANDS

The following commands implement the core LVM functionality.

pvchange — Change attributes of a Physical Volume.

pvck — Check Physical Volume metadata.

pvcreate — Initialize a disk or partition for use by LVM.

pvdisplay — Display attributes of a Physical Volume.

pvmove — Move Physical Extents.

pvremove — Remove a Physical Volume.

pvresize — Resize a disk or partition in use by LVM2.

pvs — Report information about Physical Volumes.

pvscan — Scan all disks for Physical Volumes.

vgcfgbackup — Backup Volume Group descriptor area.

vgcfgrestore — Restore Volume Group descriptor area.

vgchange — Change attributes of a Volume Group.

vgck — Check Volume Group metadata.

vgconvert — Convert Volume Group metadata format.

vgcreate — Create a Volume Group.

vgdisplay — Display attributes of Volume Groups.

vgexport — Make volume Groups unknown to the system.

vgextend — Add Physical Volumes to a Volume Group.

vgimport — Make exported Volume Groups known to the system.

vgimportclone — Import and rename duplicated Volume Group (e.g. a hardware snapshot).

vgmerge — Merge two Volume Groups.

vgmknodes — Recreate Volume Group directory and Logical Volume special files

vgreduce — Reduce a Volume Group by removing one or more

Physical Volumes.

vgremove — Remove a Volume Group.

vgrename — Rename a Volume Group.

vgs — Report information about Volume Groups.

vgscan — Scan all disks for Volume Groups and rebuild caches.

vgsplit — Split a Volume Group into two, moving any logical

volumes from one Volume Group to another by moving entire Physical Volumes.

lvchange — Change attributes of a Logical Volume.

lvconvert — Convert a Logical Volume from linear to mirror or snapshot.

lvcreate — Create a Logical Volume in an existing Volume Group.

lvdisplay — Display attributes of a Logical Volume.

lvextend — Extend the size of a Logical Volume.

lvmchange — Change attributes of the Logical Volume Manager.

lvmdiskscan — Scan for all devices visible to LVM2.

lvmdump — Create lvm2 information dumps for diagnostic purposes.

lvreduce — Reduce the size of a Logical Volume.

lvremove — Remove a Logical Volume.

lvrename — Rename a Logical Volume.

lvresize — Resize a Logical Volume.

lvs — Report information about Logical Volumes.

lvscan — Scan (all disks) for Logical Volumes.

The following commands are not implemented in LVM2 but might be in the future: lvmsadc, lvmsar, pvdata.

OPTIONS

The following options are available for many of the commands. They are implemented generically and documented here

rather than repeated on individual manual pages.

-h, -?, --help

Display the help text.

--version

Display version information.

-v, --verbose

Set verbose level. Repeat from 1 to 3 times to increase the detail of messages sent to stdout and stderr. Over‐

rides config file setting.

-d, --debug

Set debug level. Repeat from 1 to 6 times to increase the detail of messages sent to the log file and/or syslog

(if configured). Overrides config file setting.

-q, --quiet

Suppress output and log messages. Overrides -d and -v.

--yes Don't prompt for confirmation interactively but instead always assume the answer is 'yes'. Take great care if you

use this!

-t, --test

Run in test mode. Commands will not update metadata. This is implemented by disabling all metadata writing but

nevertheless returning success to the calling function. This may lead to unusual error messages in multi-stage

operations if a tool relies on reading back metadata it believes has changed but hasn't.

--driverloaded {y|n}

Whether or not the device-mapper kernel driver is loaded. If you set this to n, no attempt will be made to con‐

tact the driver.

-A, --autobackup {y|n}

Whether or not to metadata should be backed up automatically after a change. You are strongly advised not to dis‐

able this! See vgcfgbackup(8).

-P, --partial

When set, the tools will do their best to provide access to Volume Groups that are only partially available (one

or more Physical Volumes belonging to the Volume Group are missing from the system). Where part of a logical vol‐

ume is missing, /dev/ioerror will be substituted, and you could use dmsetup(8) to set this up to return I/O errors

when accessed, or create it as a large block device of nulls. Metadata may not be changed with this option. To

insert a replacement Physical Volume of the same or large size use pvcreate -u to set the uuid to match the origi‐

nal followed by vgcfgrestore(8).

-M, --metadatatype Type

Specifies which type of on-disk metadata to use, such as lvm1 or lvm2, which can be abbreviated to 1 or 2 respec‐

tively. The default (lvm2) can be changed by setting format in the global section of the config file.

--ignorelockingfailure

This lets you proceed with read-only metadata operations such as lvchange -ay and vgchange -ay even if the locking

module fails. One use for this is in a system init script if the lock directory is mounted read-only when the

script runs.

--ignoreskippedcluster

Use to avoid exiting with an non-zero status code if the command is run without clustered locking and some clus‐

tered Volume Groups have to be skipped over.

--addtag Tag

Add the tag Tag to a PV, VG or LV. Supply this argument multiple times to add more than one tag at once. A tag

is a word that can be used to group LVM2 objects of the same type together. Tags can be given on the command line

in place of PV, VG or LV arguments. Tags should be prefixed with @ to avoid ambiguity. Each tag is expanded by

replacing it with all objects possessing that tag which are of the type expected by its position on the command

line. PVs can only possess tags while they are part of a Volume Group: PV tags are discarded if the PV is removed

from the VG. As an example, you could tag some LVs as database and others as userdata and then activate the data‐

base ones with lvchange -ay @database. Objects can possess multiple tags simultaneously. Only the new LVM2 meta‐

data format supports tagging: objects using the LVM1 metadata format cannot be tagged because the on-disk format

does not support it. Characters allowed in tags are: A-Z a-z 0-9 _ + . - and as of version 2.02.78 the following

characters are also accepted: / = ! : # &

--deltag Tag

Delete the tag Tag from a PV, VG or LV, if it's present. Supply this argument multiple times to remove more than

one tag at once.

--alloc {anywhere|contiguous|cling|inherit|normal}

Selects the allocation policy when a command needs to allocate Physical Extents from the Volume Group. Each Vol‐

ume Group and Logical Volume has an allocation policy defined. The default for a Volume Group is normal which

applies common-sense rules such as not placing parallel stripes on the same Physical Volume. The default for a

Logical Volume is inherit which applies the same policy as for the Volume Group. These policies can be changed

using lvchange(8) and vgchange(8) or overridden on the command line of any command that performs allocation. The

contiguous policy requires that new Physical Extents be placed adjacent to existing Physical Extents. The cling

policy places new Physical Extents on the same Physical Volume as existing Physical Extents in the same stripe of

the Logical Volume. If there are sufficient free Physical Extents to satisfy an allocation request but normal

doesn't use them, anywhere will - even if that reduces performance by placing two stripes on the same Physical

Volume.

--profile ProfileName

Selects the configuration profile to use when processing an LVM command. In addition to that, when creating a

Volume Group or a Logical Volume, it causes the ProfileName to be stored in metadata for each Volume Group or Log‐

ical Volume. If the profile is stored in metadata, it is automatically applied next time the Volume Group or the

Logical Volume is processed and the use of --profile is not necessary when running LVM commands further. See also

lvm.conf(5) for more information about profile config and the way it fits with other LVM configuration methods.

--config ConfigurationString

Uses the ConfigurationString as direct string representation of the configuration to override the existing config‐

uration. The ConfigurationString is of exactly the same format as used in any LVM configuration file. See

lvm.conf(5) for more information about direct config override on command line and the way it fits with other LVM

configuration methods.

ENVIRONMENT VARIABLES

HOME Directory containing .lvm_history if the internal readline shell is invoked.

LVM_SYSTEM_DIR

Directory containing lvm.conf(5) and other LVM system files. Defaults to "/etc/lvm".

LVM_SUPPRESS_FD_WARNINGS

Suppress warnings about openned file descriptors, when lvm command is executed.

LVM_VG_NAME

The Volume Group name that is assumed for any reference to a Logical Volume that doesn't specify a path. Not set

by default.

LVM_LVMETAD_PIDFILE

Path for the lvmetad pid file.

LVM_LVMETAD_SOCKET

Path for the lvmetad socket file.

VALID NAMES

The following characters are valid for VG and LV names: a-z A-Z 0-9 + _ . -

VG and LV names cannot begin with a hyphen. There are also various reserved names that are used internally by lvm that

can not be used as LV or VG names. A VG cannot be called anything that exists in /dev/ at the time of creation, nor can

it be called '.' or '..'. A LV cannot be called '.' '..' 'snapshot' or 'pvmove'. The LV name may also not contain the

strings '_mlog', '_mimage', '_rimage', '_tdata', '_tmeta'.

ALLOCATION

When an operation needs to allocate Physical Extents for one or more Logical Volumes, the tools proceed as follows:

First of all, they generate the complete set of unallocated Physical Extents in the Volume Group. If any ranges of Phys‐

ical Extents are supplied at the end of the command line, only unallocated Physical Extents within those ranges on the

specified Physical Volumes are considered.

Then they try each allocation policy in turn, starting with the strictest policy (contiguous) and ending with the alloca‐

tion policy specified using --alloc or set as the default for the particular Logical Volume or Volume Group concerned.

For each policy, working from the lowest-numbered Logical Extent of the empty Logical Volume space that needs to be

filled, they allocate as much space as possible according to the restrictions imposed by the policy. If more space is

needed, they move on to the next policy.

The restrictions are as follows:

Contiguous requires that the physical location of any Logical Extent that is not the first Logical Extent of a Logical

Volume is adjacent to the physical location of the Logical Extent immediately preceding it.

Cling requires that the Physical Volume used for any Logical Extent to be added to an existing Logical Volume is already

in use by at least one Logical Extent earlier in that Logical Volume. If the configuration parameter alloca‐

tion/cling_tag_list is defined, then two Physical Volumes are considered to match if any of the listed tags is present on

both Physical Volumes. This allows groups of Physical Volumes with similar properties (such as their physical location)

to be tagged and treated as equivalent for allocation purposes.

When a Logical Volume is striped or mirrored, the above restrictions are applied independently to each stripe or mirror

image (leg) that needs space.

Normal will not choose a Physical Extent that shares the same Physical Volume as a Logical Extent already allocated to a

parallel Logical Volume (i.e. a different stripe or mirror image/leg) at the same offset within that parallel Logical

Volume.

When allocating a mirror log at the same time as Logical Volumes to hold the mirror data, Normal will first try to select

different Physical Volumes for the log and the data. If that's not possible and the allocation/mirror_logs_require_sepa‐

rate_pvs configuration parameter is set to 0, it will then allow the log to share Physical Volume(s) with part of the

data.

When allocating thin pool metadata, similar considerations to those of a mirror log in the last paragraph apply based on

the value of the allocation/thin_pool_metadata_require_separate_pvs configuration parameter.

If you rely upon any layout behaviour beyond that documented here, be aware that it might change in future versions of

the code.

For example, if you supply on the command line two empty Physical Volumes that have an identical number of free Physical

Extents available for allocation, the current code considers using each of them in the order they are listed, but there

is no guarantee that future releases will maintain that property. If it is important to obtain a specific layout for a

particular Logical Volume, then you should build it up through a sequence of lvcreate(8) and lvconvert(8) steps such that

the restrictions described above applied to each step leave the tools no discretion over the layout.

To view the way the allocation process currently works in any specific case, read the debug logging output, for example

by adding -vvvv to a command.

LOGICAL VOLUME TYPES

Some logical volume types are simple to create and can be done with a single lvcreate(8) command. The linear and striped

logical volume types are an example of this. Other logical volume types may require more than one command to create.

The cache and thin provisioning types are examples of this.

Cache

The cache logical volume type uses a small and fast LV to improve the performance of a large and slow LV. It does this

by storing the frequently used blocks on the faster LV. LVM refers to the small fast LV as a cache pool LV. The large

slow LV is called the origin LV. Due to requirements from dm-cache (the kernel driver), LVM further splits the cache

pool LV into two devices - the cache data LV and cache metadata LV. The cache data LV is where copies of data blocks are

kept from the origin LV to increase speed. The cache metadata LV holds the accounting information that specifies where

data blocks are stored (e.g. on the origin LV or on the cache data LV). Users should be familiar with these LVs if they

wish to create the best and most robust cached logical volumes.

Cache Terms

origin LV OriginLV large slow LV

cache data LV CacheDataLV small fast LV for cache pool data

cache metadata LV CacheMetaLV small fast LV for cache pool metadata

cache pool LV CachePoolLV CacheDataLV + CacheMetaLV

cache LV CacheLV OriginLV + CachePoolLV

Cache Steps

The steps to create a logical volume of cache type are as follows:

0. Create an LV or identify an existing LV to be the origin LV.

1. Create the cache data LV. The size of this LV is the size of the cache and will be reported as the size of the

cache pool LV.

2. Create the cache metadata LV. The size of this LV should be 1000 times smaller than the cache data LV with a min‐

imum size of 8MiB.

3. Create the cache pool LV by combining the cache data LV (from step 1) and cache metadata LV (from step 2). When

performing this step, behavioral characteristics of the cache pool LV can be set. The name of the cache pool LV

takes the name of the cache data LV and the cache data LV and cache metadata LV are renamed to CachePoolLV_cdata

and CachePoolLV_cmeta.

4. Create a cache LV by linking the cache pool LV to the origin LV. The user accessible cache LV takes the name of

the origin LV, while the origin LV becomes a hidden LV with the name OriginLV_corig. Users can perform this step

while the origin LV is in use.

The steps above represent the best way to create a cache LV. They provide the most options and have the ability to cre‐

ate the most robust logical volumes. The examples below illustrate how these steps might be used in practice.

Cache Commands

0. create OriginLV

lvcreate -L LargeSize -n OriginLV VG SlowPVs

1. create CacheDataLV

lvcreate -L CacheSize -n CacheDataLV VG FastPVs

2. create CacheMetaLV

lvcreate -L MetaSize -n CacheMetaLV VG FastPVs

3. create CachePoolLV

lvconvert --type cache-pool --poolmetadata VG/CacheMetaLV VG/CacheDataLV

CachePoolLV takes the name of CacheDataLV.

CacheDataLV is renamed CachePoolLV_cdata and becomes hidden.

CacheMetaLV is renamed CachePoolLV_cmeta and becomes hidden.

4. create CacheLV

lvconvert --type cache --cachepool VG/CachePoolLV VG/OriginLV

CacheLV takes the name of OriginLV.

OriginLV is renamed OriginLV_corig and becomes hidden.

Cache Examples

Example 1: Creating a simple cache LV.

0. Create the origin LV

# lvcreate -L 10G -n lvx vg /dev/slow_dev

1. Create a cache data LV

# lvcreate -L 1G -n lvx_cache vg /dev/fast_dev

2. Create a cache metadata LV (~1/1000th size of CacheDataLV or 8MiB)

# lvcreate -L 8M -n lvx_cache_meta vg /dev/fast_dev

3. Create a cache pool LV, combining cache data LV and cache metadata LV

# lvconvert --type cache-pool --poolmetadata vg/lvx_cache_meta \

vg/lvx_cache

4. Create a cached LV by combining the cache pool LV and origin LV

# lvconvert --type cache --cachepool vg/lvx_cache vg/lvx

Example 2: Creating a cache LV with a fault tolerant cache pool LV.

Users who are concerned about the possibility of failures in their fast devices that could lead to data loss might con‐

sider making their cache pool sub-LVs redundant. Example 2 illustrates how to do that. Note that only steps 1 & 2

change.

0. Create an origin LV we wish to cache

# lvcreate -L 10G -n lvx vg /dev/slow_devs

1. Create a 2-way RAID1 cache data LV

# lvcreate --type raid1 -m 1 -L 1G -n lvx_cache vg \

/dev/fast1 /dev/fast2

2. Create a 2-way RAID1 cache metadata LV

# lvcreate --type raid1 -m 1 -L 8M -n lvx_cache_meta vg \

/dev/fast1 /dev/fast2

3. Create a cache pool LV combining cache data LV and cache metadata LV

# lvconvert --type cache-pool --poolmetadata vg/lvx_cache_meta \

vg/lvx_cache

4. Create a cached LV by combining the cache pool LV and origin LV

# lvconvert --type cache --cachepool vg/lvx_cache vg/lvx

Example 3: Creating a simple cache LV with writethough caching.

Some users wish to ensure that any data written will be stored both in the cache pool LV and on the origin LV. The loss

of a device associated with the cache pool LV in this case would not mean the loss of any data. When combining the cache

data LV and the cache metadata LV to form the cache pool LV, properties of the cache can be specified - in this case,

writethrough vs. writeback. Note that only step 3 is affected in this case.

0. Create an origin LV we wish to cache (yours may already exist)

# lvcreate -L 10G -n lvx vg /dev/slow

1. Create a cache data LV

# lvcreate -L 1G -n lvx_cache vg /dev/fast

2. Create a cache metadata LV

# lvcreate -L 8M -n lvx_cache_meta vg /dev/fast

3. Create a cache pool LV specifying cache mode "writethrough"

# lvconvert --type cache-pool --poolmetadata vg/lvx_cache_meta \

--cachemode writethrough vg/lvx_cache

4. Create a cache LV by combining the cache pool LV and origin LV

# lvconvert --type cache --cachepool vg/lvx_cache vg/lvx

Removing Cache Logical Volumes

If you wish to remove all logical volumes associated with a cache LV, you must remove both top-level, user-visible

devices. The cache metadata LV and cache data LV cannot be removed directly. If only the cache pool LV is specfied for

removal, any cached blocks not yet on the origin LV will be flush, the cache pool LV will be removed, and the now un-

cached origin LV will remain. If the user specifies a cache LV for removal, then the origin LV is removed and only the

cache pool LV will remain. The cache pool LV can then be used to create another cache LV with a different origin LV if

desired.

When users intend to remove all logical volumes associated with a cache LV, it is generally better to start with the ori‐

gin LV and then remove the cache pool LV. If the operations are performed in the reverse order, the user will have to

wait for the contents of the cache pool LV to be flushed before the origin LV is removed. This could take some time.

DIAGNOSTICS

All tools return a status code of zero on success or non-zero on failure.

FILES

/etc/lvm/lvm.conf

$HOME/.lvm_history