SquashFS with LZMA compression

最新推荐文章于 2022-12-07 17:35:14 发布
最新推荐文章于 2022-12-07 17:35:14 发布
阅读量2.5k 收藏
点赞数

SquashFS (.sfs) is a compressed read-only file system for Linux. SquashFS compresses files, inodes and directories, and supports block sizes up to 1 MB for greater compression. SquashFS is also free software (licensed under the GPL) for accessing SquashFS filesystems.

SquashFS is intended for general read-only file system use and in constrained block device/memory systems (e.g. embedded systems) where low overhead is needed. The standard version of SquashFS uses gzip compression.

Here is my How-To on how to get LZMA compressed SquashFS in Archlinux.

Requirements
1) squashfs-tools with LZMA patches. This package provides tools (mksquashfs, unsquashfs) to create Squashfs filesystem. With standard version (from /extra repository) you can create only gzip-compressed  Squashfs filesystem. To get LZMA support you must install package:
squashfs-tools-lzma
This package doesn't conflict with package from /extra. It provides mksquashfs.lzma, unsquashfs.lzma. So if you want create LZMA compressed filesystem you must use:

mksquashfs.lzma source dest

Otherwise for GZIP compressed filesystem:

mksquashfs source dest

2) squashfs module with LZMA patches. Without it you cant't mount your LZMA compressed filesystems. Here is two ways on how to get it.
2.1 Patch kernel with SquashFS LZMA patches.
Before proceed i recommend to read Custom Kernel Compilation with ABS
You must apply on sources three patches:
051-squashfs_pcomp.patch
052-pcomp_lzma_support.patch
053-squashfs_lzma.patch
And set in kernel config:

CONFIG_SQUASHFS=m|y
CONFIG_SQUASHFS_SUPPORT_ZLIB=y
CONFIG_SQUASHFS_SUPPORT_LZMA=y
CONFIG_CRYPTO_UNLZMA=m|y

m|y means that you can build it as a module (m) or as a part of kernel (y).
Build kernel as usual and after that you can mount LZMA compressed filesystems.

2.2 Build standalone module without patching and rebuilding the whole kernel. 
Warning. I am not Linux guru. And i think that this method is not optimal (but it works). Any suggestions are welcome from experienced users.
For ARCH kernel here is PKGBUILD on AUR:
kernel26-squashfs-lzma-modules

Create directory (called for example squashfs-module) somewhere on your hard disk. Cd to that directory;
Download here Linux kernel sources (i strongly recommend to use same sources as in PKGBUILD of your kernel);
Download LZMA patches listed in 2.1;
Now you need kernel config file. Get it from your kernel package;
Unpack linux kernel sources. Cd to that directory;
Copy kernel config file here and rename it to .config;
Now we can patch our kernel:
  patch -Np1 -i /pathto_051-squashfs_pcomp.patch
  patch -Np1 -i /pathto_052-pcomp_lzma_support.patch
  patch -Np1 -i /pathto_053-squashfs_lzma.patch
Before building modules we need some preparations:
  make prepare
  make modules_prepare
The kernel is patched and all preparations are ready. We can build modules:
  make modules SUBDIRS=crypto
  make modules SUBDIRS=fs/squashfs
Modules are ready. Now we need to copy it to our kernel modules directory:
  cp crypto/unlzma.ko /lib/modules/KERNELVERSION/misc
  cp fs/squashfs/squashfs.ko /lib/modules/KERNELVERSION/misc/squashfs_lzma.ko
Now execute:
  depmod KERNELVERSION
Modules are installed and ready for use.

Now you can load modules:

modprobe unlzma
modprobe squashfs_lzma

Remember to load both of them (unlzma and squashfs_lzma) and in that order.

I don't like this way. I think it is to possible to simplify this process. And this modules doesn't contain information about dependency. So you must manually load unlzma module.

How to use it
1) First we need to create SquashFS filesystem.
For GZIP:

mksquashfs source dest

For LZMA:

mksquashfs.lzma source dest

For example:

sudo mksquashfs.lzma /usr /usr.sqfs

2) Now we can mount it as usual.

sudo mount -t squashfs -o loop /usr.sqfs /mountpoint

Benchmark

du -hs /usr/
3.3G    /usr/

GZIP (standard):

sudo mksquashfs /usr/ /usr.sqfs
Parallel mksquashfs: Using 2 processors                
Creating 4.0 filesystem on /usr.sqfs, block size 131072.
[=============================\] 137140/137140 100%
Exportable Squashfs 4.0 filesystem, data block size 131072                                                            
        compressed data, compressed metadata, compressed fragments                                                    
        duplicates are removed                                                                                        
Filesystem size 1449060.58 Kbytes (1415.10 Mbytes)                                                                    
        46.41% of uncompressed filesystem size (3122543.79 Kbytes)                                                    
Inode table size 1365724 bytes (1333.71 Kbytes)                                                                       
        29.28% of uncompressed inode table size (4664165 bytes)                                                       
Directory table size 1437349 bytes (1403.66 Kbytes)                                                                   
        42.30% of uncompressed directory table size (3397611 bytes)                                                   
Number of duplicate files found 21775                                                                                 
Number of inodes 141741                                                                                               
Number of files 127438                                                                                                
Number of fragments 7820                                                                                              
Number of symbolic links  3853                                                                                        
Number of device nodes 0                                                                                              
Number of fifo nodes 0                                                                                                
Number of socket nodes 0                                                                                              
Number of directories 10450                                                                                           
Number of ids (unique uids + gids) 6                                                                                  
Number of uids 2                                                                                                      
        root (0)                                                                                                      
        policykit (102)                                                                                               
Number of gids 6                                                                                                      
        root (0)                                                                                                      
        locate (21)                                                                                                   
        mail (12)                                                                                                     
        tty (5)                                                                                                       
        policykit (102)                                                                                               
        dbus (81)                                                                                                     

real    7m28.167s
user    11m18.825s
sys     0m4.214s
du -h /usr.sqfs
1.4G    /usr.sqfs

LZMA (patched):

sudo mksquashfs.lzma /usr/ /usr.lzma.sqfs
Parallel mksquashfs: Using 2 processors                            
Creating 4.0 filesystem on /usr.lzma.sqfs, block size 131072.      
[=============================|] 137140/137140 100%
Exportable Squashfs 4.0 filesystem, data block size 131072                                                            
        compressed data, compressed metadata, compressed fragments                                                    
        duplicates are removed
Filesystem size 1326846.64 Kbytes (1295.75 Mbytes)
        42.49% of uncompressed filesystem size (3122543.79 Kbytes)
Inode table size 1067747 bytes (1042.72 Kbytes)
        22.89% of uncompressed inode table size (4664165 bytes)
Directory table size 1360891 bytes (1329.00 Kbytes)
        40.05% of uncompressed directory table size (3397611 bytes)
Number of duplicate files found 21775
Number of inodes 141741
Number of files 127438
Number of fragments 7820
Number of symbolic links  3853
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 10450
Number of ids (unique uids + gids) 6
Number of uids 2
        root (0)
        policykit (102)
Number of gids 6
        root (0)
        locate (21)
        mail (12)
        tty (5)
        policykit (102)
        dbus (81)

real    22m51.122s
user    28m39.617s
sys     9m46.870s
du -h /usr.lzma.sqfs
1.3G    /usr.lzma.sqfs

Results
Size of /usr: 3.1 Gib (3.327.028.002) 131.201 files, 10.642 sub-folders
Size of usr.sqfs: 1.4 Gib (1.483.841.536)
Size of usr.lzma.sqfs: 1.3 GiB (1.358.692.352)

P.s. Sorry for my poor english. Any corrections are welcome.
P.p.s. Any suggestions, improvements are welcome.


from:https://bbs.archlinux.org/viewtopic.php?id=83720

wzw88486969
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
squashfs-tools.zip
06-28
squashfs:一个无需解压就可以获取文件信息内容的插件
SQUASHFS error问题解决
07-29
QUASHFS error: sb_bread failed reading block 的错误一般都是文件系统不完整导致的。
squashfs1.3r3.tar.gz
04-21
SQUASHFS 1.3r3 - A squashed read-only filesystem for Linux Copyright 2004 Phillip Lougher ([email protected]) Released under the GPL licence (version 2 or later). Squashfs is currently at version 1.3 release 3. Please see the CHANGES file for recent changes to squashfs. Squashfs is a highly compressed read-only filesystem for Linux. It uses zlib compression to compress both files, inodes and directories. Inodes in the system are very small and all blocks are packed to minimise data overhead. Block sizes greater than 4K are supported up to a maximum of 32K. Squashfs is intended for general read-only filesystem use, for archival use (i.e. in cases where a .tar.gz file may be used), and in constrained block device/memory systems (e.g. embedded systems) where low overhead is needed. The section 'mksquashfs' gives information on using the mksquashfs tool to create and append to squashfs filesystems. The 'using squashfs' section gives information on mounting and using squashfs filesystems stored on block devices and as normal files using the loopback device. 1. Squashfs overview -------------------- 1. Data, inodes and directories are compressed. 2. Squashfs stores full uid/gids (32 bits), and file creation time. 3. Files up to 2^32 bytes are supported. Filesystems can be up to 2^32 bytes. 4. Inode and directory data are highly compacted, and packed on byte boundaries. Each compressed inode is on average 8 bytes in length (the exact length varies on file type, i.e. regular file, directory, symbolic link, and block/char device inodes have different sizes). 5. Squashfs can use block sizes up to 32K (the default size is 32K). Using 32K blocks achieves greater compression ratios than the normal 4K block size. 6. File duplicates are detected and removed. 7. Both big and little endian architectures are supported. Squashfs can mount filesystems created on different byte order machines. 2. mksquashfs ------------- As squashfs is a read-only filesystem, the mksquashfs program must be used to create populated squashfs filesystems. Beginning with Squashfs 1.2, mksquashfs will also append directories and files to pre-existing squashfs filesystems, see the following 'appending to squashfs filesystems' subsection. SYNTAX:mksquashfs source1 source2 ... dest [options] [-e list of exclude dirs/files] Options are -info print files written to filesystem -b block size size of blocks in filesystem, default 32768 -noappend Do not append to existing filesystem on dest, write a new filesystem This is the default action if dest does not exist, or if no filesystem is on it -keep-as-directory If one source directory is specified, create a root directory containing that directory, rather than the contents of the directory -root-becomes name When appending source files/directories, make the original root become a subdirectory in the new root called name, rather than adding the new source items to the original root -noI -noInodeCompression do not compress inode table -noD -noDataCompression do not compress data blocks -nopad do not pad filesystem to a multiple of 4K -check_data add checkdata for greater filesystem checks -le create a little endian filesystem -be create a big endian filesystem -ef exclude file file is a list of exclude dirs/files - one per line -version print version, licence and copyright message Source1 source2 ... are the source directories/files containing the files/directories that will form the squashfs filesystem. If a single directory is specified (i.e. mksquashfs source output_fs) the squashfs filesystem will consist of that directory, with the top-level root directory corresponding to the source directory. If multiple source directories or files are specified, mksquashfs will merge the specified sources into a single filesystem, with the root directory containing each of the source files/directories. The name of each directory entry will be the basename of the source path. If more than one source entry maps to the same name, the conflicts are named xxx_1, xxx_2, etc. where xxx is the original name. To make this clear, take two example directories. Source directory "/home/phillip/test" contains "file1", "file2" and "dir1". Source directory "goodies" contains "goodies1", "goodies2" and "goodies3". usage example 1: %mksquashfs /home/phillip/test output_fs This will generate a squashfs filesystem with root entries "file1", "file2" and "dir1". example 2: %mksquashfs /home/phillip/test goodies output_fs This will create a squashfs filesystem with the root containing entries "test" and "goodies" corresponding to the source directories "/home/phillip/test" and "goodies". example 3: %mksquashfs /home/phillip/test goodies test output_fs This is the same as the previous example, except a third source directory "test" has been specified. This conflicts with the first directory named "test" and will be renamed "test_1". Multiple sources allow filesystems to be generated without needing to copy all source files into a common directory. This simplifies creating filesystems. The -keep-as-directory option can be used when only one source directory is specified, and you wish the root to contain that directory, rather than the contents of the directory. For example: example 4: %mksquashfs /home/phillip/test output_fs -keep-as-directory This is the same as example 1, except for -keep-as-directory. This will generate a root directory containing directory "test", rather than the "test" directory contents "file1", "file2" and "dir1". The Dest argument is the destination where the squashfs filesystem will be written. This can either be a conventional file or a block device. If the file doesn't exist it will be created, if it does exist and a squashfs filesystem exists on it, mksquashfs will append. The -noappend option will write a new filesystem irrespective of whether an existing filesystem is present. The -e and -ef options allow files/directories to be specified which are excluded from the output filesystem. The -e option takes the exclude files/directories from the command line, the -ef option takes the exlude files/directories from the specified exclude file, one file/directory per line. If an exclude file/directory is absolute (i.e. prefixed with /, ../, or ./) the entry is treated as absolute, however, if an exclude file/directory is relative, it is treated as being relative to each of the sources in turn, i.e. %mksquashfs /tmp/source1 source2 output_fs -e ex1 /tmp/source1/ex2 out/ex3 Will generate exclude files /tmp/source1/ex2, /tmp/source1/ex1, source2/ex1, /tmp/source1/out/ex3 and source2/out/ex3. The -e and -ef exclude options are usefully used in archiving the entire filesystem, where it is wished to avoid archiving /proc, and the filesystem being generated, i.e. %mksquashfs / /tmp/root.sqsh -e proc /tmp/root.sqsh Multiple -ef options can be specified on the command line, and the -ef option can be used in conjuction with the -e option. The -info option displays the files/directories as they are compressed and added to the filesystem. The compression percentage achieved is printed, with the original uncompressed size. If the compression percentage is listed as 0% it means the file is a duplicate. The -b option allows the block size to be selected, this can be either 512, 1024, 2048, 4096, 8192, 16384, or 32768 bytes. The -noI and -noD options (also -noInodeCompression and -noDataCompression) can be used to force mksquashfs to not compress inodes/directories and data respectively. Giving both options generates an uncompressed filesystem. The -le and -be options can be used to force mksquashfs to generate a little endian or big endian filesystem. Normally mksquashfs will generate a filesystem in the host byte order. Squashfs, for portability, will mount different ordered filesystems (i.e. it can mount big endian filesystems running on a little endian machine), but these options can be used for greater optimisation. The -nopad option informs mksquashfs to not pad the filesystem to a 4K multiple. This is performed by default to enable the output filesystem file to be mounted by loopback, which requires files to be a 4K multiple. If the filesystem is being written to a block device, or is to be stored in a bootimage, the extra pad bytes are not needed. 2.1 appending to squashfs filesystems ------------------------------------- Beginning with squashfs1.2, mksquashfs can append to existing squashfs filesystems. Three extra options "-noappend", "-keep-as-directory", and "root-becomes" have been added. Running squashfs with the destination directory containing an existing filesystem, will add the source items to the existing filesystem. By default, the source items are added to the existing root directory. To make this clear... An existing filesystem "image" contains root entries "old1", and "old2". Source directory "/home/phillip/test" contains "file1", "file2" and "dir1". example 1: %mksquashfs /home/phillip/test image Will create a new "image" with root entries "old1", "old2", "file1", "file2" and "dir1" example 2: %mksquashfs /home/phillip/test image -keep-as-directory Will create a new "image" with root entries "old1", "old2", and "test". As shown in the previous section, for single source directories '-keep-as-directory' adds the source directory rather than the contents of the directory. example 3: %mksquashfs /home/phillip/test image -keep-as-directory -root-becomes original-root Will create a new "image" with root entries "original-root", and "test". The '-root-becomes' option specifies that the original root becomes a subdirectory in the new root, with the specified name. The append option with file duplicate detection, means squashfs can be used as a simple versioning archiving filesystem. A squashfs filesystem can be created with for example the linux-2.4.19 source. Appending the linux-2.4.20 source will create a filesystem with the two source trees, but only the changed files will take extra room, the unchanged files will be detected as duplicates. 3. Using squashfs ----------------- Squashfs filesystems should be mounted with 'mount' with the filesystem type 'squashfs'. If the filesystem is on a block device, the filesystem can be mounted directly, e.g. %mount -t squashfs /dev/sda1 /mnt Will mount the squashfs filesystem on "/dev/sda1" under the directory "/mnt". If the squashfs filesystem has been written to a file, the loopback device can be used to mount it (loopback support must be in the kernel), e.g. %mount -t squashfs image /mnt -o loop Will mount the squashfs filesystem in the file "image" under the directory "/mnt". 4. Filesystem layout -------------------- Brief filesystem design notes follow. A squashfs filesystem consists of five parts, packed together on a byte alignment: --------------- | superblock | |---------------| | data | | blocks | |---------------| | inodes | |---------------| | directories | |---------------| | uid/gid | | lookup table | --------------- Compressed data blocks are written to the filesystem as files are read from the source directory, and checked for duplicates. Once all file data has been written the completed inode, directory and uid/gid lookup tables are written. 4.1 Metadata ------------ Metadata (inodes and directories) are compressed in 8Kbyte blocks. Each compressed block is prefixed by a two byte length, the top bit is set if the block is uncompressed. A block will be uncompressed if the -noI option is set, or if the compressed block was larger than the uncompressed block. Inodes are packed into the metadata blocks, and are not aligned to block boundaries, therefore inodes overlap compressed blocks. An inode is identified by a two field tuple <start address of compressed block : offset into de-compressed block>. Inode contents vary depending on the file type. The base inode consists of: base inode: Inode type Mode uid index gid index The inode type is 4 bits in size, and the mode is 12 bits. The uid and gid indexes are 4 bits in length. Ordinarily, this will allow 16 unique indexes into the uid table. To minimise overhead, the uid index is used in conjunction with the spare bit in the file type to form a 48 entry index as follows: inode type 1 - 5: uid index = uid inode type 5 -10: uid index = 16 + uid inode type 11 - 15: uid index = 32 + uid In this way 48 unique uids are supported using 4 bits, minimising data inode overhead. The 4 bit gid index is used to index into a 15 entry gid table. Gid index 15 is used to indicate that the gid is the same as the uid. This prevents the 15 entry gid table filling up with the common case where the uid/gid is the same. The data contents of symbolic links are stored immediately after the symbolic link inode, inside the inode table. This allows the normally small symbolic link to be compressed as part of the inode table, achieving much greater compression than if the symbolic link was compressed individually. Similarly, the block index for regular files is stored immediately after the regular file inode. The block index is a list of block lengths (two bytes each), rather than block addresses, saving two bytes per block. The block address for a given block is computed by the summation of the previous block lengths. This takes advantage of the fact that the blocks making up a file are stored contiguously in the filesystem. The top bit of each block length is set if the block is uncompressed, either because the -noD option is set, or if the compressed block was larger than the uncompressed block. 4.2 Directories --------------- Like inodes, directories are packed into the metadata blocks, and are not aligned on block boundaries, therefore directories can overlap compressed blocks. A directory is, again, identified by a two field tuple <start address of compressed block containing directory start : offset into de-compressed block>. Directories are organised in a slightly complex way, and are not simply a list of file names and inode tuples. The organisation takes advantage of the observation that in most cases, the inodes of the files in the directory will be in the same compressed metadata block, and therefore, the inode tuples will have the same start block. Directories are therefore organised in a two level list, a directory header containing the shared start block value, and a sequence of directory entries, each of which share the shared start block. A new directory header is written once/if the inode start block changes. The directory header/directory entry list is repeated as many times as necessary. The organisation is as follows: directory_header: count (8 bits) inode start block (24 bits) directory entry: * count inode offset (13 bits) inode type (3 bits) filename size (8 bits) filename This organisation saves on average 3 bytes per filename. 4.3 File data ------------- File data is compressed on a block by block basis and written to the filesystem. The filesystem supports up to 32K blocks, which achieves greater compression ratios than the Linux 4K page size. The disadvantage with using greater than 4K blocks (and the reason why most filesystems do not), is that the VFS reads data in 4K pages. The filesystem reads and decompresses a larger block containing that page (e.g. 32K). However, only 4K can be returned to the VFS, resulting in a very inefficient filesystem, as 28K must be thrown away. Squashfs, solves this problem by explicitly pushing the extra pages into the page cache. 5. Author info -------------- Squashfs was written by Phillip Lougher, email [email protected], in Chepstow, Wales, UK. If you like the program, or have any problems, then please email me, as it's nice to get feedback!
squashfs2.2-r2.tar.gz
04-21
SQUASHFS 2.2 - A squashed read-only filesystem for Linux Copyright 2005 Phillip Lougher ([email protected]) Released under the GPL licence (version 2 or later). Welcome to Squashfs version 2.2-r2. Please see the CHANGES file for details of changes. Squashfs is a highly compressed read-only filesystem for Linux. It uses zlib compression to compress both files, inodes and directories. Inodes in the system are very small and all blocks are packed to minimise data overhead. Block sizes greater than 4K are supported up to a maximum of 64K. Squashfs is intended for general read-only filesystem use, for archival use (i.e. in cases where a .tar.gz file may be used), and in constrained block device/memory systems (e.g. embedded systems) where low overhead is needed. 1. SQUASHFS OVERVIEW -------------------- 1. Data, inodes and directories are compressed. 2. Squashfs stores full uid/gids (32 bits), and file creation time. 3. Files up to 2^32 bytes are supported. Filesystems can be up to 2^32 bytes. 4. Inode and directory data are highly compacted, and packed on byte boundaries. Each compressed inode is on average 8 bytes in length (the exact length varies on file type, i.e. regular file, directory, symbolic link, and block/char device inodes have different sizes). 5. Squashfs can use block sizes up to 64K (the default size is 64K). Using 64K blocks achieves greater compression ratios than the normal 4K block size. 6. File duplicates are detected and removed. 7. Both big and little endian architectures are supported. Squashfs can mount filesystems created on different byte order machines. 2. USING SQUASHFS ----------------- Squashfs filesystems should be mounted with 'mount' with the filesystem type 'squashfs'. If the filesystem is on a block device, the filesystem can be mounted directly, e.g. %mount -t squashfs /dev/sda1 /mnt Will mount the squashfs filesystem on "/dev/sda1" under the directory "/mnt". If the squashfs filesystem has been written to a file, the loopback device can be used to mount it (loopback support must be in the kernel), e.g. %mount -t squashfs image /mnt -o loop Will mount the squashfs filesystem in the file "image" under the directory "/mnt". 3. MKSQUASHFS ------------- 3.1 Mksquashfs options and overview. ------------------------------------ As squashfs is a read-only filesystem, the mksquashfs program must be used to create populated squashfs filesystems. SYNTAX:mksquashfs source1 source2 ... dest [options] [-e list of exclude dirs/files] Options are -version print version, licence and copyright message -info print files written to filesystem -b <block_size> set data block to <block_size>. Default 65536 bytes -2.0 create a 2.0 filesystem -noI do not compress inode table -noD do not compress data blocks -noF do not compress fragment blocks -no-fragments do not use fragments -always-use-fragments use fragment blocks for files larger than block size -no-duplicates do not perform duplicate checking -noappend do not append to existing filesystem -keep-as-directory if one source directory is specified, create a root directory containing that directory, rather than the contents of the directory -root-becomes <name> when appending source files/directories, make the original root become a subdirectory in the new root called <name>, rather than adding the new source items to the original root -all-root make all files owned by root -force-uid uid set all file uids to uid -force-gid gid set all file gids to gid -le create a little endian filesystem -be create a big endian filesystem -nopad do not pad filesystem to a multiple of 4K -check_data add checkdata for greater filesystem checks -root-owned alternative name for -all-root -noInodeCompression alternative name for -noI -noDataCompression alternative name for -noD -noFragmentCompression alternative name for -noF -sort <sort_file> sort files according to priorities in <sort_file>. One file or dir with priority per line. Priority -32768 to 32767, default priority 0 -ef <exclude_file> list of exclude dirs/files. One per line Source1 source2 ... are the source directories/files containing the files/directories that will form the squashfs filesystem. If a single directory is specified (i.e. mksquashfs source output_fs) the squashfs filesystem will consist of that directory, with the top-level root directory corresponding to the source directory. If multiple source directories or files are specified, mksquashfs will merge the specified sources into a single filesystem, with the root directory containing each of the source files/directories. The name of each directory entry will be the basename of the source path. If more than one source entry maps to the same name, the conflicts are named xxx_1, xxx_2, etc. where xxx is the original name. To make this clear, take two example directories. Source directory "/home/phillip/test" contains "file1", "file2" and "dir1". Source directory "goodies" contains "goodies1", "goodies2" and "goodies3". usage example 1: %mksquashfs /home/phillip/test output_fs This will generate a squashfs filesystem with root entries "file1", "file2" and "dir1". example 2: %mksquashfs /home/phillip/test goodies output_fs This will create a squashfs filesystem with the root containing entries "test" and "goodies" corresponding to the source directories "/home/phillip/test" and "goodies". example 3: %mksquashfs /home/phillip/test goodies test output_fs This is the same as the previous example, except a third source directory "test" has been specified. This conflicts with the first directory named "test" and will be renamed "test_1". Multiple sources allow filesystems to be generated without needing to copy all source files into a common directory. This simplifies creating filesystems. The -keep-as-directory option can be used when only one source directory is specified, and you wish the root to contain that directory, rather than the contents of the directory. For example: example 4: %mksquashfs /home/phillip/test output_fs -keep-as-directory This is the same as example 1, except for -keep-as-directory. This will generate a root directory containing directory "test", rather than the "test" directory contents "file1", "file2" and "dir1". The Dest argument is the destination where the squashfs filesystem will be written. This can either be a conventional file or a block device. If the file doesn't exist it will be created, if it does exist and a squashfs filesystem exists on it, mksquashfs will append. The -noappend option will write a new filesystem irrespective of whether an existing filesystem is present. 3.2 Changing compression defaults used in mksquashfs ---------------------------------------------------- There are a large number of options that can be used to control the compression in mksquashfs. By and large the defaults are the most optimum settings and should only be changed in exceptional circumstances! The -noI, -noD and -noF options (also -noInodeCompression, -noDataCompression and -noFragmentCompression) can be used to force mksquashfs to not compress inodes/directories, data and fragments respectively. Giving all options generates an uncompressed filesystem. The -no-fragments tells mksquashfs to not generate fragment blocks, and rather generate a filesystem similar to a Squashfs 1.x filesystem. It will of course still be a Squashfs 2.0 filesystem but without fragments, and so it won't be mountable on a Squashfs 1.x system. The -always-use-fragments option tells mksquashfs to always generate fragments for files irrespective of the file length. By default only small files less than the block size are packed into fragment blocks. The ends of files which do not fit fully into a block, are NOT by default packed into fragments. To illustrate this, a 100K file has an initial 64K block and a 36K remainder. This 36K remainder is not packed into a fragment by default. This is because to do so leads to a 10 - 20% drop in sequential I/O performance, as a disk head seek is needed to seek to the initial file data and another disk seek is need to seek to the fragment block. Specify this option if you want file remainders to be packed into fragment blocks. Doing so may increase the compression obtained BUT at the expense of I/O speed. The -no-duplicates option tells mksquashfs to not check the files being added to the filesystem for duplicates. This can result in quicker filesystem generation and appending although obviously compression will suffer badly if there is a lot of duplicate files. The -b option allows the block size to be selected, this can be either 4096, 8192, 16384, 32768 or 65536 bytes. 3.3 Specifying the UIDs/GIDs used in the filesystem --------------------------------------------------- By default files in the generated filesystem inherit the UID and GID ownership of the original file. However, mksquashfs provides a number of options which can be used to override the ownership. The options -all-root and -root-owned (both do exactly the same thing) force all file uids/gids in the generated Squashfs filesystem to be root. This allows root owned filesystems to be built without root access on the host machine. The "-force-uid uid" option forces all files in the generated Squashfs filesystem to be owned by the specified uid. The uid can be specified either by name (i.e. "root") or by number. The "-force-gid gid" option forces all files in the generated Squashfs filesystem to be group owned by the specified gid. The gid can be specified either by name (i.e. "root") or by number. 3.4 Excluding files from the filesystem --------------------------------------- The -e and -ef options allow files/directories to be specified which are excluded from the output filesystem. The -e option takes the exclude files/directories from the command line, the -ef option takes the exlude files/directories from the specified exclude file, one file/directory per line. If an exclude file/directory is absolute (i.e. prefixed with /, ../, or ./) the entry is treated as absolute, however, if an exclude file/directory is relative, it is treated as being relative to each of the sources in turn, i.e. %mksquashfs /tmp/source1 source2 output_fs -e ex1 /tmp/source1/ex2 out/ex3 Will generate exclude files /tmp/source1/ex2, /tmp/source1/ex1, source2/ex1, /tmp/source1/out/ex3 and source2/out/ex3. The -e and -ef exclude options are usefully used in archiving the entire filesystem, where it is wished to avoid archiving /proc, and the filesystem being generated, i.e. %mksquashfs / /tmp/root.sqsh -e proc /tmp/root.sqsh Multiple -ef options can be specified on the command line, and the -ef option can be used in conjuction with the -e option. 3.5 Appending to squashfs filesystems ------------------------------------- Running squashfs with the destination directory containing an existing filesystem will add the source items to the existing filesystem. By default, the source items are added to the existing root directory. To make this clear... An existing filesystem "image" contains root entries "old1", and "old2". Source directory "/home/phillip/test" contains "file1", "file2" and "dir1". example 1: %mksquashfs /home/phillip/test image Will create a new "image" with root entries "old1", "old2", "file1", "file2" and "dir1" example 2: %mksquashfs /home/phillip/test image -keep-as-directory Will create a new "image" with root entries "old1", "old2", and "test". As shown in the previous section, for single source directories '-keep-as-directory' adds the source directory rather than the contents of the directory. example 3: %mksquashfs /home/phillip/test image -keep-as-directory -root-becomes original-root Will create a new "image" with root entries "original-root", and "test". The '-root-becomes' option specifies that the original root becomes a subdirectory in the new root, with the specified name. The append option with file duplicate detection, means squashfs can be used as a simple versioning archiving filesystem. A squashfs filesystem can be created with for example the linux-2.4.19 source. Appending the linux-2.4.20 source will create a filesystem with the two source trees, but only the changed files will take extra room, the unchanged files will be detected as duplicates. 3.6 Miscellaneous options ------------------------- The -info option displays the files/directories as they are compressed and added to the filesystem. The original uncompressed size of each file is printed, along with DUPLICATE if the file is a duplicate of a file in the filesystem. The -le and -be options can be used to force mksquashfs to generate a little endian or big endian filesystem. Normally mksquashfs will generate a filesystem in the host byte order. Squashfs, for portability, will mount different ordered filesystems (i.e. it can mount big endian filesystems running on a little endian machine), but these options can be used for greater optimisation. The -nopad option informs mksquashfs to not pad the filesystem to a 4K multiple. This is performed by default to enable the output filesystem file to be mounted by loopback, which requires files to be a 4K multiple. If the filesystem is being written to a block device, or is to be stored in a bootimage, the extra pad bytes are not needed. 4. FILESYSTEM LAYOUT -------------------- Brief filesystem design notes follow for the original 1.x filesystem layout. A description of the 2.0 filesystem layout will be written sometime! A squashfs filesystem consists of five parts, packed together on a byte alignment: --------------- | superblock | |---------------| | data | | blocks | |---------------| | inodes | |---------------| | directories | |---------------| | uid/gid | | lookup table | --------------- Compressed data blocks are written to the filesystem as files are read from the source directory, and checked for duplicates. Once all file data has been written the completed inode, directory and uid/gid lookup tables are written. 4.1 Metadata ------------ Metadata (inodes and directories) are compressed in 8Kbyte blocks. Each compressed block is prefixed by a two byte length, the top bit is set if the block is uncompressed. A block will be uncompressed if the -noI option is set, or if the compressed block was larger than the uncompressed block. Inodes are packed into the metadata blocks, and are not aligned to block boundaries, therefore inodes overlap compressed blocks. An inode is identified by a two field tuple <start address of compressed block : offset into de-compressed block>. Inode contents vary depending on the file type. The base inode consists of: base inode: Inode type Mode uid index gid index The inode type is 4 bits in size, and the mode is 12 bits. The uid and gid indexes are 4 bits in length. Ordinarily, this will allow 16 unique indexes into the uid table. To minimise overhead, the uid index is used in conjunction with the spare bit in the file type to form a 48 entry index as follows: inode type 1 - 5: uid index = uid inode type 5 -10: uid index = 16 + uid inode type 11 - 15: uid index = 32 + uid In this way 48 unique uids are supported using 4 bits, minimising data inode overhead. The 4 bit gid index is used to index into a 15 entry gid table. Gid index 15 is used to indicate that the gid is the same as the uid. This prevents the 15 entry gid table filling up with the common case where the uid/gid is the same. The data contents of symbolic links are stored immediately after the symbolic link inode, inside the inode table. This allows the normally small symbolic link to be compressed as part of the inode table, achieving much greater compression than if the symbolic link was compressed individually. Similarly, the block index for regular files is stored immediately after the regular file inode. The block index is a list of block lengths (two bytes each), rather than block addresses, saving two bytes per block. The block address for a given block is computed by the summation of the previous block lengths. This takes advantage of the fact that the blocks making up a file are stored contiguously in the filesystem. The top bit of each block length is set if the block is uncompressed, either because the -noD option is set, or if the compressed block was larger than the uncompressed block. 4.2 Directories --------------- Like inodes, directories are packed into the metadata blocks, and are not aligned on block boundaries, therefore directories can overlap compressed blocks. A directory is, again, identified by a two field tuple <start address of compressed block containing directory start : offset into de-compressed block>. Directories are organised in a slightly complex way, and are not simply a list of file names and inode tuples. The organisation takes advantage of the observation that in most cases, the inodes of the files in the directory will be in the same compressed metadata block, and therefore, the inode tuples will have the same start block. Directories are therefore organised in a two level list, a directory header containing the shared start block value, and a sequence of directory entries, each of which share the shared start block. A new directory header is written once/if the inode start block changes. The directory header/directory entry list is repeated as many times as necessary. The organisation is as follows: directory_header: count (8 bits) inode start block (24 bits) directory entry: * count inode offset (13 bits) inode type (3 bits) filename size (8 bits) filename This organisation saves on average 3 bytes per filename. 4.3 File data ------------- File data is compressed on a block by block basis and written to the filesystem. The filesystem supports up to 32K blocks, which achieves greater compression ratios than the Linux 4K page size. The disadvantage with using greater than 4K blocks (and the reason why most filesystems do not), is that the VFS reads data in 4K pages. The filesystem reads and decompresses a larger block containing that page (e.g. 32K). However, only 4K can be returned to the VFS, resulting in a very inefficient filesystem, as 28K must be thrown away. Squashfs, solves this problem by explicitly pushing the extra pages into the page cache. 5. AUTHOR INFO -------------- Squashfs was written by Phillip Lougher, email [email protected], in Chepstow, Wales, UK. If you like the program, or have any problems, then please email me, as it's nice to get feedback!
squashfs4.2
10-23
SquashFS 是一套基于Linux内核使用的压缩只读文件系统
内核添加对jffs2 的支持
kunkliu的博客
06-17 1312
转载地址：https://blog.csdn.net/whereisdog/article/details/82314860 海思最新出了一款超低内存的芯片HI3518EV201（内存只有32M） 在开发过程中对内核的裁剪考验相当大，当然海思也提供了参考的裁剪方案《hi3518ev20x_mini_config.txt》。 但是不能完全按照海思提供的裁剪方案来玩，因为该内核不支持USB、不支...
linux内核压缩包解压方法,Squashfs文件解压缩及”xz compression”报错问题解决
weixin_31577315的博客
05-04 3602
1、普通squashfs文件解压缩及查看(1) 可使用unsuqashfs命令直接解压缩。[root@localhost squashfs]# unsquashfs squashfs.img(2)也可mount到临时目录[root@localhost squashfs]# mount -o loop squashfs.img -t squashfs temp/2、XZ算法压缩后的squashfs....
SquashFS 增加压缩解压lz4支持
贪嘴的博客
12-07 1935
一般 SquashFS 工具是我们逆向固件解包时经常会使用的工具，今天刚好有个SquashFS格式的包是lz4压缩的，发现使用 unsquashfs 的时候报 命令查看日志 发现有一条这个报错 没有不支持lz4压缩解决方案如下： 到这里找到lz4下载源码 下载好后解压，进入目录 安装完之后，注意系统有没有把下面这个路径加入库搜索目录中 配置文件在：/etc/ld.so.conf文件中 将该目录加入到共享库的配置文件中将动态库文件加入配置：执行 在 下方增加 保存后，在命令行终端执行： 然后到 下载源码 下
无法解决：安装ubuntu错误代码刷屏squashfc error
weixin_43648213的博客
10-02 6670
嵌入式linux系统运行过程中报告SQUASHFS error错误 SQUASHFS error: sb_bread failed reading block 的错误一般都是文件系统不完整导致的。
【Linux】【文件系统】squashfs文件系统挂载失败问题
vickytong1018的博客
03-18 9356
最近负责将A公司一个比较稳定的代码分支移植到新平台上，新平台中包含了M公司和B公司的代码，这样的一个代码架构方便以后不同公司芯片方案的添加和开发。 在移植的过程中，编译成功后DUT上电，但是出现解压文件系统失败，串口信息： 0.760000] unlzma 632 [    0.760000] unlzma 634 [    0.764000] unlzma 632 [    0
mksquashfs
09-22
mksquashfs,制作squashfs文件系统需要用的工具
lzma linux 源码,手动修改2.6.30内核源码 让其同时支持squashfs和lzma
weixin_29421409的博客
05-12 493
fs/squashfs/Kconfig2 2 tristate "SquashFS 4.0 - Squashed file system support"3 3 depends on BLOCK4 4 select CRYPTO5 select CRYPTO_ZLIB6 5 help7 6 Sa...
安装ubuntu出现filesystem.squashfs错误 - 解决方案(作参考)
weixin_33862188的博客
11-25 8997
2019独角兽企业重金招聘Python工程师标准>>> ...
SQUASHFS error
热门推荐
不积跬步无以至千里
07-18 2万+
嵌入式linux系统运行过程中报告SQUASHFS error错误, 并且在系统重启之后就能恢复正常. [ 5742.104968] SQUASHFS error: xz decompression failed, data probably corrupt [ 5742.118815] SQUASHFS error: squashfs_read_data failed to read blo...
详解制作根文件系统
weixin_59665492的博客
02-16 7310
系列文章目录 提示：这里可以添加系列文章的所有文章的目录，目录需要自己手动添加 例如：第一章 Python 机器学习入门之pandas的使用 提示：写完文章后，目录可以自动生成，如何生成可参考右边的帮助文档 文章目录 系列文章目录 前言 一、pandas是什么？ 二、使用步骤 1.引入库 2.读入数据 总结 前言 提示：这里可以添加本文要记录的大概内容： 例如：随着人工智能的不断发展，机器学习这门技术也越来越重要，很多人都开启了学习机器学习，本文就介绍了机器学习的基
编译squashfs和lzma
lurayvis的专栏
08-23 9804
编译squashfs和lzma 早年写的笔记，压箱底了，翻出来晒晒 目          录 编译squashfs和lzma. 1 openwrt里的squashfs-lzma. 1 利用openwrt源码编译squashfs-lzma. 2 编译lzma432.tar.bz2. 2 编译squashfs-lzma. 3 摸索的过程... 4 3.4版本的
只读文件系统 squashFS
daydayup3stones的专栏
05-12 2020
squashfs是眾多Live CD使用的檔案系統,Why?因為它那嚇死人的壓縮率,舉個例子,以筆者製做的root filesystem為例,壓縮過後的initial ram disk大小為1.3MB,但squashfs製做出來的file systemimage只有500多k,squashfs image居然只有compressed initial ram diskimage一半的容量,這種節省儲
linux lzma c语言接口,编译squashfs和lzma
weixin_33260057的博客
05-12 684
编译squashfs和lzma早年写的笔记，压箱底了，翻出来晒晒目 录南瓜文件系统(Squashfs)采用gzip或lamz(更高压缩率)来制作只读的映像文件。LZMA 是 7-Zip 程序中 7z 格式 的默认压缩算法。LZMA 能提供给用户极高的压缩比及较快的压缩速度，它非常适合与应用程序集成，拥有不错的压缩比。Squashfs也可以使用LZMA的算法进行更好的压缩。Squa...
squashfs blk
最新发布
07-13
SquashFS (SQUash File System) 是一种只读的压缩文件系统，用于在 Linux 系统中进行文件系统的镜像和压缩。它可以将文件和目录压缩为单个文件，并且可以在运行时以只读方式挂载这个压缩文件系统。 "blk" 是 SquashFS 文件系统中的一个常见选项。它用于指定块大小，即文件系统中用于压缩的块的大小。块大小的选择会影响文件系统的压缩比率和性能。通常，较小的块大小可以提供更好的压缩比率，但会增加读取和写入操作的开销。相反，较大的块大小可以提供更高的读取和写入性能，但可能导致较低的压缩比率。 在创建 SquashFS 文件系统时，可以使用 "mksquashfs" 命令并通过 "-b" 选项来指定块大小。例如，"mksquashfs source_directory output.squashfs -b 4096" 将创建一个块大小为 4096 字节的 SquashFS 文件系统。 需要注意的是，blk 是 SquashFS 中的一个选项，而不是 SquashFS 文件系统本身的名称。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值