EXT4介绍和分析

本文未经作者许可，严禁任何转载、发表，本文为原创文章，版权归作者本人所有

There are two reference resources:

1.https://ext4.wiki.kernel.org/index.php/Main_Page

2.http://www.ibm.com/developerworks/cn/linux/l-cn-filesrc5/

3.http://hi.baidu.com/hgggg18/blog/item/b4f7524ab436fc3c08f7efec.html

1.Overview

Ext4 divides a storage device into an array of logical blocks，the point number of block start from 0.For each block, the physical address is block_number * block_size. The default size of block is 4kb. Actually, the ext4 file system is consisted of many Group Blocks. Each group is combined by several blocks. 

2.Layout

The layout of a standard block group is approximately as follows (each of these fields is discussed in a separate section below):

Group 0 Padding	ext4 Super Block	Group Descriptors	Reserved GDT Blocks	Data Block Bitmap	inode Bitmap	inode Table	Data Blocks
1024 bytes	1 block	many blocks	many blocks	1 block	1 block	many blocks	many more blocks

The following figure shows the combination of any disk. The light blue area before Super block is Group 0 Padding which is unsed.

So You could find the global structure from the figure above.

3. Super Block

We need to define the structure of the Super Block. But I just want to show the Block 0 to tell you how to analyses it.

00000000 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000001 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000002 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000003 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000004 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000005 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000006 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000007 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000008 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000009 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000A : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000B : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000C : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000E : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000F : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000010 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000011 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000012 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000013 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000014 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000015 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000016 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000017 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000018 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000019 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001A : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001B : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001C : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001E : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001F : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000020 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000021 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000022 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000023 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000024 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000025 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000026 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000027 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000028 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000029 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002A : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002B : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002C : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002E : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002F : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000030 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000031 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000032 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000033 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000034 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000035 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000036 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000037 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000038 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000039 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000003A : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000003B : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000003C : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000003D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000003E : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000003F : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

from 0x0 to 0x3FF, the values are 0 because of Group 0 Padding.

The super block start from 0x400

00000040 : 50 09 0f 00 f8 17 3c 00 32 01 03 00 cb 80 3a 00
00000041 : 43 09 0f 00 00 00 00 00 02 00 00 00 02 00 00 00
00000042 : 00 80 00 00 00 80 00 00 d0 1f 00 00 2b 53 a9 4f
00000043 : 44 53 a9 4f 05 00 1c 00 53 ef 01 00 01 00 00 00
00000044 : 7b 45 83 4f 00 4e ed 00 00 00 00 00 01 00 00 00
00000045 : 00 00 00 00 0b 00 00 00 00 01 00 00 3c 00 00 00
00000046 : 42 02 00 00 7b 00 00 00 42 13 2a 64 66 a1 4e 14
00000047 : 92 0e 60 7e d4 f4 b8 12 00 00 00 00 00 00 00 00
00000048 : 00 00 00 00 00 00 00 00 2f 6d 65 64 69 61 2f 34
00000049 : 32 31 33 32 61 36 34 2d 36 36 61 31 2d 34 65 31
0000004A : 34 2d 39 32 30 65 2d 36 30 37 65 64 34 66 34 62
0000004B : 38 31 32 00 89 ee fd f0 80 0a b2 f4 d8 e7 29 f5
0000004C : 00 c2 a8 f4 94 9e b6 f4 00 00 00 00 00 00 c1 03
0000004D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000004E : 08 00 00 00 00 00 00 00 00 00 00 00 72 96 98 b0
0000004F : 41 5b 4b 96 81 26 a9 e5 f7 f9 4f 29 01 01 00 00
00000050 : 00 00 00 00 00 00 00 00 7b 45 83 4f 0a f3 02 00
00000051 : 04 00 00 00 00 00 00 00 00 00 00 00 ff 7f 00 00
00000052 : c3 83 18 00 ff 7f 00 00 01 00 00 00 c2 03 19 00
00000053 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000054 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08
00000055 : 00 00 00 00 00 00 00 00 00 00 00 00 1c 00 1c 00
00000056 : 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000057 : 00 00 00 00 04 00 00 00 0f d4 05 00 00 00 00 00
00000058 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000059 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000005A : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000005B : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000005C : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000005D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000005E : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000005F : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

We need to pay more attention to those values:

0x0

__le32

s_inodes_count

Total inode count.

0x4

__le32

s_blocks_count_lo

Total block count.

0x18

__le32

s_log_block_size

Block size is 2 ^ (10 + s_log_block_size).

0x20

__le32

s_blocks_per_group

Blocks per group.

0x28

__le32

s_inodes_per_group

Inodes per group.

0x58

__le16

s_inode_size

Size of inode structure, in bytes.

we need to calculate following values:

block size = 2 ^ (10 + s_log_block_size)

amount of group descriptor = s_blocks_count_lo/s_blocks_per_group

amount of blocks for inodes per group  = s_indoes_per_group * s_inode_size/block_size

3. Group Descriptors

We need pay more attention that, group descriptors are start from the next block of super block. It means that if the super block is in Block 0, the group descriptors are start from Block 1. All the group descriptors must be in one block! For each group descriptor, it is 64 bytes

So at most , the amount of group descriptor = 4096 bytes / 64 bytes. 

I just show you one descripor(64 bytes):

  
00000000 : c3 03 00 00 d3 03 00 00 e3 03 00 00 47 5c c5 1f
00000001 : 02 00 04 00 00 00 00 00 00 00 00 00 c5 1f c2 a6

We need to pay attention to some important values in each group descriptor

0xE

__le16

bg_free_inodes_count_lo

Lower 16-bits of free inode count.

if bg_free_inodes_count = s_inodes_per_group, it means this group is unused

and s_inodes_per_group - bg_free_inodes_count = how many inodes are used in this group.

For example, s_inodes_per_group - bg_free_inodes_count = 12, it means the inode 0(offest is 0, rather than the global serial number of inode ) to inode 11 are used, others are unused

0x8

__le32

bg_inode_table_lo

Lower 32-bits of location of inode table.

This is the block number which point to the inode table located in this group. The block number is very very important! The physical address of this block is:

Physical_inode_table_address = bg_inode_table_lo * block_size

for example, the picture above, the inode block number(bg_inode_table_lo) =0x03e3 and the real physical address is 0x3e3*4096

So I will show you the code of  inode table (the address for me is 0x3e3*4096)

00000000 : 00 00 00 00 00 00 00 00 ad 45 83 4f ad 45 83 4f
00000001 : ad 45 83 4f 00 00 00 00 00 00 00 00 00 00 00 00
00000002 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000003 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000004 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000005 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000006 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000007 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000008 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000009 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000A : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000B : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000C : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000E : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000F : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000010 : ed 41 00 00 00 10 00 00 3d 52 a9 4f c3 36 97 4f
00000011 : c3 36 97 4f 00 00 00 00 00 00 04 00 08 00 00 00
00000012 : 00 00 00 00 01 00 00 00 b3 23 00 00 00 00 00 00
00000013 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000014 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000015 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000016 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000017 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000018 : 1c 00 00 00 7c 34 27 1b 7c 34 27 1b 78 df d7 73
00000019 : ad 45 83 4f 00 00 00 00 00 00 00 00 00 00 00 00
0000001A : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001B : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001C : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001E : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001F : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000020 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000021 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000022 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000023 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000024 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000025 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000026 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000027 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000028 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000029 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002A : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002B : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002C : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002E : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000002F : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

From the super block, we know each inode is 0x0100 bytes(default), also you could find this value in s_inode_size in super block!

From 00-FF, it is the inode 1, pay more attention, the number of inode start from 1 rather 0!

Your root of files system is start from inode 2.

4.Inode Table

Now , we need to know some thing about inode

0x28

__le32

i_block[EXT4_N_BLOCKS=15]

Block map or extent tree. See the section "The Contents of i_block".

It means we have 15*32bits area for i_block, 15*32=480bit = 60bytes

There are two kinds of i_block, the one is that it could store at most 4 directly file blocks, another one is that it store a extent tree.

I can show  you two kinds of inode.

1. The type one (store file blcok directly)

00000010 : ed 41 00 00 00 10 00 00 3d 52 a9 4f c3 36 97 4f
00000011 : c3 36 97 4f 00 00 00 00 00 00 04 00 08 00 00 00
00000012 : 00 00 00 00 01 00 00 00 b3 23 00 00 00 00 00 00
00000013 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000014 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000015 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000016 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000017 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000018 : 1c 00 00 00 7c 34 27 1b 7c 34 27 1b 78 df d7 73
00000019 : ad 45 83 4f 00 00 00 00 00 00 00 00 00 00 00 00
0000001A : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001B : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001C : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001E : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000001F : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Thus, the file block number is 0x23b3

2.Inode with extent tree

00000070 : 80 81 00 00 00 00 00 08 ea 45 83 4f ea 45 83 4f
00000071 : ea 45 83 4f 00 00 00 00 00 00 01 00 00 00 04 00
00000072 : 00 00 08 00 00 00 00 00 0a f3 02 00 04 00 00 00
00000073 : 00 00 00 00 00 00 00 00 ff 7f 00 00 c3 83 18 00
00000074 : ff 7f 00 00 01 00 00 00 c2 03 19 00 00 00 00 00
00000075 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000076 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000077 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000078 : 1c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000079 : ea 45 83 4f 00 00 00 00 00 00 00 00 00 00 00 00
0000007A : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000007B : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000007C : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000007D : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000007E : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000007F : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Extents are arranged as a tree. Each node of the tree begins with a struct ext4_extent_header. If the node is an interior node (eh.eh_depth > 0), the header is followed by eh.eh_entries instances of struct ext4_extent_idx; each of these index entries points to a block containing more nodes in the extent tree. If the node is a leaf node (eh.eh_depth == 0), then the header is followed by eh.eh_entries instances of struct ext4_extent; these instances point to the file's data blocks. The root node of the extent tree is stored in inode.i_block, which allows for the first four extents to be recorded without the use of extra metadata blocks.

EXT4_EXTENT_HEADER

Offset	Size	Name	Description
0x0	__le16	eh_magic	Magic number, 0xF30A.
0x2	__le16	eh_entries	Number of valid entries following the header.
0x4	__le16	eh_max	Maximum number of entries that could follow the header.
0x6	__le16	eh_depth	Depth of this extent node in the extent tree. 0 = this extent node points to data blocks; otherwise, this extent node points to other extent nodes.
0x8	__le32	eh_generation	Generation of the tree. (Used by Lustre, but not standard ext4).

So we have found magic number which is 0xF30A, and the eh_entries is 0x02, it means there are two other extents in same i_block which will follow the header directly but we don't know the type of those extents ! the eh_depth is 0. it means the following extents are leaf extent, if eh_depth>0, it means the following extents are idx_extents!

For the current situation, we need to do analyses EXT4_EXTENT_LEAF

Offset	Size	Name	Description
0x0	__le32	ee_block	First file block number that this extent covers.
0x4	__le16	ee_len	Number of blocks covered by extent.
0x6	__le16	ee_start_hi	Upper 16-bits of the block number to which this extent points.
0x8	__le32	ee_start_lo	Lower 32-bits of the block number to which this extent points.

Header  0a f3 02 00 04 00 00 00 00 00 00 00

Extent  00 00 00 00 ff 7f 00 00 c3 83 18 00 #Point to 0x001883c3, Covers 7ffblocks

Extent  ff 7f 00 00  01 00 00 00 c2 03 19 00 #Point to  0x001903c2Covers 1 bloks