先自我介绍一下,小编浙江大学毕业,去过华为、字节跳动等大厂,目前在阿里
深知大多数程序员,想要提升技能,往往是自己摸索成长,但自己不成体系的自学效果低效又漫长,而且极易碰到天花板技术停滞不前!
因此收集整理了一份《2024年最新Linux运维全套学习资料》,初衷也很简单,就是希望能够帮助到想自学提升又不知道该从何学起的朋友。
既有适合小白学习的零基础资料,也有适合3年以上经验的小伙伴深入学习提升的进阶课程,涵盖了95%以上运维知识点,真正体系化!
由于文件比较多,这里只是将部分目录截图出来,全套包含大厂面经、学习笔记、源码讲义、实战项目、大纲路线、讲解视频,并且后续会持续更新
前言
本文主要描述了 I/O 端口和 I/O 内存。
I/O 内存:/proc/iomem
I/O 端口:/proc/ioports
ioremap:完成物理地址到内核虚拟地址空间的转换,用于外部设备的物理寄存器,不能用于System RAM。 (几乎每一种外设都是通过读写设备上的相关寄存器来进行的,通常包括控制寄存器、状态寄存器和数据寄存器三大类,外设的寄存器通常被连续地编址)
mmap:完成物理地址到用户虚拟地址空间的转换。
一、/proc/iomem
1.1 简介
大多数适用于PCI总线的current cards(以及其他cards)都向总线提供一个或多个I/O内存区域(I/O memory regions)。通过访问这些区域,处理器可以与外设通信。查看/proc/iomem将显示在给定系统上已经注册的I/O内存区域。
我们常说的内存条和物理内存是不一样的,物理内存是指物理地址空间 ,内存条只是映射到这个地址空间的一部分,其余的还有各种PCI设备,IO端口。
/proc/iomem 文件显示每个物理设备的系统内存的当前映射,/proc/iomem 是一个虚拟文件,在 kernel/resource.c 中创建。 它列出了映射到物理地址空间的各种 I/O 内存区域,包括 RAM,即我们所说的内存条。
1.2 ioremap
为了处理I/O内存区域( ioremap 只处理 soc 的 io memory ,不处理 System RAM)驱动程序应该通过调用ioremap()来映射该区域。ioremap主要是检查传入地址的合法性,建立页表(包括访问权限),完成物理地址到内核虚拟地址的转换。
// linux-3.10/arch/x86/include/asm/io.h
\* phys_addr:内存总线地址(即要映射的物理地址)
\* size:要映射的资源大小
\* void __iomem \*:返回一个内核虚拟地址
/\*
\* The default ioremap() behavior is non-cached:
\*/
static inline void __iomem \*ioremap(resource\_size\_t offset, unsigned long size)
{
return ioremap\_nocache(offset, size);
}
// linux-3.10/arch/x86/mm/ioremap.c
/\*\*
\* ioremap\_nocache - map bus memory into CPU space
\* @phys\_addr: bus address of the memory
\* @size: size of the resource to map
\*
\* ioremap\_nocache performs a platform specific sequence of operations to
\* make bus memory CPU accessible via the readb/readw/readl/writeb/
\* writew/writel functions and the other mmio helpers. The returned
\* address is not guaranteed to be usable directly as a virtual
\* address.
\*
\* This version of ioremap ensures that the memory is marked uncachable
\* on the CPU as well as honouring existing caching rules from things like
\* the PCI bus. Note that there are other caches and buffers on many
\* busses. In particular driver authors should read up on PCI writes
\*
\* It's useful if some control registers are in such an area and
\* write combining or read caching is not desirable:
\*
\* Must be freed with iounmap.
\*/
void __iomem \*ioremap\_nocache(resource\_size\_t phys_addr, unsigned long size)
{
/\*
\* Ideally, this should be:
\* pat\_enabled ? \_PAGE\_CACHE\_UC : \_PAGE\_CACHE\_UC\_MINUS;
\*
\* Till we fix all X drivers to use ioremap\_wc(), we will use
\* UC MINUS.
\*/
unsigned long val = _PAGE_CACHE_UC_MINUS;
return \_\_ioremap\_caller(phys_addr, size, val,
\_\_builtin\_return\_address(0));
}
EXPORT\_SYMBOL(ioremap_nocache);
// linux-3.10/arch/x86/mm/ioremap.c
/\*
\* Remap an arbitrary physical address space into the kernel virtual
\* address space. Needed when the kernel wants to access high addresses
\* directly.
\*
\* NOTE! We need to allow non-page-aligned mappings too: we will obviously
\* have to convert them into an offset in a page-aligned mapping, but the
\* caller shouldn't need to know that small detail.
\*/
static void __iomem \*\_\_ioremap\_caller(resource\_size\_t phys_addr,
unsigned long size, unsigned long prot_val, void \*caller)
{
struct vm\_struct \*area;
......
/\*
\* Don't remap the low PCI/ISA area, it's always mapped..
\*/
if (is\_ISA\_range(phys_addr, last_addr))
return (__force void __iomem \*)phys\_to\_virt(phys_addr);
/\*
\* Don't allow anybody to remap normal RAM that we're using..
\*/
//检查要映射的物理地址phys\_addr是否始于System RAM,如果是,则不映射
last_pfn = last_addr >> PAGE_SHIFT;
for (pfn = phys_addr >> PAGE_SHIFT; pfn <= last_pfn; pfn++) {
int is_ram = page\_is\_ram(pfn);
if (is_ram && pfn\_valid(pfn) && !PageReserved(pfn\_to\_page(pfn)))
return NULL;
WARN\_ON\_ONCE(is_ram);
}
......
/\*
\* Ok, go for it..
\*/
//将 area区域家兔vmalloc区中的vmlist
area = get\_vm\_area\_caller(size, VM_IOREMAP, caller);
.......
}
可以看到在ioremap映射的过程中,会检查要映射的物理地址是否是System RAM,如果是则不会映射。
struct vm\_struct \*get\_vm\_area\_caller(unsigned long size, unsigned long flags,
const void \*caller)
{
return \_\_get\_vm\_area\_node(size, 1, flags, VMALLOC_START, VMALLOC_END,
NUMA_NO_NODE, GFP_KERNEL, caller);
}
get_vm_area_caller调用__get_vm_area_node函数,接下来流程就与vmalloc的实现基本一致了,请参考:Linux vmalloc原理与实现,找到一个vmalloc子区域,建立vmalloc子区域与物理地址的四级页表映射:PGD -> PUD -> PMD -> PTE。
几乎每一种外设都是通过读写设备上的相关寄存器来进行的,通常包括控制寄存器、状态寄存器和数据寄存器三大类,外设的寄存器通常被连续地编址。
ioremap的返回值可以传递给一组访问器函数(名称如 readb() 或 writel()),以实际将数据移入或移出 I/O 内存。 在某些体系结构(尤其是 x86)上,I/O 内存真正映射到内核的内存空间(内核虚拟地址空间),因此这些访问器函数变成了直接的指针解引用。
使用ioremap映射完成后,内核空间可以使用ioremap返回的地址对I/O memory进行读写。
从 ioremap 返回的地址不应当直接解引用; 相反, 应当使用内核提供的存取函数。ioremap_nocache 执行平台特定的操作序列,以使总线内存 CPU 可通过 readb/readw/readl/writeb/writew/writel 函数访问。返回的地址不保证可直接用作虚拟地址。
简单点说就是不应该直接使用通过ioremap返回的内核虚拟地址,而是通过辅助函数 readb/readw/readl/writeb/writew/writel 等来进行访问ioremap返回的内核虚拟地址。
尽管在 x86 上解引用一个指针能工作, 不使用正确的辅助函数(宏定义)不利于驱动的移植性和可读性。
依赖计算机平台和使用的总线, I/O 内存可以或者不可以通过页表来存取. 当通过页表存取, 内核必须首先安排从你的驱动可见的物理地址, 并且这常常意味着你在做任何 I/O 之前必须调用 ioremap 。
备注: __iomem 注释,用于标记指向 I/O 内存的指针。 这些注释的工作方式与 __user 标记非常相似,只是它们引用了不同的地址空间。 与 __user 一样,__iomem 标记在内核代码中充当文档角色; 它被编译器忽略。
ioremap不能映射System RAM。
1.3 mmap
简单介绍下mmap:
mmap映射一个设备意味着关联一些用户空间地址到设备内存. 无论何时程序在给定范围内读或写, 它实际上是在存取设备。 为实现 mmap, 驱动只要建立合适的页表给这个地址范围。通过调用 remap_pfn_range来建立页表。
struct file\_operations {
int (\*mmap) (struct file \*, struct vm\_area\_struct \*);
};
vma 包含关于用来存取设备的虚拟地址范围的信息,指定了用户地址空间内连续区间的一个独立内存范围。
// /include/linux/mm\_types.h
/\*
\* This struct defines a memory VMM memory area. There is one of these
\* per VM-area/task. A VM area is any part of the process virtual memory
\* space that has a special rule for the page-fault handlers (ie a shared
\* library, the executable area etc).
\*/
struct vm\_area\_struct {
/\* The first cache line has the info for VMA tree walking. \*/
unsigned long vm_start; /\* Our start address within vm\_mm. \*/
unsigned long vm_end; /\* The first byte after our end address
within vm\_mm. \*/
/\* linked list of VM areas per task, sorted by address \*/
struct vm\_area\_struct \*vm_next, \*vm_prev;
struct rb\_node vm_rb;
/\*
\* Largest free memory gap in bytes to the left of this VMA.
\* Either between this VMA and vma->vm\_prev, or between one of the
\* VMAs below us in the VMA rbtree and its ->vm\_prev. This helps
\* get\_unmapped\_area find a free area of the right size.
\*/
unsigned long rb_subtree_gap;
/\* Second cache line starts here. \*/
struct mm\_struct \*vm_mm; /\* The address space we belong to. \*/
pgprot\_t vm_page_prot; /\* Access permissions of this VMA. \*/
unsigned long vm_flags; /\* Flags, see mm.h. \*/
};
remap_pfn_range,建立页表,完成物理地址到用户空间虚拟地址的映射。remap_pfn_range将物理页帧号pfn_start对应的物理内存映射到用户空间的vm->vm_start处,映射长度为该虚拟内存区的长。
// mm/memory.c
/\*\*
\* remap\_pfn\_range - remap kernel memory to userspace
\* @vma: user vma to map to
\* @addr: target user address to start at
\* @pfn: physical address of kernel memory
\* @size: size of map area
\* @prot: page protection flags for this mapping
\*
\* Note: this is only safe if the mm semaphore is held when called.
\*/
int remap\_pfn\_range(struct vm\_area\_struct \*vma, unsigned long addr,
unsigned long pfn, unsigned long size, pgprot\_t prot)
{
pgd\_t \*pgd;
unsigned long next;
unsigned long end = addr + PAGE\_ALIGN(size);
struct mm\_struct \*mm = vma->vm_mm;
int err;
/\*
\* Physically remapped pages are special. Tell the
\* rest of the world about it:
\* VM\_IO tells people not to look at these pages
\* (accesses can have side effects).
\* VM\_PFNMAP tells the core MM that the base pages are just
\* raw PFN mappings, and do not have a "struct page" associated
\* with them.
\* VM\_DONTEXPAND
\* Disable vma merging and expanding with mremap().
\* VM\_DONTDUMP
\* Omit vma from core dump, even when VM\_IO turned off.
\*
\* There's a horrible special case to handle copy-on-write
\* behaviour that some programs depend on. We mark the "original"
\* un-COW'ed pages by matching them up with "vma->vm\_pgoff".
\* See vm\_normal\_page() for details.
\*/
if (is\_cow\_mapping(vma->vm_flags)) {
if (addr != vma->vm_start || end != vma->vm_end)
return -EINVAL;
vma->vm_pgoff = pfn;
}
err = track\_pfn\_remap(vma, &prot, pfn, addr, PAGE\_ALIGN(size));
if (err)
return -EINVAL;
vma->vm_flags |= VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP;
BUG\_ON(addr >= end);
pfn -= addr >> PAGE_SHIFT;
pgd = pgd\_offset(mm, addr);
flush\_cache\_range(vma, addr, end);
do {
next = pgd\_addr\_end(addr, end);
err = remap\_pud\_range(mm, pgd, addr, next,
pfn + (addr >> PAGE_SHIFT), prot);
if (err)
break;
} while (pgd++, addr = next, addr != end);
if (err)
untrack\_pfn(vma, pfn, PAGE\_ALIGN(size));
return err;
}
EXPORT\_SYMBOL(remap_pfn_range);
一个简单在驱动中实现的例子:
my\_mmap\_func(){
......
remap\_pfn\_range(struct vm\_area\_struct \*vma, unsigned long addr,
unsigned long pfn, unsigned long size, pgprot\_t prot);
......
}
struct file\_operations my_dev{
.mmap = my\_mmap\_func(struct file \*, struct vm\_area\_struct \*);
};
用户打开定义的设备,然后调用mmap时,驱动中的file_operations->mmap: my_mmap_func会被调用, my_mmap_func中调用remap_pfn_range完成物理地址到用户虚拟地址空间的映射。
mmap映射一个设备意味着关联一些用户空间地址到设备内存. 无论何时程序在给定范围内读或写, 它实际上是在存取设备。
二、struct resource
Linux设计了一个通用的数据结构resource来描述各种I/O资源(如:I/O端口、外设内存、DMA和IRQ等)。
Linux是以一种倒置的树形结构来管理每一类I/O资源(如:I/O端口、外设内存、DMA和IRQ)的。每一类I/O资源都对应有一颗倒置的资源树,树中的每一个节点都是个resource结构,而树的根结点root则描述了该类资源的整个资源空间。
基于上述这个思想,Linux将基于I/O映射方式的I/O端口和基于内存映射方式的I/O端口资源统称为“I/O区域”(I/O Region)。
// /include/linux/ioport.h
/\*
\* Resources are tree-like, allowing
\* nesting etc..
\*/
struct resource {
resource\_size\_t start;
resource\_size\_t end;
const char \*name;
unsigned long flags;
struct resource \*parent, \*sibling, \*child;
};
/\*
\* IO resources have these defined flags.
\*/
#define IORESOURCE\_BITS 0x000000ff /\* Bus-specific bits \*/
#define IORESOURCE\_TYPE\_BITS 0x00001f00 /\* Resource type \*/
#define IORESOURCE\_IO 0x00000100 /\* PCI/ISA I/O ports \*/
#define IORESOURCE\_MEM 0x00000200
#define IORESOURCE\_REG 0x00000300 /\* Register offsets \*/
#define IORESOURCE\_IRQ 0x00000400
#define IORESOURCE\_DMA 0x00000800
#define IORESOURCE\_BUS 0x00001000
#define IORESOURCE\_PREFETCH 0x00002000 /\* No side effects \*/
#define IORESOURCE\_READONLY 0x00004000
#define IORESOURCE\_CACHEABLE 0x00008000
#define IORESOURCE\_RANGELENGTH 0x00010000
#define IORESOURCE\_SHADOWABLE 0x00020000
#define IORESOURCE\_SIZEALIGN 0x00040000 /\* size indicates alignment \*/
#define IORESOURCE\_STARTALIGN 0x00080000 /\* start field is alignment \*/
#define IORESOURCE\_MEM\_64 0x00100000
#define IORESOURCE\_WINDOW 0x00200000 /\* forwarded by bridge \*/
#define IORESOURCE\_MUXED 0x00400000 /\* Resource is software muxed \*/
#define IORESOURCE\_EXCLUSIVE 0x08000000 /\* Userland may not map this resource \*/
#define IORESOURCE\_DISABLED 0x10000000
#define IORESOURCE\_UNSET 0x20000000
#define IORESOURCE\_AUTO 0x40000000
#define IORESOURCE\_BUSY 0x80000000 /\* Driver has marked this resource busy \*/
对于flags主要是:
#define IORESOURCE\_BUSY 0x80000000 /\* Driver has marked this resource busy \*/
#define IORESOURCE\_IO 0x00000100 /\* ioports \*/
#define IORESOURCE\_MEM 0x00000200 /\* iomem \*/
三、System RAM
3.1 System RAM 简介
在 /proc/iomem 的输出中,所有 RAM 范围都被命名为“系统 RAM”。
System RAM:DDR物理内存,内存条。
System RAM 不一定位于物理地址空间的开头,也不总是在一个连续的块中。 为了确定物理地址空间的哪些部分是System RAM(相对于内存映射 I/O),我们通过 /proc/iomem来查找。
cat /proc/iomem | grep "System RAM"
3.2 page_is_ram
该函数功能:给定的页框号是否属于物理内存,主要是在iomem_resource 这颗资源树上查找名为"System Ram" 的资源,如果包含在其中的话,就说明该页框号属于物理内存。
如果指定地址在 iomem_resource 列表中注册为“系统 RAM”,则此通用 page_is_ram() 返回 true。
// /kernel/resource.c
/\*
\* This function calls callback against all memory range of "System RAM"
\* which are marked as IORESOURCE\_MEM and IORESOUCE\_BUSY.
\* Now, this function is only for "System RAM".
\*/
int walk\_system\_ram\_range(unsigned long start_pfn, unsigned long nr_pages,
void \*arg, int (\*func)(unsigned long, unsigned long, void \*))
{
struct resource res;
unsigned long pfn, end_pfn;
u64 orig_end;
int ret = -1;
res.start = (u64) start_pfn << PAGE_SHIFT;
res.end = ((u64)(start_pfn + nr_pages) << PAGE_SHIFT) - 1;
res.flags = IORESOURCE_MEM | IORESOURCE_BUSY;
orig_end = res.end;
while ((res.start < res.end) &&
(find\_next\_system\_ram(&res, "System RAM") >= 0)) {
pfn = (res.start + PAGE_SIZE - 1) >> PAGE_SHIFT;
end_pfn = (res.end + 1) >> PAGE_SHIFT;
if (end_pfn > pfn)
ret = (\*func)(pfn, end_pfn - pfn, arg);
if (ret)
break;
res.start = res.end + 1;
res.end = orig_end;
}
return ret;
}
/\*
\* This generic page\_is\_ram() returns true if specified address is
\* registered as "System RAM" in iomem\_resource list.
\*/
int __weak page\_is\_ram(unsigned long pfn)
{
return walk\_system\_ram\_range(pfn, 1, NULL, __is_ram) == 1;
}
3.3 Kernel code、data、bss
系统会使用System RAM其中的一部分自用,放置code、data、bss和crash kernel。这部分物理内存已经被使用,系统不会使用这部分物理内存用来别的用途。
cat /proc/iomem | grep -i kernel
// /arch/x86/kernel/setup.c
/\*
\* Machine setup..
\*/
static struct resource data_resource = {
.name = "Kernel data",
.start = 0,
.end = 0,
.flags = IORESOURCE_BUSY | IORESOURCE_MEM
};
static struct resource code_resource = {
.name = "Kernel code",
.start = 0,
.end = 0,
.flags = IORESOURCE_BUSY | IORESOURCE_MEM
};
static struct resource bss_resource = {
.name = "Kernel bss",
.start = 0,
.end = 0,
.flags = IORESOURCE_BUSY | IORESOURCE_MEM
};
// /include/linux/sched.h
extern struct mm\_struct init_mm;
// /arch/x86/kernel/setup.c
void __init setup\_arch(char \*\*cmdline_p){
......
init_mm.start_code = (unsigned long) _text;
init_mm.end_code = (unsigned long) _etext;
init_mm.end_data = (unsigned long) _edata;
init_mm.brk = _brk_end;
code_resource.start = \_\_pa\_symbol(_text);
code_resource.end = \_\_pa\_symbol(_etext)-1;
data_resource.start = \_\_pa\_symbol(_etext);
data_resource.end = \_\_pa\_symbol(_edata)-1;
bss_resource.start = \_\_pa\_symbol(__bss_start);
bss_resource.end = \_\_pa\_symbol(__bss_stop)-1;
......
/\* after parse\_early\_param, so could debug it \*/
insert\_resource(&iomem_resource, &code_resource);
insert\_resource(&iomem_resource, &data_resource);
insert\_resource(&iomem_resource, &bss_resource);
......
}
网上学习资料一大堆,但如果学到的知识不成体系,遇到问题时只是浅尝辄止,不再深入研究,那么很难做到真正的技术提升。
一个人可以走的很快,但一群人才能走的更远!不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人,都欢迎加入我们的的圈子(技术交流、学习资源、职场吐槽、大厂内推、面试辅导),让我们一起学习成长!
ata)-1;
bss_resource.start = __pa_symbol(__bss_start);
bss_resource.end = __pa_symbol(__bss_stop)-1;
......
/\* after parse\_early\_param, so could debug it \*/
insert\_resource(&iomem_resource, &code_resource);
insert\_resource(&iomem_resource, &data_resource);
insert\_resource(&iomem_resource, &bss_resource);
......
}
**网上学习资料一大堆,但如果学到的知识不成体系,遇到问题时只是浅尝辄止,不再深入研究,那么很难做到真正的技术提升。**
**[需要这份系统化的资料的朋友,可以点击这里获取!](https://bbs.csdn.net/topics/618542503)**
**一个人可以走的很快,但一群人才能走的更远!不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人,都欢迎加入我们的的圈子(技术交流、学习资源、职场吐槽、大厂内推、面试辅导),让我们一起学习成长!**