Linux 实现自定义系统调用，支持参数和结果返回

本文实现一个简单的系统调用实现，支持输入字符串参数，并返回一个结果字符串。
以下是验证步骤：

1. 添加系统调用编号

试验使用的是 x86_64 架构的 Linux 内核。
找到并编辑 arch/x86/entry/syscalls/syscall_64.tbl 文件，在文件末尾添加新的系统调用号和函数映射，这里使用 548 作为新系统调用编号：

548  64  say_hello   sys_say_hello

2. 添加系统调用实现

在合适的内核源码文件中（如 kernel/sys.c ），末尾添加系统调用函数实现：

SYSCALL_DEFINE3(say_hello, const char __user *, name, char __user *, result, size_t, result_len)
{
    char kname[256];  // 假设最大长度为256
    char msg[512];
    size_t msg_len;

    // 从用户态复制参数
    if (copy_from_user(kname, name, sizeof(kname))) {
        return -EFAULT;
    }

    // 构建返回消息
    snprintf(msg, sizeof(msg), "Welcome %s", kname);
    msg_len = strnlen(msg, sizeof(msg)) + 1;

    // 检查结果缓冲区长度是否足够
    if (result_len < msg_len) {
        return -ENOSPC;
    }

    // 将结果复制到用户态缓冲区
    if (copy_to_user(result, msg, msg_len)) {
        return -EFAULT;
    }

    return 0;
}

SYSCALL_DEFINE3 宏将自动处理系统调用的关联注册、参数传递处理等，无需其它操作。

3. 编译并安装内核

编译环境准备参见：https://blog.csdn.net/shida_csdn/article/details/139780103

make INSTALL_MOD_STRIP=1 binrpm-pkg -j 8
cd /root/rpmbuild/RPMS/x86_64
yum localinstall *
reboot

重启后，选择新内核

4. 验证系统调用

编写用户态测试程序 say_hello.c ：

#include <stdio.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <string.h>

int main() {
    char name[] = "Alice";
    char result[120];

    long res = syscall(548, name, result, sizeof(result));
    if (res == 0) {
        printf("%s\n", result);
    } else {
        printf("System call failed with error code: %ld\n", res);
    }

    return 0;
}

编译并运行：

gcc -o say_hello say_hello.c
./say_hello

# 会看到输出如下内容，证明系统调用成功
Welcome Alice

x86_64 的系统调用的参数传递使用的寄存器，可以在源码 arch/x86/entry/entry_64.S 中找到，最多支持 6 个参数：

* 64-bit SYSCALL instruction entry. Up to 6 arguments in registers.
 *
 * This is the only entry point used for 64-bit system calls.  The
 * hardware interface is reasonably well designed and the register to
 * argument mapping Linux uses fits well with the registers that are
 * available when SYSCALL is used.
 *
 * SYSCALL instructions can be found inlined in libc implementations as
 * well as some other programs and libraries.  There are also a handful
 * of SYSCALL instructions in the vDSO used, for example, as a
 * clock_gettimeofday fallback.
 *
 * 64-bit SYSCALL saves rip to rcx, clears rflags.RF, then saves rflags to r11,
 * then loads new ss, cs, and rip from previously programmed MSRs.
 * rflags gets masked by a value from another MSR (so CLD and CLAC
 * are not needed). SYSCALL does not save anything on the stack
 * and does not change rsp.
 *
 * Registers on entry:
 * rax  system call number
 * rcx  return address
 * r11  saved rflags (note: r11 is callee-clobbered register in C ABI)
 * rdi  arg0
 * rsi  arg1
 * rdx  arg2
 * r10  arg3 (needs to be moved to rcx to conform to C ABI)
 * r8   arg4
 * r9   arg5
 * (note: r12-r15, rbp, rbx are callee-preserved in C ABI)
 *
 * Only called from user space.