linux内核调试

环境

编译Busybox系统环境

macOS下面编译Busybox有点问题,换成同架构的Linux系统进行编译

创建文件系统

下载BusyBox源码并编译:

1
2
3
4
5
sudo dnf install make gcc -y
wget https://busybox.net/downloads/busybox-1.37.0.tar.bz2
tar -xvf busybox-1.37.0.tar.bz2
cd busybox-1.37.0
make menuconfig

常见错误:

1
2
3
4
5
6
7
8
# fix: fatal error: curses.h: No such file or directory
sudo dnf install ncurses-devel -y
sudo dnf install ncurses -y

# fix: Install ncurses (ncurses-devel) and try again
vim scripts/kconfig/lxdialog/Makefile
# hash out this line
always := $(hostprogs-y) dochecklxdialog

勾选Build static binary (no shared libs)

静态链接

创建磁盘镜像

1
2
3
4
5
dd if=/dev/zero of=rootfs.img bs=1M count=2048
sudo dnf install e2fsprogs -y # mkfs.ext4
mkfs.ext4 rootfs.img
mkdir root
sudo mount -t ext4 -o loop rootfs.img ./root

常见错误:

1
libbb/hash_md5_sha.c:1316:35: error: ‘sha1_process_block64_shaNI’ undeclared (first use in this function); did you mean ‘sha1_process_block64’?

使用vim编辑替换符号:%s#sha1_process_block64_shaNI#sha1_process_block64#g

1
networking/tc.c:236:27: error: ‘TCA_CBQ_MAX’ undeclared (first use in this function); did you mean ‘TCA_CBS_MAX’?

删除networking/tc.c文件,见Bug

1
2
3
4
/usr/sbin/ld: cannot find -lm: No such file or directory
/usr/sbin/ld: have you installed the static version of the m library ?
/usr/sbin/ld: cannot find -lresolv: No such file or directory
/usr/sbin/ld: have you installed the static version of the resolv library ?

安装glibc-staticlibgcc:

1
2
sudo dnf install glibc-static -y
sudo dnf install libgcc -y

拷贝BusyBox到根文件系统

1
sudo make install CONFIG_PREFIX=./root

修改根文件系统

1
2
3
4
5
6
cd root
sudo mkdir proc dev etc home mnt
sudo cp -r ../examples/bootfloppy/etc/* etc/
sudo chmod -R 777 .
cd ..
sudo umount ./root

启动内核

内核文件和文件系统

使用QEMU启动内核和文件系统:

1
brew install qemu

启动命令

1
qemu-system-aarch64 -smp 4 -M virt,accel=hvf -cpu host -nographic -m 2048M -kernel linux-6.7.9/arch/arm64/boot/Image -hda busybox-1.37.0/rootfs.img -append "root=/dev/vda rw console=ttyAMA0" -s

内核启动成功

关闭内核地址随机化

关闭内核地址随机化

其他问题:

1
qemu-system-aarch64: Error: r = HV_BAD_ARGUMENT (0xfae94003, at ../target/arm/hvf/hvf.c:2234)

解决办法:
问题已修复,需要更新QEMU版本。

预编译二进制没有发版,可以源码编译QEMU

1
2
3
4
5
git clone https://gitlab.com/qemu-project/qemu.git
cd qemu
./configure --target-list=aarch64-softmmu,riscv64-softmmu
make -j$(nproc)
codesign -d --entitlements accel/hvf/entitlements.plist --force -s - ./build/qemu-system-aarch64-unsigned
1
/Volumes/Work/qemu/build/qemu-system-aarch64-unsigned -smp 4 -M virt,accel=hvf -cpu host -nographic -m 2048M -kernel linux-6.7.9/arch/arm64/boot/Image -hda busybox-1.37.0/rootfs.img -append "root=/dev/vda rw console=ttyAMA0" -s

Xcode调试

打开Xcode自定义LLDB命令支持

1
defaults write com.apple.dt.Xcode IDEDebuggerFeatureSetting 12

启用自定义lldb命令

使用下面的LLDB命令连接QEMU调试:

1
2
target create $PROJECT_DIR/vmlinux
gdb-remote localhost:1234

调试配置

调试信息

生成Xcode项目

把下面的内容保存为project.yml文件,然后使用xcodegen生成Xcode项目:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
name: Linux
options:
bundleIdPrefix: top.ourfor.opensource
settings:
HEADER_SEARCH_PATHS: $SRCROOT/**
targets:
Kernel:
type: tool
platform: macOS
deploymentTarget: 10.15
preBuildScripts:
- script: export PATH="/opt/homebrew/bin:$PATH" && lkmake ARCH=arm64 LLVM=1 -j $(nproc)
name: Makefile Build
postCompileScripts:
- script: cp $PROJECT_DIR/vmlinux $BUILT_PRODUCTS_DIR/$EXECUTABLE_NAME
name: Copy Executable
- script: |
export PATH="/opt/homebrew/bin:/Volumes/Work/qemu/build:$PATH"
cd /Volumes/Work
# 检查并关闭已存在的 qemu 会话
tmux has-session -t qemu 2>/dev/null && tmux kill-session -t qemu
# 在新的 tmux 会话中启动 QEMU
tmux new-session -d -s qemu '/Volumes/Work/qemu/build/qemu-system-aarch64-unsigned \
-smp 4 \
-M virt,accel=hvf \
-cpu host \
-nographic \
-m 2048M \
-kernel linux-6.7.9/arch/arm64/boot/Image \
-hda busybox-1.37.0/rootfs.img \
-append "root=/dev/vda rw console=ttyAMA0" \
-s'
sleep 1
echo "QEMU started in tmux session 'qemu'"
echo "Use 'tmux attach -t qemu' to connect to QEMU console"
name: Launch QEMU
runOnlyWhenInstalling: false
Dummy:
type: tool
platform: macOS
deploymentTarget: 10.15
sources:
- path: arch
includes: ["**/*.[chsS]"]
- path: block
includes: ["**/*.[chsS]"]
- path: crypto
includes: ["**/*.[chsS]"]
- path: drivers
includes: ["**/*.[chsS]"]
- path: fs
includes: ["**/*.[chsS]"]
- path: include
includes: ["**/*.[chsS]"]
- path: init
includes: ["**/*.[chsS]"]
- path: io_uring
includes: ["**/*.[chsS]"]
- path: ipc
includes: ["**/*.[chsS]"]
- path: kernel
includes: ["**/*.[chsS]"]
- path: lib
includes: ["**/*.[chsS]"]
- path: mm
includes: ["**/*.[chsS]"]
- path: net
includes: ["**/*.[chsS]"]
- path: security
includes: ["**/*.[chsS]"]
- path: sound
includes: ["**/*.[chsS]"]
- path: virt
includes: ["**/*.[chsS]"]
schemes:
Linux:
build:
targets:
Kernel: [run]
run:
debugEnabled: true
executable: none
customLLDBInitFile: |
target create $PROJECT_DIR/vmlinux
gdb-remote localhost:1234

修改源码并调试

修改后编译,打开终端,执行tmux a -t qemu查看Linux系统输出

修改源码并调试

查看局部变量

查看局部变量