Lab: system calls

In the last lab you used system calls to write a few utilities. In this lab you will add some new system calls to xv6, which will help you understand how they work and will expose you to some of the internals of the xv6 kernel. You will add more system calls in later labs.

Before you start coding, read Chapter 2 of the xv6 book, and Sections 4.3 and 4.4 of Chapter 4, and related source files:

• The user-space "stubs" that route system calls into the kernel are in user/usys.S, which is generated by user/usys.pl when you run make. Declarations are in user/user.h

• The kernel-space code that routes a system call to the kernel function that implements it is in kernel/syscall.c and kernel/syscall.h.

• Process-related code is kernel/proc.h and kernel/proc.c.

To start the lab, switch to the syscall branch:

$ git fetch
$ git checkout syscall
$ make clean

If you run make grade you will see that the grading script cannot exec trace. Your job is to add the necessary system calls and stubs to make trace work. Furthermore, you will notice attacktest fails.

Using gdb

In many cases, print statements will be sufficient to debug your kernel, but sometimes it is useful to single step through code or get a stack back-trace. The GDB debugger can help.

To help you become familiar with gdb, run make qemu-gdb and then fire up gdb in another window (see the gdb material on the guidance page). Once you have two windows open, type in the gdb window:

(gdb) b syscall
Breakpoint 1 at 0x80002142: file kernel/syscall.c, line 243.
(gdb) c
Continuing.
[Switching to Thread 1.2]

Thread 2 hit Breakpoint 1, syscall () at kernel/syscall.c:243
243     {
(gdb) layout src
(gdb) backtrace

The layout command splits the window in two, showing where gdb is in the source code. backtrace prints a stack backtrace.

Answer the following questions in answers-syscall.txt.

Looking at the backtrace output, which function called syscall?

Type n a few times to step past struct proc *p = myproc(); Once past this statement, type p /x *p, which prints the current process's proc struct (see kernel/proc.h) in hex.

What is the value of p->trapframe->a7 and what does that value represent? (Hint: look user/initcode.S, the first user program xv6 starts.)

The processor is running in supervisor mode, and we can print privileged registers such as sstatus (see RISC-V privileged instructions for a description):

(gdb) p /x $sstatus

What was the previous mode that the CPU was in?

The xv6 kernel code contains consistency checks whose failure causes the kernel to panic; you may find that your kernel modifications cause panics. For example, replace the statement num = p->trapframe->a7; with num = * (int *) 0; at the beginning of syscall, run make qemu, and you will see something similar to:

xv6 kernel is booting

hart 2 starting
hart 1 starting
scause=0xd sepc=0x80001bfe stval=0x0
panic: kerneltrap

Quit out of qemu.

To track down the source of a kernel page-fault panic, search for the sepc value printed for the panic you just saw in the file kernel/kernel.asm, which contains the assembly for the compiled kernel.

Write down the assembly instruction the kernel is panicing at. Which register corresponds to the variable num?

To inspect the state of the processor and the kernel at the faulting instruction, fire up gdb, and set a breakpoint at the faulting epc, like this:

(gdb) b *0x80001bfe
Breakpoint 1 at 0x80001bfe: file kernel/syscall.c, line 138.
(gdb) layout asm
(gdb) c
Continuing.
[Switching to Thread 1.3]

Thread 3 hit Breakpoint 1, syscall () at kernel/syscall.c:138

Confirm that the faulting assembly instruction is the same as the one you found above.

Why does the kernel crash? Hint: look at figure 3-3 in the text; is address 0 mapped in the kernel address space? Is that confirmed by the value in scause above? (See description of scause in RISC-V privileged instructions)

Note that scause was printed by the kernel panic above, but often you need to look at additional info to track down the problem that caused the panic. For example, to find out which user process was running when the kernel paniced, you can print the process's name:

(gdb) p p->name

What is the name of the process that was running when the kernel paniced? What is its process id (pid)?

You may want to revisit Using the GNU Debugger as needed. The guidance page also has debugging tips.

System call tracing

In this assignment you will add a system call tracing feature that may help you when debugging later labs. You'll create a new trace system call that will control tracing. It should take one argument, an integer "mask", whose bits specify which system calls to trace. For example, to trace the fork system call, a program calls trace(1 << SYS_fork), where SYS_fork is a syscall number from kernel/syscall.h. You have to modify the xv6 kernel to print a line when each system call is about to return, if the system call's number is set in the mask. The line should contain the process id, the name of the system call and the return value; you don't need to print the system call arguments. The trace system call should enable tracing for the process that calls it and any children that it subsequently forks, but should not affect other processes.

We provide a trace user-level program that runs another program with tracing enabled (see user/trace.c). When you're done, you should see output like this:

$ trace 32 grep hello README
3: syscall read -> 1023
3: syscall read -> 966
3: syscall read -> 70
3: syscall read -> 0
$
$ trace 2147483647 grep hello README
4: syscall trace -> 0
4: syscall exec -> 3
4: syscall open -> 3
4: syscall read -> 1023
4: syscall read -> 966
4: syscall read -> 70
4: syscall read -> 0
4: syscall close -> 0
$
$ grep hello README
$
$ trace 2 usertests forkforkfork
usertests starting
test forkforkfork: 407: syscall fork -> 408
408: syscall fork -> 409
409: syscall fork -> 410
410: syscall fork -> 411
409: syscall fork -> 412
410: syscall fork -> 413
409: syscall fork -> 414
411: syscall fork -> 415
...
$

In the first example above, trace invokes grep tracing just the read system call. The 32 is 1<<SYS_read. In the second example, trace runs grep while tracing all system calls; the 2147483647 has all 31 low bits set. In the third example, the program isn't traced, so no trace output is printed. In the fourth example, the fork system calls of all the descendants of the forkforkfork test in usertests are being traced. Your solution is correct if your program behaves as shown above (though the process IDs may be different).

Some hints:

• Add $U/_trace to UPROGS in Makefile

• Run make qemu and you will see that the compiler cannot compile user/trace.c, because the user-space stubs for the trace system call don't exist yet: add a prototype for trace to user/user.h, a stub to user/usys.pl, and a syscall number to kernel/syscall.h. The Makefile invokes the perl script user/usys.pl, which produces user/usys.S, the actual system call stubs, which use the RISC-V ecall instruction to transition to the kernel. Once you fix the compilation issues, run trace 32 grep hello README; it will fail because you haven't implemented the system call in the kernel yet.

• Add a sys_trace() function in kernel/sysproc.c that implements the new system call by remembering its argument in a new variable in the proc structure (see kernel/proc.h). The functions to retrieve system call arguments from user space are in kernel/syscall.c, and you can see examples of their use in kernel/sysproc.c. Add your new sys_trace to the syscalls array in kernel/syscall.c.

• Modify fork() (see kernel/proc.c) to copy the trace mask from the parent to the child process.

• Modify the syscall() function in kernel/syscall.c to print the trace output. You will need to add an array of syscall names to index into.

Attack xv6

The xv6 kernel isolates user programs from each other and isolates the kernel from user programs. As you saw in the above assignments, an application cannot directly call a function in the kernel or in another user program; instead, interactions occur only through system calls. However, if there is a bug in the implementation of a system call, an attacker may be able to exploit that bug to break the isolation boundaries. To get a sense for how bugs can be exploited, we have introduced a bug into xv6 and your goal is to exploit that bug to trick xv6 into revealing a secret from another process.

The bug is that the call to memset(mem, 0, sz) at line 272 in kernel/vm.c to clear a newly-allocated page is omitted when compiling this lab. Similarly, when compiling kernel/kalloc.c for this lab the two lines that use memset to put garbage into free pages are omitted. The net effect of omitting these 3 lines (all marked by ifndef LAB_SYSCALL) is that newly allocated memory retains the contents from its previous use.

user/secret.c writes an 8-byte secret in its memory and then exits (which frees its memory). Your goal is to add a few lines of code to user/attack.c to find the secret that a previous execution of secret.c wrote to memory, and write the 8 secret bytes to file descriptor 2. You'll receive full credit if attacktest prints: "OK: secret is ebb.ebb". (Note: the secret may be different for each run of attacktest.)

You are allowed to modify user/attack.c, but you cannot make any other changes: you cannot modify the xv6 kernel sources, secret.c, attacktest.c, etc.

Some hints:

• Run attacktest in the xv6 shell. It should the following output:

  $ attacktest
  FAIL: no/incorrect secret

  Note that despite the 3 deleted lines, xv6 appears to work correctly: it started the shell and it ran attacktest. In fact, if you run usertests most of them pass!

• Read user/attacktest.c. It generates a random 8-byte string, which it passes to the program secret, which writes it into its memory. After secret exits, attacktest spawns attack and waits for attack to write the secret string to file descriptor 2.

• Read user/secret.c and think about how you could trick xv6 into revealing the secret to attack.c.

• Test your exploit by running attacktest in the xv6 shell.

user/secret.c copies the secret bytes to memory whose address is 32 bytes after the start of a page. Change the 32 to 0 and you should see that your attack doesn't work anymore; why not?

Small bugs that do not directly affect correctness but still can be exploited to break security (like the one above) make kernel programming challenging. xv6 is likely to have such bugs, although we try to not have them. Real kernels, which have many more lines of code than xv6, have a long history of such bugs. For example, see the public Linux vulnerabilities and how to report vulnerabilities.

Submit the lab

Time spent

Create a new file, time.txt, and put in a single integer, the number of hours you spent on the lab. git add and git commit the file.

Answers

If this lab had questions, write up your answers in answers-*.txt. git add and git commit these files.

Submit

Assignment submissions are handled by Gradescope. You will need an MIT gradescope account. See Piazza for the entry code to join the class. Use this link if you need more help joining.

When you're ready to submit, run make zipball, which will generate lab.zip. Upload this zip file to the corresponding Gradescope assignment.

If you run make zipball and you have either uncomitted changes or untracked files, you will see output similar to the following:

 M hello.c
?? bar.c
?? foo.pyc
Untracked files will not be handed in.  Continue? [y/N]

Inspect the above lines and make sure all files that your lab solution needs are tracked, i.e., not listed in a line that begins with ??. You can cause git to track a new file that you create using git add {filename}.

Warning

• Please run make grade to ensure that your code passes all of the tests. The Gradescope autograder will use the same grading program to assign your submission a grade.
• Commit any modified source code before running make zipball.
• You can inspect the status of your submission and download the submitted code at Gradescope. The Gradescope lab grade is your final lab grade.

Optional challenge exercises

• Print the system call arguments for traced system calls.

• Find a bug in xv6 that allows an adversary to break process isolation or crash the kernel and let us know. (Side channels such as Meltdown are out of scope, although we will cover them in lecture.)