Jeff Cooper

Kernel | Part 1: Let There Be Light

06 Jun 2014

Disclaimer

If you're currently taking a course on Operating Systems, or if you plan to in the future, I actually recommend that you don't read this series of articles. A lot of this stuff is best learned by trial and error, and more of it still is much more magical to discover on your own.

The Bare Minimum

My first task was to create the smallest possible chunk of code that could boot, do something, and not crash. As it turns out, that isn't too much. In fact, the actual code part is totally trivial. We call it boot/main.c, and it looks like this:

void kernel_main()
{
  while(1);
}

The hard part, of course, is getting this code to boot.

I Cheated a Little

Remember when I said I wanted the smallest possible thing that I could boot? Turns out that even that was something of an overstatement of the work I wanted to do. Actually booting a machine running the code would require making a bootable disk image of some kind (OSDev.org has a great walkthrough). QEMU, however, is capable of booting a kernel image straight from the disk! Since that was less effort, I decided to do that. Once it's worth booting on real hardware, we'll talk about making a disk image with a bootloader.

Some Code

So what does it take to boot something? The article that got me started, and which basically mirrors this entire post, was this one, by Arjun Sreedharan. As he explains, there's a standard called "multiboot" for booting x86 kernels. Somewhere in the first 8KB of the kernel image, there needs to exist three 32-bit values:

  • A magic number, 0x1BADB002. This signals the start of the multiboot header.
  • Some flags. We don't care about any of the magic that multiboot can do (for now), so this should just be 0x00.
  • A checksum, such that MAGIC + FLAGS + CHECKSUM = 0.

(note: Some of the flags (which, again, we don't care about right now) can require more information after the checksum)

So let's put this in code! I called this file boot/entry.S

    .section .multiboot
    multiboot_header:
    .align 4

    .long 0x1BADB002  /* magic number */
    .long 0x00        /* flags */
    .long - (0x1BADB002 + 0x00)  /* checksum */

Linking

Now, let's stop and take a breath. On one hand, one of the coolest things about low-level development is that you can't take much for granted. On the other hand, one of the most frustrating things about low-level development is that you can't take much for granted. For example: when you're writing regular code, you typically just compile it into an executable without thinking. Maybe you call gcc by hand, maybe you write a Makefile and then just call make. Maybe you just hit the "compile" button in your favorite IDE. With kernel-level code, though, we have to think a bit about the build process before we can hide it all behind our fancy tools.

If you've done systems-level code before or if you've travelled to the north pole, you may have heard of ELF. The specific layout of an ELF file isn't important (yet), but you do need to know a few things about what's in these files. Put simply, ELF files hold binary stuff (like your compiled code, static data, debugging information, etc) separated into sections. Some sections of note:

  • .text : This is your code, as x86 assembly.
  • .data : Variables go here, for the most part.
  • .rodata : Read-only data, like constant strings, embedded bitmaps, etc
  • .bss : The "better save space" section -- this doesn't take up any space in the ELF file, but specifies how much space will be needed for yet-to-be-initialized variables.

At a high level, the process of turning your source code into an ELF file involves compiling it (turning C into assembly) and then linking it (separating that assembly into sections, and resolving symbol names like "main" into memory addresses). The compilation step can be handled normally (I'll be using gcc), but the linking step takes a little bit of extra thought.

Because we need to make sure the multiboot header appears in the first 8K of the blob we create, we actually need to specify where the different sections appear. You might have also noticed the line .section .multiboot in the code above: that line says "the following goes in the .multiboot section until I say otherwise." It's worth noting that .multiboot isn't a standard ELF section. However, we can combine this with the following linker script, which I called linker/link.ld

OUTPUT_FORMAT(elf32-i386)
ENTRY(x86_entry)
SECTIONS
{
  . = 0x100000;
  .text : { *(.multiboot) *(.text) }
  .data : { *(.data) }
  .bss  : { *(.bss)  }
}

Linker scripts are described in great detail elsewhere on the internet, but the simple explanation of this script is:

  • OUTPUT_FORMAT(elf32-i386) simply says "make a 32-bit ELF file for x86." The GNU linker can do lots of other types as well.
  • ENTRY(x86_entry) says that a symbol called x86_entry will be the entrypoint for the kernel. Remember that, we'll come back to it in just a second.
  • The SECTIONS block describes what sections should go in the file, in what order.
    • First, we say . = 0x100000 to set the "location counter" to address 0x100000. If you think about writing a book, this would be like saying "go to page 20, then keep writing." We move the counter this far just to make sure there's plenty of space for the ELF header.
    • Next, we create a section called .text. In it, we first include the .multiboot section from every file we're linking together, then we include the .text section from every file we're linking together.
    • The next two lines aren't strictly necessary. They just tell the linker to create sections called .data and .bss which contain the corresponding sections from the input files being linked. This is the behavior that we want, but it's also the behavior that happens even if we don't specify these sections specifically.

Makefile

At this point, let's put together a Makefile. If you've never used Makefiles before, the idea is this: write a description of how to build your files, write down which files depend on which other files, and then just type make to build everything. Makefiles are a very powerful tool and can do much more than we'll use them for here, and I don't claim to be an expert in Makefiles. With that said, the makefile I'm using looks like this:

    CC=gcc
    CFLAGS= -std=c99 -m32 -Wall -Werror -pedantic -g -I.
    ASM=as
    ASMFLAGS= --32
    LD=ld
    LDFLAGS= -m elf_i386 -T linker/link.ld

    kernel_name=kernel

    ASM_SOURCES= $(shell find . -type f -name '*.S')
    C_SOURCES= $(shell find . -type f -name '*.c')

    OBJECTS= $(ASM_SOURCES:.S=.o)
    OBJECTS+=$(C_SOURCES:.c=.o)

    .PHONY: all clean run debug doc

    all: $(kernel_name)

    clean:
            rm $(OBJECTS) $(kernel_name)

    run: $(kernel_name)
            qemu-system-i386 -kernel $(kernel_name)

    debug: $(kernel_name)
            qemu-system-i386 -s -S -kernel $(kernel_name)

    doc:
            doxygen

    $(kernel_name) : $(OBJECTS)
            echo $(OBJECTS)
            $(LD) $(LDFLAGS) -o $@  $(OBJECTS)

    %.o : %.S
            $(ASM) $(ASMFLAGS)  $*.S -o $@

    %.o : %.c
            $(CC) $(CFLAGS) -c $*.c -o $@

You should note that, in Makefiles, the indents must be tab characters (\t), not spaces. If you want to download my Makefile, you can get it on github

Don't worry too much about the makefile, but you should know the following commands:

  • make (with no arguments) builds the kernel. Right now, it generates a file called kernel. You can also use make all.
  • make clean deletes all the object files (compiled versions of individual source files), and the kernel file.
  • make run runs the kernel in QEMU.
  • make debug runs the kernel in QEMU, but waits for a debugger to connect on port 1234. You can check out the documentation for QEMU and GDB for more information on this, or I'll cover it in a later post.
  • make doc just runs Doxygen. Documentation is good.

Code!

Now, after all of that, we can actually start writing some code that will, you know, run.

First, remember when we said that there was a symbol called x86_entry, which would serve as the entrypoint for our kernel. Now would be a good time to write that function. Because we're eventually going to want to do some fine-grain memory management which can't be done from C, I wrote this function in assembly. Ready?

  .global x86_entry
  .extern kernel_main

  .section .text
  x86_entry:
    cli
    call kernel_main
    hlt

First, we declare that x86_entry is a symbol (in this case, a function) which something outside of this file will care about. Then, we say "there's a symbol somewhere else called kernel_main that we're going to reference." Finally, we create x86_entry in the .text section with three instructions. First, we disable interrupts with cli (we'll talk about interrupts in the next article). Then, we call a function called kernel_main. When we get back from that call, we halt the CPU. This is ever so slightly misleading, since the kernel won't ever return, but we want to do something reasonable if for some reason we ever do. kernel_main, you might recall, is the C function that we wrote way up at the beginning of this post.

Hello World? Nope.

If you've made it this far, you should be able to type make (assuming you have a GCC toolchain installed) and have a bootable kernel! And if we boot it, it looks like this: (yours may vary)

You know why it looks like that? Because our kernel does nothing. No "hello world," no "program returned successfully," just a blinking cursor. However, the fact that the cursor continues to blink, and that the emulator doesn't just crash and burn, is a testament to the fact that our kernel is running. Even if it does nothing.

Where are we?

So the post is over, and we've spent several pages getting to the point where our kernel does nothing. However, the fact that it actually does the nothing, and that it doesn't fail to do that nothing, means that we're on the right track.

The next post will actually get us to the ever-pervasive "hello world" when we write a driver and basic library for console output.

Mistakes? Questions? Comments?

I'm not an expert in any of this by any means. If I made a mistake, or I wasn't clear about something, or if you just want to leave a comment, do so below or let me know on Facebook or Twitter.

comments powered by Disqus