noxos
/
kernel
4
1
Fork 0
Code Issues 5 Pull Requests Packages Projects Releases Wiki Activity
kernel/.wiki/Kernel-documentation.md

633 lines
25 KiB
Markdown
Raw Blame History

# Kernel Documentation
The kernel is booted using the limine boot protocol.
## Directory structure
- **boot** - all stuff related to booting / jumping into the kernel
- **drivers** - everything from the graphics driver, to the FS drivers
- **mm** - memory management stuff like page frames and page maps
- **platform** - universal API to the platform specific code in the subdirs
- **proc** - all the process/thread related stuff like the scheduler
- **utils** - utilities like type definitions, math functions, high-level memory management
---
# General concepts
## Kernel initialization
The single parts of the kernel are initialized in the following order:
- **Global Descriptor Table**
- **Page Frame Manager**
- **Interrupt Descriptor Table**
## Interrupt handling
OSDev Wiki: [Interrupts](https://wiki.osdev.org/Interrupts)
Unfortunatly the x86 architecture doesn't provide a method to get the ID of the current interrupt.
To solve this problem, there is a simple assembly function for every interrupt used by NoxOS.
This function pushes its ID on the stack.
After that it calls a common Interrupt handler, this handler will generate the current `cpu_state_T` and call the C interrupt handler implementation.
The C implementation returns a `cpu_state_T` that will then be loaded.
## Paging
OSDev Wiki: [Paging](https://wiki.osdev.org/Paging)
There is a difference between `Virtual Memory Spaces` and the `Physical Memory Space`.
The Physical memory space is how the data lies directly in the RAM.
Virtual memory spaces are a bit more tricky.
To understand them, we have to understand first, that the physical memory space is divided into so-called **pages** / **page frames**.
These pages have a size of 4KB.
A virtual memory space is a table of page mappings.
Per default there are no pages mapped to such a table.
When the OS maps a page to a **page table**, it says:
"This page is now accessible from this virtual space, at this address".
When the Computer is in paging mode, only mapped pages are accessible.
Now every Process gets its own page table and tada: we have successfully isolated the processes from each other,
because every process can only access the data that it needs to access.
## Format strings
Format strings are strings that are formatted at runtime.
They are created by defining a pattern, like the following one:
`"Name: %s / ID: %d"`
And giving it arguments at runtime, let's use the following ones for our example:
`"Main Process", 42`
This would format to that:
`Name: Main Process / ID: 42`
As you see, `%s` and `%d` are placeholders.
Placeholders consist of a `%` sign followed by one or two letters.
When formatting the string, the placeholders are replaced with the arguments.
The first placeholder is replaced with the first argument, the second with the second and so on.
### Numeric specifier
If you put a `.` followed by a number right after the percentage sign of a placeholder,
you will set the `Numeric specifier`.
Some placeholders use this numeric specifier to configure their output.
If you don't set a numeric specifier, the placeholders, that would use it will use a default value instead.
### Arguments
Make sure, that the arguments you pass, are really of the right type.
If you e.g. pass a negative value of type `int32_t` like `-1312`,
the formatter will have problems with that, because the `int32_t` representation of that number is as an `int64_t` a positive number.
### Placeholders
#### `%s` - string
| **Argument Type** | `string_t` |
|-------------------------------|------------------|
| **Numeric Specifier Use** | None |
| **Numeric Specifier Default** | None |
| **Description** | Inserts a string |
#### `%c` - char
| **Argument Type** | `char` |
|-------------------------------|---------------------|
| **Numeric Specifier Use** | None |
| **Numeric Specifier Default** | None |
| **Description** | Inserts a character |
#### `%u` - unsigned decimal
| **Argument Type** | `uint64_t` |
|-------------------------------|-----------------------------|
| **Numeric Specifier Use** | None |
| **Numeric Specifier Default** | None |
| **Description** | Inserts an unsigned integer |
#### `%d` - signed decimal
| **Argument Type** | `int64_t` |
|-------------------------------|--------------------------|
| **Numeric Specifier Use** | None |
| **Numeric Specifier Default** | None |
| **Description** | Inserts a signed integer |
#### `%x` - hexadecimal
| **Argument Type** | `uint64_t` |
|-------------------------------|------------------------------|
| **Numeric Specifier Use** | None |
| **Numeric Specifier Default** | None |
| **Description** | Inserts a 64 bit hex integer |
##### variants
###### `%xb` - byte hexadecimal
| **Argument Type** | `uint8_t` |
|-------------------------------|-----------------------------|
| **Numeric Specifier Use** | None |
| **Numeric Specifier Default** | None |
| **Description** | Inserts a 8 bit hex integer |
###### `%xw` - word hexadecimal
| **Argument Type** | `uint16_t` |
|-------------------------------|------------------------------|
| **Numeric Specifier Use** | None |
| **Numeric Specifier Default** | None |
| **Description** | Inserts a 16 bit hex integer |
###### `%xd` - dword hexadecimal
| **Argument Type** | `uint32_t` |
|-------------------------------|------------------------------|
| **Numeric Specifier Use** | None |
| **Numeric Specifier Default** | None |
| **Description** | Inserts a 32 bit hex integer |
###### `%xq` - qword hexadecimal
This variant is the `%x` standard.
| **Argument Type** | `uint64_t` |
|-------------------------------|------------------------------|
| **Numeric Specifier Use** | None |
| **Numeric Specifier Default** | None |
| **Description** | Inserts a 64 bit hex integer |
#### `%?` - boolean
| **Argument Type** | `bool` |
|-------------------------------|---------------------------|
| **Numeric Specifier Use** | None |
| **Numeric Specifier Default** | None |
| **Description** | Inserts `true` or `false` |
#### `%b` - binary
| **Argument Type** | `uint64_t` |
|-------------------------------|-----------------------------------------------|
| **Numeric Specifier Use** | The amount of bits that are shown |
| **Numeric Specifier Default** | 64 |
| **Description** | Inserts the binary string of the given number |
#### `%%` - mask
This is not a really a placeholder, but you can use this to mask the % sign,
so it will be interpreted as just a `%` instead of a placeholder.
---
**DISCLAIMER:** Only the headers are documented, because documenting the whole code itself would be very time intensive and the headers as 'public' API are the most important to document.
## boot
### boot_info.h
The goal of this file is to provide a universal struct of information needed by the kernel at start time.
At the moment this information is very limine specific, but the goal is to make it easy to add support for other boot protocols.
#### `boot_info_T` - struct
- **framebuffer** - struct with information about the graphics buffer
- **terminal** - bootloader terminal / log
- **memory_map** - information about the memory layout / regions
- **rsdp** - pointer to RSDP
### limine.h
This header provides the API to "communicate" with the limine bootloader.
More information can be found on the limine project's [GitHub](https://github.com/limine-bootloader/limine/blob/trunk/PROTOCOL.md).
## mm
### memory_map.h
#### `memory_map_get_total_memory_size(boot_info*)` - function (uint64_t)
Calculates the total amount of memory available, by iterating over the memory map.
The size is stored in a static variable, so no matter how often you call this function, the size will only be calculated once.
It returns the total amount of memory in bytes.
### page_frame.h
This header provides the functions for basic interactions with pages (in the physical memory space).
#### `pframe_manager_init()` - function (void)
Initializes the page frame manager, needs to be called once at kernel init.
#### `pframe_reserve(address)` - function (void) [Thread Safe]
Blocks a page, so it can't be requested or anything else.
If the page is already blocked by anything else, e.g. by a request, it won't be reserved.
#### `pframe_reserve_multi(address, n)` - function (void) [Thread Safe]
Reserves the page at the given address, plus *n* pages after that page.
#### `pframe_unreserve(address)` - function (void) [Thread Safe]
Unreserves a reserved page and makes it accessible again.
#### `pframe_unreserve_multi(address, n)` - function (void) [Thread Safe]
Unreserves the page at the given address, plus *n* pages after that page.
#### `pframe_request()` - function (void*) [Thread Safe]
Returns the physical address of a page.
This is kind of the low level version of malloc.
#### `pframe_free(address)` - function (void) [Thread Safe]
Needs a valid page address produced by `pframe_request()` as argument.
Invalidates the address and frees it, so it can be requested again.
This is kind of the low level version of free.
#### `pframe_free_multi(address, n)` - function (void) [Thread Safe]
Frees the page at the given address, plus *n* pages after that page.
### page_map.h
#### `VIRTUAL_ADDRESS_MAX` - macro
The highest mappable virtual address.
4 level page maps have a maximum address space of 256TB.
#### `page_map_flag_E` - enum
- **Present** - This indicates if the entry is used or should be ignored. Automatically set when mapping a page.
- **Read & Write** - A mapped Page is always readable. This flag allows writing to that page.
- **User Super** - If set, user mode access to the page is allowed.
- **Write Through** - Enables _Write Through Caching_ for this page.
- **Cache Disabled** - If this bit is set, the page won't be cached.
- **Accessed** - Set by the CPU, when this PDE or PTE was read. Won't be reset by the CPU.
- **Dirty** - Set when the page has been modified.
- **Larger Pages** - When this bit is set in a PDE or PTE, the entry points to a 1GB or 2MB page.
- **Custom 1 - 3** - Not used in NoxOS.
- **No Execute** - When this bit is set, the CPU won't execute code that lies in that page.
#### `page_map_T` - struct [page aligned]
This struct contains 512 entries.
These entries contain an address and flags.
The addresses link like this:
- **PML4** --> **Page Directory** or _1GB Page_
- **Page Directory** --> **Page Table** or _2MB Page_
- **Page Table** --> _4KB Page_
A pointer to a `page_map_T` can be loaded into `cr3` to load this pagemap.
#### `page_map_create()` - function (page_map_T*)
Allocates a `page_map_T` and returns a pointer to it.
#### `page_map_fetch_current()` - function (page_map_T*) [ASM implementation]
This function will return the page map, that is currently loaded.
To achieve this, it just reads the `cr3` value.
#### `page_map_load(page_map*)` - function (void) [ASM implementation]
Loads the given page map.
To achieve this, it writes the `cr3` value.
#### `page_map_map_memory(page_map*, virtual_address, physical_address, flags)` - function (void)
This maps **_physical_address_** to **_virtual_address_** in **_page_map_**.
The **_flags_** will be applied to the page mapping / page table entry.
It always applies the _Present_ flag.
#### `page_map_unmap_memory(page_map*, virtual_address)` - function (void)
Removes a page mapping from the **_page_map_**.
Page map structure intern pages won't be checked if they're still needed or not.
#### `page_map_get_physical_address(page_map*, virtual_address)` - function (void*)
Returns the physical address of the page, that is mapped to **_virtual_address_**.
#### `page_map_destruct(page_map*)` - function (void)
Clears a page map and frees all page map structure intern pages.
#### `page_map_entry_set_flags(entry, uint64_t flags)` - function (void)
This will set the provided flags to a page map entry.
#### `page_map_entry_get_flag(entry, page_map_flag_E flag)` - function (bool)
Returns if the given flag is set in the page map entry, or not.
#### `page_map_entry_set_address(entry, void* address)` - function (void)
This will set the provided address to a page map entry.
#### `page_map_entry_get_address(entry)` - function (void*)
This will read and return the address set in the page map entry.
#### `paging_init()` - function (void)
Initializes paging.
This reads the current page map set by the kernel and writes it to `g_kernel_page_map`.
#### `g_kernel_page_map` - global variable
The kernels page map. This page map is provided by the bootloader and read from `cr3` at `paging_init`.
## platform
### cpu.h
This header contains stuff directly related to the CPU.
OSDev Wiki: [x86 CPU Registers](https://wiki.osdev.org/CPU_Registers_x86)
#### `cpu_state_T` - struct
- **cr3** - Control register 3, holds the current page table
- **rax** - General purpose register
- **rbx** - General purpose register
- **rcx** - General purpose register
- **rdx** - General purpose register
- **rsi** - General purpose register
- **rdi** - General purpose register
- **rbp** - The Bottom of the current stack frame
- **interrupt_id** - The ID of the interrupt, that captured the cpu state
- **error_code** - Some exceptions such as the Page fault push more detailed information into here
- **rip** - The current instruction address
- **crs** - Segment selector of the associated IDT descriptor
- **flags** - The CPU's FLAGS register, a status bitmap
- **rsp** - The Top of the current stack frame
- **ss** - Not totally sure, what this does, but it has to do with security rings
This struct defines a *complete* CPU state, that can be saved and restored.
It is saved when the CPU fires an interrupt and restored by the interrupt handler when it's finished.
This allows multithreading and exception analysis.
#### `cpu_flags_E` - enum
- **CPU_FLAG_CARRY**
- **CPU_FLAG_PARITY**
- **CPU_FLAG_AUXILIARY**
- **CPU_FLAG_ZERO**
- **CPU_FLAG_SIGN**
- **CPU_FLAG_TRAP**
- **CPU_FLAG_INTERRUPT_ENABLE**
- **CPU_FLAG_DIRECTION**
- **CPU_FLAG_OVERFLOW**
- **CPU_FLAG_IO_PRIVILEGE_0**
- **CPU_FLAG_IO_PRIVILEGE_1**
- **CPU_FLAG_NESTED_TASK**
- **CPU_FLAG_RESUME**
- **CPU_FLAG_VIRTUAL_8086**
- **CPU_FLAG_ALIGNMENT_CHECK**
- **CPU_FLAG_VIRTUAL_INTERRUPT**
- **CPU_FLAG_VIRTUAL_INTERRUPT_PENDING**
- **CPU_FLAG_CPUID**
### exceptions.h
OSDev Wiki: [Exceptions](https://wiki.osdev.org/Exceptions)
#### `exception_type_E` - enum
These are just the definitions of the CPU-exception interrupt IDs.
#### `g_exception_type_strings` - global variable
This array of strings defines the names of the Exceptions.
#### `exception_handle(cpu_state)` - function (cpu_state_T*)
If an interrupt is an exception, the interrupt handler will call this function to handle the exception.
At the moment it will just panic, but in far future this could get expanded for page swapping, etc.
### gdt.h
OSDev Wiki: [Global Descriptor Table](https://wiki.osdev.org/GDT)
#### `gdt_selector_E` - enum
- **Null**
- **Kernel Code**
- **Kernel Data**
- **User Null**
- **User Code**
- **User Data**
#### `gdt_descriptor_T` - struct [packed]
**Well documented on the osdev wiki.**
#### `gdt_entry_T` - struct [packed]
**Well documented on the osdev wiki.**
#### `gdt_T` - struct [packed / page aligned]
A template that holds a `gdt_entry_T` for every selector
#### `g_default_gdt` - global variable
The default GDT configuration loaded when the GDT gets initialized.
#### `gdt_init()` - function (void)
Loads a descriptor of `g_default_gdt` as the system GDT.
### interrupts.h
This header contains all the stuff, needed to init and handle Interrupts.
#### `idt_register_T` - struct [packed]
This struct is very similar to the GDT descriptor.
It holds the size and address of the Table, where the interrupt handlers are looked up.
#### `idt_descriptor_entry_T` - struct
This struct stores information about one interrupt handler.
The osdev wiki explains this more detailed.
#### `g_idt_register` - global variable
The default IDT configuration loaded when the IDT gets initialized.
#### `idt_init()` - function (void)
This function fills all the interrupt gates (handlers) into the IDT and loads it.
## utils
### bitmap.h
Provides functionalities to create, destruct and work with bitmaps.
#### `bitmap_T` - struct
This struct holds a buffer for a bitmap and its size.
The size is the size of the buffer in bytes, to get the amount of storable bits multiply size with 8.
#### `bitmap_init_from_buffer(buffer, size)` - function (bitmap_T)
Creates a bitmap object from a given buffer and size
#### `bitmap_init(size)` - function (bitmap_T) [NOT IMPLEMENTED YET]
Allocates memory to hold a bitmap in the given size and returns a `bitmap_T` with that buffer and size.
#### `bitmap_destruct(bitmap*)` - function (void) [NOT IMPLEMENTED YET]
Frees the memory of the given bitmap created with `bitmap_init`.
#### `bitmap_set(bitmap*, index, value)` - function (bool)
Sets the bit at the given index in the given bitmap to the given boolean value.
Returns **false**, if the index is out of the bitmaps size bounds.
Returns **true**, if the operation was successful.
#### `bitmap_get(bitmap*, index)` - function (bool)
Returns the boolean value stored at the given index in the given bitmap.
Always returns **false**, if the index is out of the bitmaps size bounds.
### core.h
All the utils, which I didn't know how to name.
#### `CORE_HALT_WHILE(a)` - macro
This halts until **_a_** is true.
Used when working with blocking variables in e.g. thread safe functions.
#### `CORE_HALT_FOREVER` - macro
This halts forever and warns about this in the log.
### io.h
Provides basic Input/Output functionalities.
#### `io_out_byte(port, data)` - function (void)
Writes one byte of **_data_** to **_port_**.
This is a wrapper around the assembly `outb` instruction.
#### `io_in_byte(port)` - function (uint8_t)
Reads one byte from **_port_** and returns it.
This is a wrapper around the assembly `inb` instruction.
### logger.h
Functionalities to write logs to QEMU's serial port.
#### `log_level_E` - enum
- **Info** - General information, that could be useful
- **Debug** - Should only be used to find bugs and removed (or commented out) after the bug is found
- **Warning** - Used for warnings and not important errors
- **Error** - Used for Fatal Errors / Will be printed to the screen (graphics driver is not Implemented yet)
#### `log(log_level, string, ...)` - function (void)
Logs the given string to QEMU's log port, the string is prefixed with the log type.
Format strings are supported.
### math.h
Mathematical functions, definitions, etc.
#### `MAX(a, b)` - macro
Returns the bigger one of the given values.
#### `MIN(a, b)` - macro
Returns the smaller one of the given values.
#### `CEIL_TO(a, b)` - macro
Aligns **_a_** upwards to **_b_**.
Example: `CEIL_TO(13, 8)` would return 16, because 16 is the next higher multiple of 8 after 13.
#### `FLOOR_TO(a, b)` - macro
Aligns **_a_** downwards to **_b_**.
Example: `FLOOR_TO(13, 8)` would return 8, because 8 is the next smaller multiple of 8 before 13.
#### `pow(base, exponent)` - function (uint64_t)
Returns the power of `base ^ exponent`.
#### `abs(number)` - function (uint64_t)
Returns the absolute value of **_number_**.
### memory.h
Basic memory functionalities.
#### `memory_copy(source, destination, num)` - function (void)
Copies **_num_** bytes from **_source_** to **_destination_**.
On linux this function is called _memcpy_.
#### `memory_set(destination, data, num)` - function (void)
Sets **_num_** bytes at **_destination_** to **_data_**.
On linux this function is called _memset_.
#### `memory_compare(a, b, num)` - function (bool)
Compares the first **_num_** bytes at **_a_** and **_b_**.
Returns **false** if there is a different byte.
Returns **true** if the data is the same.
There is a similar function on linux called _memcmp_.
### panic.h
Ahhhhh - the kernel is burning!
#### `panic(state, message)` - function (void)
This prints out the error message, a stack backtrace (planned) and a register dump (planned).
After that, the kernel halts forever.
This function is called, when a fatal error occurs
### stdtypes.h
Standard type definitions, that are used almost everywhere.
#### `uint8_t` - typedef
8-bit wide unsigned int.
Range: `0` - `255`
#### `int8_t` - typedef
8-bit wide signed int.
Range: `-128` - `127`
#### `uint16_t` - typedef
16-bit wide unsigned int.
Range: `0` - `65536`
#### `int16_t` - typedef
16-bit wide signed int.
Range: `-32768` - `32767`
#### `uint32_t` - typedef
32-bit wide unsigned int.
Range: `0` - `4294967296`
#### `int32_t` - typedef
32-bit wide signed int.
Range: `-2147483648` - `2147483647`
#### `uint64_t` - typedef
64-bit wide unsigned int.
Range: `0` - `18446744073709551616`
#### `int64_t` - typedef
64-bit wide unsigned int.
Range: `-9223372036854775808` - `9223372036854775807`
#### `bool` - typedef
Boolean type, can hold a logical value **true** or **false**.
#### `true` - macro
Logical **true** value.
#### `false` - macro
Logical **false** value
#### `NULL` - macro
A pointer to nowhere.
### string.h
#### `string_t` - typedef
A null-terminated array of chars.
#### `string_length(string)` - function (uint32_t)
Returns the amount of chars a string has before it's null-terminator.
#### `string_compare(a, b)` - function (bool)
Returns **true** when the strings **_a_** and **_b_** are equal.
Returns **false** if they aren't equal.
#### `variadic_format_size(string, args)` - function (uint64_t)
Returns how long a format string with the given pattern (**_string_**) and **_args_** would be.
Useful to create a big enough buffer before formatting a string.
#### `format_size(string, ...)` - function (uint64_t)
This calls `variadic_format_size`, but instead of giving it a `va_list` you can give this function the actual arguments.
#### `variadic_format(output, string, args)` - function (void)
Formats **_string_** with **_args_** and writes the product to **_output_**.
The rules for format strings are specified on top of this document in the _General concepts_ block.
#### `format(output, string, ...)` - function (void)
This calls `variadic_format`, but instead of giving it a `va_list` you can give this function the actual arguments.
#### `string_unsigned_dec_to_alpha(string, value)` - function (void)
Converts the unsigned integer in **_value_** to an alphanumeric string.
The representation is decimal.
This string will be written into **_string_**.
#### `string_dec_to_alpha(string, value)` - function (void)
Converts the signed integer in **_value_** to an alphanumeric string.
If it is negative it will be prefixed with a hyphen.
The representation is decimal.
This string will be written into **_string_**.
#### `string_hex_8bit_to_alpha(string, value)` - function (void)
Converts the byte in **_value_** to an alphanumeric string.
The representation is hexadecimal.
This string will be written into **_string_**.
#### `string_hex_16bit_to_alpha(string, value)` - function (void)
Converts the word(16-bits) in **_value_** to an alphanumeric string.
The representation is hexadecimal.
This string will be written into **_string_**.
#### `string_hex_32bit_to_alpha(string, value)` - function (void)
Converts the dword(32-bits) in **_value_** to an alphanumeric string.
The representation is hexadecimal.
This string will be written into **_string_**.
#### `string_hex_64bit_to_alpha(string, value)` - function (void)
Converts the qword(64-bits) in **_value_** to an alphanumeric string.
The representation is hexadecimal.
This string will be written into **_string_**.
#### `string_bin_to_alpha(string, num_bits, value)` - function (void)
Converts the data in **_value_** to an alphanumeric string.
The representation is binary.
**_num_bits_** specifies how many bits, starting at the least significant bit, will be converted.
This string will be written into **_string_**.
#### `string_bool_to_alpha(string, value`) - function (void)
Converts the boolean in **_value_** to an alphanumeric string.
The representation is `true` or `false`.
This string will be written into **_string_**.
Reference in New Issue View Git Blame Copy Permalink