What we've been up to

I know, it's been a long while, and quite a bit has happened since.

A summary of what's been done:

1. A driver model has been designed and implemented and is nearing completion (aside from the actual implementation of drivers).
2. A virtual file system has been designed and implementations are on the way.
3. Network stack development is currently being done by Michel.
4. Ideas have formed on a new memory allocation algorithm.

Now this doesn't sound like much but for two people it's quite a bit of work, considering the fact that in the design of core features in the kernel we prefer to guarantee quality.

In following posts we'll go deeper into the separate items.

• The driver model
• The virtual file system

Cleaning up

Currently a lot of work is going into getting the repository clean again.

A lot of files have been placed in directories where they shouldn't be. For example, the directory src/sys shoudn't have existed.

We've been moving most files out to where they ought to be. For example, the error directory, which contained the panic function, has been moved to the kern directory, which by the way has been renamed to andromeda.

This generated a file name conflict with the output binary, and as such we renamed that to andromeda.bin.

In the mean time I've been working on support for multitasking. We've covered quite a lot of ground there, but as we speak a lot of code still has to be written. For example, we still don't have the possibility of switching contexts. Something I think is mandatory for multitasking. Now of course it can be done through some weird function pointer using scheduler, but I personally prefer to go with a traditional task switch.

What does that mean?

Well, basically it means that when an interrupt comes along, before we do anything else, we push all our registers to stack. When the interrupt is handled a couple more functions are called, probably pushing more onto the stack.

When the time comes to switch tasks, what we do is we only store the stack pointer, just before the moment we swap the memory context, by loading another page directory pointer.

When we have to load the old task again, all we have to do is first restore the directory pointer, and then use a link in kernel space (which is always present) to restore the stack pointer.

When we then return to the interrupt handler, the registers will be restored to the state when the process was interrupted and the interrupt will be finished.

Basically that's the normal task switch. Does it sound complex? A little. Does it have to be this way? No, but it sure is the most extensible way of doing things. When a new register is added, all we have to do is remember to push in onto the stack in the interrupt handler and the rest can be dealt with in a more generic location.

Out with the ugly hacks

Considering the present time (0:14, at UTC+1) I'm going to keep this one short.

I've been working on getting scheduling working, and I found the GDT trick to be a little bit of a nuisance. As it turns out, I had a design in mind as a replacement for this trick.

What does it do?

Basically what we want is to have the kernel start at somewhere very high in memory, both tricks allow us to migrate the kernel code without endangering our more volatile code.

The GDT trick requires us to exploit some feature in the CPU which isn't guaranteed (as far as I could find). Now this doesn't sound very nice, and isn't really portable too.

Now what I did was remove this trick and enabled paging very early on. This means that paging will actually replace the function of the GDT trick. Now this was kinda tricky since booting with the GDT trick might require one to do some very nasty stuff later on.

This code still exists, so I had to enable that in the paging trick. Basically what we have now works.

More info on this can be expected later.

Still waiting

So we didn't quite reach our deadline for November 4th.

I think it has been a little too ambitious, considering the situations all developers are in now.

So we're definitely working towards the 0.1.1 release, and among the things we already mentioned would be in there, we'll also feature file buffer support.

Now that doesn't say a lot though. Basically all it says is that we have something that resembles a file, which you can write to, and read from. Seeks are also supported, but unfortunately, we can't sync with disk, or even clean up read items in a one way stream/pipe ...

Michel still is working on his PCI and APIC support and is thinking of including C++ code into the project (I wish him luck implementing the special operators, I'll continue doing C).

Doubted is whether or not we will be able to put PCI and APIC into the next release. There'll be a small discussion on that and we'll get back to you when we've got more news about the next release.

Until that time, the developers will be working on both school and the kernel.

Also, I've managed to set up gitweb on my own server, so if you like you can subscribe to the rss/atom feeds on there: http://orion-os.eu/gitweb/

I hope to have more news soon.

A new release already?

A new release is coming up on the horizon. We now have the date of Friday 4th of November 2011 planned for the release of Andromeda-0.1.1.

This will feature a brand new paging system, a new memory mapping system, minimal ACPI support and basic support for the Advanced Programmable Interrupt Controllers (APIC).

The paging system has been made in such a fashion that the memory requirement scales to the pages requested, instead of permanently occupying 4 Megs of RAM.

The same goes for the new memory map, and did I already mention it is based on the size of the actual physical memory!

This means that we always occupy 0.2% of physical memory available for the memory map, and depending of the amount of virtual memory usage a little bit more for the page tables.

Also we've conjured up a way to put the kernel at 3 GiB. It's called the overflow trick. The principle behind it is pretty simple. All you do is set up a segment which maps memory address 0  to the 3GiB location and make it 4 GiB long or so. Then you jump into the higher half code.

Simple as that. All you need to do is page the code to both low and high memory, disable the overflow trick and unmap the low memory code, leaving only high memory in place.

Whenever some low address is required, all you need to do is add 1 GiB to the address and the 32-bits integer flows over when the segment is applied, and the correct pointer is used.

Simple, stable and effective (do watch out with virtual machines that don't properly emulate the 32-bits overflow, they can be a real pain sometimes).

The ACPI support is created by Michel Megens, so not much for me to mention on that part. The same pretty much goes for APIC support.

And speaking of dates! My birthday was last Sunday (October 30th) at which I turned 19!

Continuing to move

So I found this bug I was telling you about last blog post. It basically boils down to me having forgotten to map the entire lowest 15 MiB.

That means page faults, which cause a double fault (because the page fault isn't mapped), which causes a triple fault (reboot, because the double fault isn't mapped).

That's nice, I think, but now I have that settled, I need to remove it again after I disabled segmentation, otherwise we've got a security leak on our hands, greater than the great wall of china.

In the mean time I am trying to build a way to keep track of the memory that is mapped, and the memory that isn't. At the moment I've got a system in place that produces 0.29296875% overhead on the entire memory system, at MOST versus a huge 6.6666....% using the old system (and yes I used a calculator for that! I'm not that smart).

This is without counting the heap overhead, which go through the roof on very small allocations, but because that's too unpredictable I've chosen to only count the paging system in.

What needs to be done now?

First of all, the page fault system is to be written. This will in the future include swapping, but since file systems aren't supported yet, a swap file would be pretty useless. So if we run out of memory now, we just panic!

Second the lowest 15 MiB must be released again. Once that's done, we can start doing work on getting all the features we broke by this, to get to work again.

I see a mammoth task in the future, which means I'm happy it's nearly autumn brake.

In the mean time Michel is working on the ACPI tables and stuff like that. I hope he'll get to explain it to me in the near future. I sure wish to know this.

Also I'm getting an Arduino board. Just for playing around. Maybe I'll attempt to port Andromeda, if the chip is even capable of the tricks we require.

And there is a new server coming up. It's made with absolutely NO moving parts so that I can keep it running longer.

Up, up and, well, you know the story

Basically what I've been working on is getting back to the higher half mode.

The trick I've chosen this time is the GDT trick, designed by Tim Robinson.

What this means is that the image is linked to be at 3GiB and then we use paging to trick the CPU into thinking the code is where it should be.

This is nice and all, but it does make things with physical memory addresses a little harder, and since that's the case I still haven't been able to get paging to work.

The weirdest thing is that the debugging values seem to be correct, while the virtual machine still triple-faults (for non OS-developers, that's basically the PC throwing it's hands up in the air, saying: "I can't fix this, do it your self!").

In the mean time Michel has been continuing work on the ACPI tables.

Besides that there is lack of a clearly defined website to go to for the kernel.

We're working on that, but do keep in mind that we have higher priorities at the moment (such as school).