The dust is settling

While the dust of the cosmic collision starts to settle we're still a long way away from the total merge of the projects.

At the moment it's more the drivers that are being implemented in the Andromeda kernel. Michel is the one responsible for that.

In the mean time Steven feels like he can't do a thing since the change is going so rapidly.

And me, what am I doing?

Well, I'm primarily focussing on the same stuff as before the merge and that is getting the paging system up to par. At the moment work is being done on the basics of the paging system and while I feel it isn't sufficient in the long run (when this might be ported to 64 bits systems) for now it'll do and will probably even be good (according to the standards we're working with now).

So what is going to change?

To start with work is being done on drivers such as Intel's Advanced Programmable Interrupt Controller or APIC. In the mean time work also is being done on ACPI or Advanced Configuration and Power Interface (which is a disaster to work with). That is what is imported from Openloader.

In the mean time we're also taking over the Openloader boot procedure and this is also a work in progress.

Also we're doing work on getting the kernel into higher half, which is where the new paging system comes in. It has to support identity paging which is that the same page can be referenced by multiple virtual addresses. This, in combination with relocatable code and a whole lot of rather simple arithmetic on relocating the heap, will allow us to relocate the entire kernel into the 3GiB region or the -2 GiB region on AMD 64 systems.

Work also is being done on the graphics systems, but that is basically what has been going on all the time since Steven joined, and we're looking to merge the makefiles.

The heap code had some significant bugs, which are now resolved, but no guarantees can be given that it is now bug free.

So while the dust is settling, the merging is far from complete and the legacy of the two projects will probably remain visible for months, if not years to come.

Cosmic collision

This blog post marks quite an important moment in what now used to be Andromeda as we know it, since we're doing something bonkers.

Basically what we're doing is calling the boot loader by Michel Megens the Milky way and sped up time.

If you're not aware of what's going to happen in a couple of million years, well, here's a quick explanation: "Boom!"

If that doesn't cut it for you, well, Andromeda and the Milky way are both galaxies (not sweets) and Andromeda is the only galaxy moving towards the Milky way (aka us).

This means the two will collide in a couple of million years and the two galaxies will likely merge.

That's exactly what Openloader and Andromeda (the kernel) are doing. We're fusing.

Since the end result will still be a kernel the name of the kernel will persist, but the project structure will likely change.

In galactic terms this means the two black holes at the centre of the galaxies will revolve around each other but not merge. In project terms, this will mean we have two project leaders.

Michel being involved with the direct hardware interfacing, while Bart will be focussing on the higher level algorithms such as scheduling, paging and other kinds of memory management.

The merging of two galaxies is a lengthy process and often messy. I don't see why the merging of the two projects will be different, but all together we will likely gain more mass and momentum (at least that's the general idea).

You can still use https://github.com/bemk/andromeda as main repository, but the official one now resides at https://github.com/Andromeda-Kernel/andromeda.

We hope the dust of this collision will soon settle down so we can continue what we were doing before this.

Bugs, bugs, bugs and more bugs

Yeah, the title says enough, I'm afraid.

I think it would be an understatement to say that we found SOME bugs last summer break. We found a lot.

First up was a series of bugs related to the fact that the ELF loader didn't fully comply with the standards. Once that was found it was an easy fix.

Further, what we've unearthed is quite an impressive bug, this bug is related to the paging system, which by the way, is by no means finished yet.

It's this last bug that I've been working on since a while and the last several days I've been getting some small clues of where the bug could be.

At first we thought it could be a memset issue, but by replacing the current memset with a simpler one, we eliminated that as the possible and most obvious cause.

We then continued with some other obvious culprit which was the memory allocation system, which we started testing. During these tests we figured out that the allocator wasn't the issue.

This led me to thinking about the issue in a new light. From the paging perspective, and so I did a little test, to figure out where in memory this bug starts.

As it turns out, once we enter a level of memory above a certain number, we cause page-faults, which in turn cause more page-faults, and so forth.

This issue is probably due to the way I approach paging, but it leads to the more interesting question. What is precisely going wrong, since without knowing that, we can't fix the issue.

That's what we're doing now, so we probably won't meet this sprints target, but that doesn't really matter, as the bugs resolved will have a way higher priority.

That's it for now,

Hope you had a nice summer break, and see you next post.

Chaos in the project?

Strange title, maybe?

Well, yes. There is a little bit of chaos in the project, mainly because it's vacation season, so that means that everybody will spend some time away from the project and most things which would normally go well, don't now.

Another source of chaos might be the fact that, up until the last release I had a pretty good idea of where I would like to go with the project. I need it to be in high memory and I want that to be done in a simple fashion, which I think I have completed by now.

Next up I thought I'd look at streams and everything, but it turns out now that we need at the very least a simple virtual file system. Now that very simply put means that I don't have a clue what I'm doing.

I have been reading this book written by Andrew S. Tanenbaum and it does give a nice overview, but it's all theory, and like we all know, there is a difference between theory and practice. Oh-yeah, there are some differences all right.

Now I'm not going to sum it all up for you, but it boils down to the fact that I have a lot of questions and pretty much nothing at hand to answer it all, besides the book, which I'm still reading.

I'm at the same time also looking at the OS dev wiki and have found some interesting bits of info there as well.

So, what can one expect to happen until September the first, when this sprint should be done.

Basically just us lagging severely behind. I think I might have overlooked the gravity of some of the functions and that in planning terms will cost us dearly.

Now this doesn't matter anyway, as I still don't have a clue of when the 1.0.0 release should be.

One thing is for sure though, and that is that the 0.0.5 release might have a little less than one'd expect it to have, more because of the gravity of the features, than because of the lack of planning in the vacation (which is why the sprint ends first of September instead of August the first).

Speaking of summer break, what am I going to do?

Well, I'm going to France for a couple of weeks, and the neighbours will be watching over the plants and everything.

My sister is not going with us for the very first time, because she gets to go to language training camp in Cambridge (lucky arse).

In the mean time I'm pretty sure Steven will keep going with the graphics drivers and stuff like that.

So I think that'll be it for now and I plan on posting again when I'm back.

And here comes the fourth release

Yep, its time for the fourth development release also known as 0.0.4.

This release features the ability to:

• Respond to interrupts (again),
• Load in the core image and
• Compile the core image.

The previous release (the overhaul of the project layout) should actually have been part of this release, but it was so much of a change that I thought it was worth its own release, this making this the fourth release.

I certainly did a short silly dance once I had the elf loading done for sure (probably not a soul that's seen it, lucky me), because this was a point I was working toward from the beginning of the project.

Up next (at least for me) is the revising of the printf functions and the way text is displayed on screen. This actually might get in the way of Steven a little, but I don't care as he is on vacation anyway.

The next release is scheduled in September, and that might seem an awful long time from now, but keep in mind that the summer breaks are in the middle of this, and that I'll be gone for 21 days here.

Steven has been delivering code, but unfortunately this is not testable, and thus is shifted to the next sprint, which actually starts today (Monday, June the 11th of 2011).

We wish Steven a nice vacation and hope he will be able to make some good progress in my absence.

We also wish to thank him for the server space he's provided.

This is a link to the tag formed during the 0.0.4 release.

Major break through

I know this post is a little early, but I don't care, since we have had this major breakthrough last nigt.

It doesn't seem like much, but if you're a little familiar with the project you'll realise that the fact that we have Elf loading working, is a HUGE thing.

The reason why it such an impressive feat is because everything has been made from scratch.

"So?" You might ask.

Well, We've had to initialise a heap, use that heap (did that with a first fit algorithm), print some text to screen, and set up paging.

The first big breakthrough was when I had a heap working, the second was paging, even though that one isn't really finished yet. The last in this sequence is Elf loading, since it puts every thing together.

It basically pulls the interesting stuff out of the image and pasts it somewhere in memory (defined by the image). The weird thing is that the batch of memory requested might not even exist. That's difficult since we actually need it.

Not to worry, we have a way of doing things to make this all possible. It's called virtual memory. Basically what it does is split the address space up into 4KB pages, and a page is then connected to a physical batch of memory, allowing for pages to exist in memory addresses which don't exist in the physical address space.

This does however come with a problem. Since the virtual address space is 4 GB in size and the actual memory size is something random, that might spawn the problem of a lack of physical memory. Now that's solved by swapping the pages out to disk.

This however hasn't been implemented, which has a very simple reason. We don't have file system support, and thus we aren't able to write to or read from disk.

Guess what's up next in the planning?

Well, yes, graphics, but besides that?

Indeed, file systems support. This will likely be a three tier approach.

Tier 1: The virtual filesystem (vfs), this is what you interface with as user/programmer.

Tier 2: The filesystem driver(fs driver), which is what looks up the physical position on a physical hard drive, regardless of what type of drive you use.

Tier 3: The disk driver, which takes commands from the filesystem driver to access specific blocks on the drive, without the need to know about filesystems.

How this will be implemented needs some thinking though. Are we going to implement the inodes in the vfs, or will that be supported in the fs driver, and if so where will we keep track of mount points.

If you have ideas, they're always welcome of course.

Another thing that desperately needs to be done is the design of a logo. Now for more info go here.

The fourth release already?

So it's been a while since last blog post. I know. Last time we were still in 0.0.2, and now I'm already talking about 0.0.4!

What's happened you might ask. Well, if we open the new site, we will see several things there. There is a sprint backlog over there, which explains what we're currently working on before the next release. There's a product backlog which represents all the future features, and there's some more general info.

The sprint page shows you what we're doing and what's done. I'll make a quick summary for you, since this page will likely be updated in the near future.

We've:

• Set up a site,
• Finished the header analysis,
• Done a new project layout,
• and verified the elf header.

What's more is, I've actually just finished work on making the core image compile.

Now the work on the core image isn't finished yet, as it still needs testing (it's now in to verify) en might thus go back into the in progress, or even the todo if the issues appear to severe.

In the mean time Steven has been doing some work on getting the graphics going, and even though we don't always agree, he does his best, and quite often produces good work. Hence he's also got several topics.

He's also working on a feature which isn't in the sprint. The feature needs to be done for sure, but will probably be introduced in the master branch at around 0.0.5 (Oh my god, I'm already talking about an 0.0.5 release!).

In the mean time I have had official verification that I may continue at school into the second year!

Now for more info check out the site at http://orionos.tk/.

A quick side note though. Because of energy saving and the target audience primarily being in Europe, the server will be down between 23:00 UTC until 7:00 UTC.