Ra­tion­ale

A rant on why things should be done differently

While consider­ing various extensions to RISC OS, in both kernel and support libraries, I've found that a rethink of the common underlying mechanisms associated with multi­tasking is in order. Here are some scenarios expressing those consider­ations.

Shared memory, shared/​dynamic libraries and forking

Shared libraries are useful to RISC OS, but we really only have one, the Shared C Library, and it's implemented in a slightly unsatisfactory way as a relocatable module. I gather that it's not particularly easy to make and maintain other libraries this way.

So I wrote a shared-library system for RISC OS based on dynamic loading and linking of AOF files. It worked, but was never taken up and deployed, possibly because of (inter alia) the overhead of parsing an AOF file at run time. ELF would be better in this respect, and is supported by certain development tools.

ELF is designed to exploit an underlying shared-memory system, possibly with copy-on-write capability for efficient UNIX fork() implement­ation. Since there is a drive for UNIX compatibility on RISC OS, including the fork() function, I figured it would make sense to have a stand-alone shared-memory implement­ation first, and then build the ELF shared-library system on top of that.

In order for a shared-memory system's copy-on-write facility to be exploited by fork, that system must be informed when a fork is taking place. But it mustn't be coupled to the informer (i.e. the fork call must not inform the shared-memory system directly), since other systems may want to be similarly informed.

(Note that I'm not suggesting that an ELF shared-library implementation isn't possible without a major overhaul like PipedreamOS – Here's one already! – merely that it might be much easier.)


Supporting a portable reactor/​proactor interface

The reactor pattern is a way of managing multiple I/O events in a single-threaded environment. A program designed this way sets up a reactor, and makes it available to the components used to build it, e.g. GUI, network middleware, other device controllers (e.g. audio). These all have different I/O require­ments on the kernel, and if they tried to use traditional blocking calls unilaterally, each would interfere with the concurrent running of the others (in the same thread). (And non-blocking calls would have to be used either continuously, which is highly inefficient, or periodically, which is unresponsive.) Instead, the components register their interests in I/O events (such as data arriving from the network) or timing events (e.g. a particular moment in time, or a few seconds from the present time) with the shared reactor, and multiplexes their requirements into a single kernel call. When that returns, it demultiplexes the results, and up-calls the components for which relevant events have occurred. (This is not unlike the co-operative multi­tasking in RISC OS, but it occurs between components in a single thread in a single program.) A portable reactor interface allows components to be written to co-operate with each other in a portable manner, because even platform-specific versions of a component can all have the same, portable interface.

I wrote a ?reactor? as a C library for use on UNIX systems. This is easy: almost all I/O in a UNIX process is through a ?file? descriptor – which includes network sockets, device descriptors and pipes to other processes – and you can monitor several descriptors at once with a single select() or pselect() call, so the process can be put to sleep completely while the kernel efficiently watches for interesting I/O on behalf of the process. (There are also poll() and epoll_wait calls which offer improvements over select().)

I also adapted the reactor library to work portably on Windows. Windows provides Msg​Wait​On​Multiple​Objects​Ex() (and some related functions) to do the same sort of job as select(), although it uses ?system handles? rather than file descriptors — these are the ?multiple objects? one waits on with the call. (In fact, the use of handles, including semaphores and such, allows the reactor to behave as a proactor, whereby the kernel informs the process that an I/O operation is complete, rather than that an anticipated operation would not block. Something similar in PipedreamOS might be advantageous.)

I figured I could do a simultaneous port to RISC OS, but I find it's too awkward. Multiplexing of events outside the WIMP is not achieved in a single, standard way — e.g. for sockets, it seems to have been bolted on afterwards. And the network is just one of several subsystems that might benefit from event multiplexing, e.g. a floppy drive, USB devices, etc. Will these be designed with multiplexing in mind, or will it be an after-thought? How will these subsystems interact in a program that uses more than one, particularly if the components used to access individual systems are libraries independent of each other? Is it going to make my reactor impossibly complex to build and maintain?

Some further info on reactors and proactors:

[Edit:

Hmm, maybe there's a way…

]


Decoupling of GUI and multi­tasking/​context switching

The WIMP system of RISC OS is an unnecessary conflation of context switching and GUI, leading to poor inter­action with other subsystems that need to be aware of context switching.

For example, subsystems with blocking operations (such as sockets) must use mechanisms to keep in touch with the WIMP, which are non-uniform, somewhat ad-hoc and complex for the developer, and incapable of dealing with pre-emption (should that be available). Other blocking subsystems are unaware of the WIMP, because they (rightly) should not be GUI-aware, even if they should be context-aware, so their use puts responsive­ness at risk (e.g. when a floppy-disc access occurs). Memory management, as another example, is tied in to the WIMP implement­ation, and so cannot be replaced and improved by itself — a shared-memory system would need to be informed of when a context is switched, for it to switch the memory mappings correspondingly.


The conclusions I draw from these consider­ations are that: