Re: [MiNT] virtual address spaces

[Evan Langlois <Evan@CoolRunningConcepts.com>]

On Thu, 2005-12-08 at 09:47 +0100, Adam Kłobukowski wrote:

> 1. Disk drivers
> 
> Atm. disk drivers (XHDI) live outside the kernel. That means the are 
> hard to be put into virtual address space.

Disk Drivers should be working at an interrupt level, in supervisor
mode, and so should be using physical addresses already.  Pretty sure
MiNT handles this although an integrated disk driver would be nicer.
I'm thinking disk access blocks the kernel and all other apps, and an
in-kernel driver would fix that.

> 2. AUTO
> 
> User can put lots of stuff to run before FreeMiNT kernel. Arrgghh...

I'm sure we can work around it.  Most of them would all be running in
the kernel space.  Without an MMU, it just works.  With a real MMU, you
could make AUTO folder programs have global memory protection so that
RAM space would be mapped into every application at the physical
address.  Basically its a hole in the memory protection - easily patched
by not running AUTO folder programs.

> 3. Apps that have idea how memory looks, GEM protocols and pointer 
> sharing....

Most *GEM* protocols don't pass pointers between applications.  Thats a
hack that has nothing to do with GEM.  No AES functions pass pointers
between applications.   Programs that use AES features to pass pointers
to another application would break unless they are set to global memory
protection.

However ...
1 - Any approach to shared text areas that don't use an MMU will
require, at the minimum, a recompile to implement some form of pointer
indirection.
2 - Any app that sends or recieves a pointer must, for safety reasons,
run in the same address space as the program it wants to communicate
with.

So, you'd now have 3 classes of application (2 if you remove the third).

1 - Older compatibility mode - call it TOSBOX or something.  All
applications that want "Global" memory protection go here.  If you have
an MMU you can make sure applications don't access outside THIS address
space, but they can access each other inside this space.  Any
application running in this mode isn't likely to need shared text or a
functional and efficient fork().  Physical and Virtual address space can
be different if an MMU is present, but each app shares the same virtual
address space.  No new apps should go here.

2 - Standard "Clean" Applications - text is shared only if an MMU is
present. If an MMU is present, each program gets its own address space
and memory protection, paging is possible, etc.  Without an MMU there
isn't much gain over #1 except you don't get to share memory with
pointers .. you have to use the OS features to allocate a shared memory
block and follow the rules.  The lack of any benefit without an MMU
present has made implementing such features low priority.

3 - Shareable App - This would be like #2 except that the OS gives you a
pointer to a table of function pointers to your shared library functions
(can easily re-use LDG for this I think), a stack pointer, and a pointer
to your heap for globals and such (if you need more than 64K of
globals/statics, you should be allocating it dynamically).  Its
basically the old shared-test segment hack that used base-relative
addressing except we'd add in some hacks so that libraries can find
their own static areas from pointers relative to your base.  

For example, make the start of program heap (where the base register
points) contain a table of pointers to blocks of data.  Each library
might have its own index into this table and would use this table to
find any non-stack data.  The OS can allocate all of these blocks and
make them owned by the application for easy cleanup while the library
text segments themselves would be completely shared and OS controlled.
You could even allow libs to have a "global data" pointer in that table
that would be allocated and deleted with the library itself and shared
between all apps (like if a library loads bitmaps such as standard
icons, or fonts that all users of the lib should share).  Even Linux has
trouble with the last part which is why my Mail reader is using 169M of
virtual memory, 12M is shared text, and 76M is resident but not
shareable by the layers of libs, and the rest is not shareable, but not
even important enough to be in RAM while I'm typing - likely more
library data that couldn't be shared.

> First world is an "old style". W keep AUTO, XHDI drivers, old apps 
> there. Any acces to ST-RAM address space is a phisical (ie. not 
> translated). Any accest to TT-RAM is translated. All processes live in 
> one virtual memory space

Yes, #1 above, although I forgot that ST RAM can't be translated since
it must be DMA accessible.  Don't know if we even care about TT RAM for
these apps ... might just restrict them all to ST RAM only and assume
that pointers are all 24 bit.  You should be able to emulate the memory
address scheme of the original ST using the MMU for extra compatibility.
Of course, you'd be limited to about 14MB of RAM to be shared between
all the apps within this "TOSBOX" space but it would sure be compatible!

> First world is created at startup from phisical memory - ie each memory 
> chunk that is malloced before FreeMiNT starts goes, and stays there.

We think alike!

> Second world: for all new apps, every app have their own adress space, 
> (almost) full memory virtualization for each process, every process have 
> his own sandbox.

Correct, except that with an MMU the AUTO folder programs are part of
the kernel and must NOT be writable by the previous set of applications.
It should be possible for the entire "TOSBOX" space to crash and take
down everything within that space without hurting the kernel, modern
apps, or anything that is in an interrupt vector!

> We can keep a "window" at fixed memory address (ie. the same address in 
> each sandbox) between both worlds to allow some limited support for old 
> style GEM protocols...

No!  I don't think this is a good idea at all.  This wouldn't be 100%
compatible with older applications and would only encourage new apps to
do things the wrong way.  If we get new shoes, lets not keep the wearing
the old dirty shoes.  The old shoes get thrown in the TOSBOX if you want
to save them, but new ones have to be clean!  We have the OS support
ALREADY to not do such dirty things with new shoes.

-- Evan