Deos64 Project
The following is the preliminary plan for making incremental progress towards 64-bit support in Deos as part of the multi-core effort while providing incremental and managed risk and delay to the multi-core product release.
Management Summary
Porting the kernel to 64-bit requires, at least, the following steps. The thought is that while we're doing multi-core we have to review essentially every kernel file, so adding a review and update for proper 64-bit aware type usage would be a small incremental cost.
If we add a kernel checklist item to say we document every situation where a "64-bit kernel issues have been identified but not resolved here" with some magic token, then we can defer actually resolving the issues if they get hard, and effectively get most of the "busy" work of porting to 64-bit done at a substantially reduced cost but stop after, or even part way through, so we can manage any delay to the multi-core release.
The tests could either remain unchanged, or be updated, or a mix, as long as we record, e.g., in the process status file, whether they have been updated and reviewed to reflect 64-bit. A similar marking could be done for kernel files if we have to stop short of a full propagation of 64-bit reviews.
Task List
| Task | Priority | Assignee | Status | Effort (hours) | Remarks |
|---|---|---|---|---|---|
| Define types that work appropriately when compiled for 32 vs 64-bit. | Done | ||||
| Convert kernel public headers to use those types. | In Work | Applicable to all components | |||
| Update kernel documentation for new types | In Work | Applicable to all components | |||
| Adapt common build utils to new DESK directory structure | Pending | ||||
| Adapt OpenArbor to new DESK directory structure | Pending | ||||
| Propagate new data types throughout the kernel | Pending | Applicable to all components | |||
| Resolve 64bit issue: Limits previously impossible, but now aren't | Pending | Some variant of this likely for each component. | |||
| Resolve 64bit issue: Effect of numeric type promotion, etc. | Pending | ||||
| Implement new 64-bit HALs | Pending | ||||
| Implement new 64-bit BSPs | Pending | ||||
| Update kernel tests to address 64-bit | Pending | Applicable to all components. Suggest linking tests at an address above 4GB. | |||
| Total | @sum(column) | xxx original estimate |
Product Overview
In the following Deos32 means 32-bit Deos, Deos64 is 64-bit.
There will be both 32 and 64 bit executables. n-bit executables will only run on the n-bit kernel. I.e., no running 32-bit apps on 64-bit kernel and vice versa.
Deos32 has 32-bit general purpose (GP) registers and 32-bit virtual address space. On some platforms Deos32 supports 64-bit physical addresses for platform resources.
- 64-bit physical addresses for Deos RAM unlikely any time soon.
Deos64 has 64-bit GP registers, uses 64-bit physical addresses, and supports more than 32-bit virtual address space. Architecture restrictions currently limit the actual size of virtual addresses:
- x86 : 48 bits
- arm : 48 bits
- PPC TBD
- MIPS TBD
Unused virtual addresses bits will always be sign extensions of the most significant address bit (a hardware requirement, and for future compatibility). Most interfaces will support a full 64-bit virtual address space.
Deos32 currently supports 64K of any kernel object (processes, threads, semaphores, etc.). Deos64 limits are TBD, but object quotas can be limited to 32 bits even on Deos64. RAM quotas could be left at 32 bits (4billion pages is ~16TB), but I suggest we go to 64 bits for RAM quota values.
Explicit padding will be added to structures where placement of 64-bit values causes odd 64-bit alignment.
Applications will be able to be compiled from single source to run on both Deos32 and Deos64.
- ISSUE: Can we remove deprecated APIs from Deos64?
- ISSUE: Can we remove PE file support from Deos64?
PAGESIZE would remain at 4KB.
- Rationale: cache partitioning becomes less effective at larger sizes.
Proposed Changes
Shall means it will happen, should means it is just important, so it really doesn't matter.
Directory Structure
- Current: /desk/x86, /desk/ppc, /desk/include/x86
- Desired: /desk/arch/x86, /desk/arch/ppc
Might be able to get by with symlinks, or perhaps have to stick with /desk/x86 for now. Comments solicited.
New architectures will be (derived from GCC preprocessor macros):
- x86_64
- ppc64
- aarch64 for 64-bit ARM (yuck)
In general, a given header file will support both Deos32 and Deos64. Architecture specific info will be in /desk/arch/$ARCH/include, transitionally /desk/$ARCH/include. Most variability at the API level will be handled by having types which vary in size depending on architecture, e.g., size_t and void*.
Preprocessor Macros
Use of "#ifdef __DEOS64__" and similar should be minimized.
Built-in preprocessor macros for 64-bit GCC compilers
- __x86_64__
- __PPC64__ (Note: __PPC__ is defined by both 32 and 64 bit compilers)
- __ARM_ARCH (Note: Defined with a value that can tell more about particular ARM architecture)
- __LP64__ Supported by all 3 64 bit compilers indicating GCC uses LP64 model
Types
We're going to take this opportunity to fix our API type names.
Pointers will naturally be sized appropriately for the virtual address of the underlying system.
Types should reflect semantic usage, not word sizes. In the past things were called DWORD to mean many things and, quite frankly, due to laziness. If something is a 32-bit CRC, then the type should be something that says "crc 32", e.g., crc32_t, not DOWRD, UNSIGNED32, unsigned int, etc.
For example, where a type is expected to hold a bitwise "or" of some enum values, use a "bitset", e.g., kernelAttributes, the result for the logical or of several kernelAttributes whould be kernelAttributeBitset. This provides better documentation, and hopefully eventually tooling support. Example:
- typedef enum {... } kernelAttributes;
- typedef uint32_t kernelAttributeBitset; /* Bitwise "or" of kernelAttributes. */
enum types can be passed as parameters by value. However, enum types should not be embedded in public structures or passed as pointers.
- The C standard leaves the size of the enum as implementation-defined.
- The ABI may say more.
- For clarity, where needed in a struct or pointer, use [u]int32_t with comment about the enum type which can be used to interpret the value.
CPU times will get their own signed and unsigned types. The unsigned type is the only publicly visible type, and the signed type can be used internal to the kernel for comparisons.
Types shall be based on explicitly sized underlying types (to make structure sizes explicit), not the underconstrained "int" or "long int", etc., types.
- Exception is cases where an external standard explicitly specifies an under specified sized C object, e.g., "int".
For explicitly sized values we'll use more modern names for types, e.g., uint8_t, uint16_t, uint32_t, uint64_t.
- The [u]intXX_t types should be used in preference to [UN]SIGNEDXX
- DWORD shall be eliminated.
- WORD and BYTE should be eliminated (there are very few references)
POSIX semantics will be assumed for some basic types. These types should be used for offsets, lengths, and return values:
- size_t Used for sizes of objects.
- ssize_t Used for a count of bytes or an error indication.
- uintptr_t unsigned integer type capable of holding a pointer
- E.g. readProcessMemory() for an addresses inside some other processes.
When referencing the contents of a GP register as uninterpreted data, and the size of the register depends on context, e.g., the "K" functions in the kernel, or the kernel debugger library, supervisorModeCallback(), etc., then use:
- gpr_t (bettername solicited, __ugpr_t, __gpregister_t?)
- the underlying type would be uint32_t or uint64_t.
When referencing data that can hold the contents of a GPR, pointer, or some other integer, then use:
- uintData_t
- the underlying type is the larger of gpr_t or uintptr_t
Handles
Currently handles support 64K of any kernel object. Suggest keeping that same restriction (for now) for Deos64, but Deos64 handles will be 64 bits. Comments solicited.
- Rationale, most of the time we don't save handles in structures, so the overhead will be minimal and it leaves room for the future.
Impact To Other Components
IOI
API
- User visible header files (new types as described above)
Ring Buffer
- The size of the counter at *systemTickPointer() will matter
- The age of a message (for freshness checking) is determined by now-then assuming 32-bit unsigned rollover of the clock. The clock is currently required to be 32-bit, whether its the system tick or other.
Config Tool/ioi-cvt
- Alignment of data structures and data structure members.
653
Runtime
- User visible header files (new types as described above)
Config Tool/deos653-cvt
- Alignment of data structures and data structure members.
MLD and gdbserver
MLD history
The MLD has always been an Ada83 native tool, supported by the DACS tools developed by DDC-I itself. It has been considered to move from DACS to SCORE - moving from Ada83 to Ada95 - but there are some technical differences that are not straightforward regarding primarily object layouts of dynamically sized objects, so this has still not been attempted for real.
Both DACS and SCORE can only generate 32 bit native target applications (not 64 bit), and there are likely unintended 32 bit host and target address 'mixups' that must be sorted out.
DWARF impacts
The current used DWARF already describes 64 bit integer types and the MLD can manage 64 bit registers, but the sizes of pointer types and target addresses may implicitly expect to be able to use the similar host types in some contexts.
In general DWARF is another area of some concern. Traditionally the MLD uses DWARF2 and the various C/C++ compilers (gcc and DDC-I's own tcc) are asked to use this format. The tcc/tepp of course do exactly that, but the gcc-compilers tend to use at least some DWARF3 and DWARF4 extensions that up to now have all been incorporated in the MLD. The internal DWARF encoding format has also been extended to not limit a DWARF section to what 32 bits can reach. The MLD currently only supports the old ('small') format, and provided that the gcc will continue to use this format when asked to, we should be ok with that. Deos applications are likely to use about the same DWARF constructs for 64 as for 32 bit, so we are not likely to run into DWARF2 format size restrictions. This also means that the DWARF dumper should likewise be more or less unchanged (except when showing register references).
MLD register showing
Showing registers in the MLD may involve not just showing the large 64 bit registers, but perhaps also the low 32 bit parts. However the MLD already has mechanisms for that in Deos context that will help out here. With the current 32 bit registers the MLD needs with SPE v2 to construct floats from the corresponding lower and upper 32 bit registers.
MLD disassembler
Another area is the MLD's disassembler which must be extended to display the extra instructions that are likely introduced with the 64 bit mode. In some cases the MLD looks for specific code sequences to detect e.g. the use of a stepping stone, and they must all be examined to match a possible 64 bit sequence.
MLD interface to gdbserver
A switch to 64 bit will of course also affect the MLD's interface to the gdbserver. The gdbserver shall accept 64 bit addresses in the relevant packages, as well as return 64 bit rather than 32 bit registers. If handling single instruction stepping involves decoding instructions this may need trimming. Thread information may need adjusting too. There are probably a lot more minor modifications needed here and there.
MLD change effort conclusion
The effort to allow the MLD to support 64 bit targets as well as still allowing support of the current 32 bit targets is definitely not insignificant!
How you can get screwed
We'll need to adjust our checklists. There are some very twisted situations where C's numeric promotion rules will get you completely hosed. E.g., if the argument to f is unsigned, f(expression resulting in -1) might call f with -1, but it might call it with 4bn depending on the expression. See the porting issue in the #References.
