Kernel 9 BSP Changes
From DDCIDeos
Jump to navigationJump to search
Description
This project proposes what is hoped to be the final changes to the Deos Kernel version 9.0 BSP interface.
The first version of this page is a direct copy of (part of) Deos_Modular_Filesystem. See there for history and rationale.
Work Assignments
- RLR: Get qemu-arm using latest dev-kit against kernel head. Done!
- AL: update SRT and boot iface with wiki image file header proposal
- AL: update hypstart & hypdump
- Generate error if no video memory
- Application base addr (make sure video memory doesn't muck with it).
- RLR: update dev-kit
- RLR: incorporate into qemu-arm
- RLR: incorporate into kernel code
- AL: update hypstart & hypdump
- PAL Interface
- kernel
- RLR: put simflash into kernel so kernel can run without pal flash functions. Done!
- AL: Finish libkfs.so
- update deos-kernel-download-mode.fp.xml (DEBUG mode, ...)
- TBD kernel updates
- FTP
- IT tool (remove downloading permitted)?
- RLR: dev-kit
- RLR: qemu-arm
- kernel
For each BSP:
- Update openarbor.options file to latest version
- Create platform project, then copy .options to the BSP's open-arbor.options
- Change all refrences to various froms of "test results" to TestResults
Proposed BSP Interface
Timing of changes to the BSP interface
Management has tasked the kernel team to minimize the BSP interface changes that cause all of the BSPs to be "respun". The immediate driver is several pending customer requests that will require more non-backward compatible boot interface changes in the near future.
Management has asked the kernel team to make changes at this time to the boot interface that address all current requests for changes to the parts of the hyperstart image touched by boot. This is driven by the reality that all the BSPs need to be respun at this time for other reaons.
Future proofing
Strategy for managing future changes:
- Permit hyperstart to support multiple versions of the "boot only" hypstart data structures.
- Add "BIO converter" function (that would replace the kernel entry portion of the SRT?)
- This would be something injected by hypstart as a "shim" for older BSPs.
- Some self descriptive header with types and offsets
- Add some padding/unused fields
The rationale for the first two is that many of the previous changes were non-backward compatible because complicating the kernel was not a good trade for maintaining backward compatibility. Putting the onus on the kfs writer library (a level E component), or hypstart (a non-qualified tool) makes that overhead more manageable.
Making the data structures that boot has to deal with more descriptive permits boot to treat the files as "blobs" which means it will be less susceptible to format changes. Adding padding is the final backstop.
By moving all the kernel file writes out of the kernel we also have eliminated several classes of changes.
Things still shared between boot and kernel, and known by hypstart
- BIO
Things that used to be known by the kernel, now boot/hypstart/kfs-library only:
- SRT
Things that used to be known by the kernel, now hypstart/kfs-library only:
- freemap
- freshmap
Our strategy is to define the data structures now, then only implement the ones needed at this time and add new capabilities later. Specifically:
- Only implement 1 kfs using above strategy for gables, add "n" later.
- What is "need by" date for "individual part number" and "special binary"?
Boot Data Structure Layout
Term definitions:
- Storage Block : A physical memory range that might hold a hyperstart image or a kernel file system image, e.g., DEOS_FLASH
- Image : The memory contents of a hyperstart image or KFS. In target Endian.
- Image File: A file that describes how to place images into one or more storage blocks. Only place where compression can occur.
- Image File Location : The physical address where an image file is stored, this can be:
- A Storage Block, in which case the image is already loaded.
- Some other physical address. Boot, or possibly a kernel module, would be responsible for loading the image file into a storage block at boot time.
There will be several different kinds of image files, called "contentType" below:
- KFS
- Boot wouldn't know the format beyond the header, hypdump would have deeper knowledge.
- Hyperstart
- Compressed image
- Image reference
- An image file header with "load length=0", effectively only the destination address is meaningful.
- Composite Archive
- Describes how to populate several storage blocks
- For example, a hyperstart image and some KFSes.
- This permits a reference BSP to provide a testing base for multiple KFSes and as a feature demo.
- Suggestion: If Composite Archive contains hyperstart image file(s?), it/they will be first. TODO: plural?
- Selection Archive
- Provides alternate ways to populate an image, or select storage blocks.
- For example this would permit boot to implement multiple hyperstart images
- Depending on the content of a subsection this could include the means to populate the storage block, or just list the address.
- E.g., if the loadable length is zero, then an Image File Header would just be a pointer to the storage block.
Archives are a wrapper for a collection of other image files. The sub-sections of an archive would be other image files. Likely only the "top level" image file would be a selection archive and the hypBootIndex would be the selector.
Every image file starts with the following header:
Image File Header
label? comment? // 32 char array
contentType // The kind of the image file, e.g., Hypstart, KFS, Composite Archive, etc.
endian // big/little? DEO5?
version
destination storage block : physical address and length // ignored for (selection) archives
image file length // in bytes
integrity key : file offset, length, and type
some unused fields reserved for future, all zero initially.
number of sub-sections (number and meaning of each is defined by contentType)
array of :
offset (relative to Image File Header) to each (decompresssed) sub-section
length of the subsection
---- end of header / start of first sub-section
subsection content is here.
integrity key
A compressed image only has one sub-section. A compressed image file never becomes an image. I.e., the compressed image file header is never copied, the decompressed sub-section is copied in its entirety.
image operations:
validate integrity key
load(imageAddr, index):
composite archive
for each subsection:
// each sub-section must be an image file.
subsection_header = "image file header" + subsection offset
load(subsection_header)
selection archive(image, index)
subsection_header = "image file header" + subsection offset[index]
load(subsection_header)
compressed image
decompress into destination storage block physical address
load(destination storage block physical address)
image reference
load(destination storage block physical address)
otherwise: // kfs, or hypstart TODO:
kfs
same as hypstart
hypstart(image)
if (destination storage block physical address != &image)
memcpy(destination storage block physical address, &image, image.file length)
reference boot code:
load-hyp(well known address, index=0)
validate integrity of image at well known address
if (well known address.contentType == selection archive)
image-to-load = well known address.array[index]
else
image-to-load = well known address
load(image-to-load)
load(imageAddr) // must return a hypstart address. That logic not shown.
validate integrity key
switch imageAddr.contentType
composite archive
for each subsection:
// each sub-section must be an image file.
subsection_header = "image file header" + subsection offset
load(subsection_header)
compressed image
decompress into destination storage block physical address
load(destination storage block physical address)
image reference
load(destination storage block physical address)
otherwise: // kfs, or hypstart, boot treats "unknown" generically.
if (destination storage block physical address != &image)
memcpy(destination storage block physical address, &image, image.file length)
start deos(well known address) // start deos, enter kernel, something like that.
index = 0;
load-hyp(well known address, index)
while 1
enter kernel
if (index != boot interface.hypbootindex)
index = boot interface.hypbootindex
// This causes file data put to simulated flash to be lost from the "old" image
load-hyp(well known address, index)
Use cases:
- image file location != storage addr (copy/decompress, into simultated flash, or a "reference")
- image file location == storage addr (image in place)
- choose image to load ("golden image" + flight image), each could be either 1 or 2.
- image to load could be hyp, kfs, or hyp + 1 or more KFSes.
TODO: Should there be an ASCII label (aka SRT "header")? Might be useful as a part number field.
The above format lets the sizes and exact layout vary somewhat, and also provides some information for offline tools to parse the structure.
KFS Image
DRT for a file system containing an array of PartitionInformationBlock objects, the vasPtr for each entry is a "file" that contains the "directory" (an array of PartitionInformationBlock) objects that is the current list of files in the file system, and probably some other header info. The header for a DRT will permit variable sized and version information, etc. For all designs that don't need FLASH based KFS, the sizeof(SRT)+sizeof(DRT) will be < 1 page for KFS[>0], so the kernel will only support that size until we need it larger.
KFS
Image File Header DRT freemap freshmap? pages of storage making up the file system.
DRT
Like an SRT but the PDT defines a KFSDirectory, not an SVAS.
KFSDirectory (i.e., the existing *pib* entries that were in the PVA)
Max number of directory entries
Current number of active directory entries
Array of {filename, length, crc, user-data, etc}, sorted by filename.
Hyperstart Image
Hyperstart Image
Image File Header sub-sections: boot interface object "template" SRT Note: KFS root directory is in the PVA of the SVAS. freemap freshmap? ... (frozen pages are not part of any section)
SRT: (new fields marked with a '*')
* number of entries in the SRT
* size of each entry in the SRT
array of {SVAS PDT + crc info}
- The existing SRT stuff unaccounted for:
- header
- Stuff moved to image file header:
- load address (physical, used by loader software)
- totalFileSize
TODO: This is suspect: Each SVAS has a PVA, but the "new" PVA has all KFS related items removed. Removed items:
- FLASH start+length
- all references to *pib*
TODO: This is suspect: The hyperstart tool would map the VolumnLabel for the hyperstart image and KFS[0]'s KFSDirectory into the SVAS (presumably into the heap area) and put pointers to them into the PVA. The VolumnLabel is needed to compute physicalAddressInFLASH, and the directory is needed to find files. The kernel could map the KFSDirectory into the heap at runtime, but I don't see any value in that. I think the PVA thus needs a value for "kernel heap start", which would be after the mapped areas.
BSP Changes
- Add a "boot interface version" marker to the .hyp file.
- This could be filled in at config component build time based on the version of the dev-kit.
- This enables hypstart to know which version of the hypstart image to construct.
- DEOS_FLASH (or some canonical name) must exist as a platform resource.
- In the future multiple KFS resources may need to exist as well. Should we add any now?
Issues
- Should archives support be added now or later?
- Define what part of an image file is compressed.
- Any reason to have a storage block writer library expand Composite Archives?
- How to get fields for kernelFileSystemInfo()
Kernel Changes
Existing Code
Kernel updates:
remove a bunch of stuff.
code
deprecate status enums?
requirements
tests
update tests to use new library (are there any such tests?)
srtiface.h
Kernel no longer depends on this file. Still known by hypstart, hypdump, kfslib and Boot. kernelEntryFunctionType no longer accepts pdt Changes are on wiki.
bootiface.h
Address changes to 64-bit PDTAddr must now be initialized rather than passing to kernel entry. warmStartPDTcrc removed.
palintfc.h
functions being removed from PAL
flash routines
PALidleFunction
PALVideoMemoryAddress
config component must map the address
boot must clear video
PAL: qemus not sure, others likely delete the function.
kernintfc.h
testapiuk.h (changes made inline to that file).
Should the checkIntegrityAtStartup be moved to BIO? If so remove checkIntegrityAtStartupPtr from testapiuk.h
Kernel source patch:
Index: HAL/palintfc.h
===================================================================
--- HAL/palintfc.h (revision 58030)
+++ HAL/palintfc.h (working copy)
@@ -2,7 +2,7 @@
**MHdr-beg ***********************************************************************
**
** Copyright Honeywell Inc. 2009
- ** Copyright DDC-I, Inc. 2012, 2013, 2014, 2015, 2016, 2017, 2018
+ ** Copyright DDC-I, Inc. 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019
** All Rights Reserved
**
** Honeywell Proprietary
@@ -51,68 +51,12 @@
Preconditions: [Must match descriptions in pal.sgm]
- Must be called within a masterCritical.
*/
-DEOSBASEAPI PhysicalAddress32 DEOSPALPPI PALVideoMemoryAddress(void);
-
-
-/*
- See [PALSRD] for descriptions and definitions.
- Preconditions: [Must match descriptions in pal.sgm]
- - Must be called within a masterCritical.
-*/
DEOSBASEAPI uintData_t DEOSPALPPI powerLossDuration(void);
-/*
- See [PALSRD] for descriptions and definitions.
- Preconditions: [Must match descriptions in pal.sgm]
- - Must be called within a masterCritical.
-*/
-DEOSBASEAPI size_t DEOSPALPPI FLASHnumPagesPerBlock(void);
-
/*
-
See [PALSRD] for descriptions and definitions.
Preconditions: [Must match descriptions in pal.sgm]
- - Must not be called within a critical.
- - addr must be naturally aligned.
- - addr is non-cached
- - The underlying physical address of addr is within the updatable FLASH region.
- - every zero bit in *addr also contains a zero bit in newValue
- - may not be called concurrently with FLASHupdate32, FLASHwrite, or FLASHeraseBlock.
-*/
-/* TODO: [64-bit]: Need a FLASHupdatePtr(uintptr_t *addr, uintptr_t newValue); also. SRT writes use both Ptr and 32 types */
-DEOSBASEAPI BOOL DEOSPALPPI FLASHupdate32(uint32_t *addr, uint32_t newValue);
-
-
-/*
- See [PALSRD] for descriptions and definitions.
- Preconditions: [Must match descriptions in pal.sgm]
- - Must not be called within a critical.
- - addr must be page aligned
- - numBytes must be a multiple of bytes per page
- - newValue must be naturally aligned.
- - newValue must point to a RAM-based address.
- - may not be called concurrently with FLASHupdate32, FLASHwrite, or FLASHeraseBlock.
- - all bytes of FLASH from addr to addr+numBytes-1 are all ones and are non cached.
-*/
-DEOSBASEAPI BOOL DEOSPALPPI FLASHwrite(uint32_t *addr, /*const*/ uint32_t *newValue, size_t numBytes);
-
-
-/*
- See [PALSRD] for descriptions and definitions.
- Preconditions: [Must match descriptions in pal.sgm]
- - Must not be called within a critical.
- - The address must be a multiple of FLASHnumPagesPerBlock from the
- bootInterfaceObjectPtr()->DEOS_FLASHStart (see boot.h).
- - May not be called concurrently with FLASHupdate32, FLASHwrite, or FLASHeraseBlock.
- - all addressesin the block starting at addr must be non-cached.
-*/
-DEOSBASEAPI BOOL DEOSPALPPI FLASHeraseBlock(uint32_t *addr);
-
-
-/*
- See [PALSRD] for descriptions and definitions.
- Preconditions: [Must match descriptions in pal.sgm]
- Must be called within a masterCritical.
- Note especially the restriction on PAL device states described in [PALSRD]!
*/
@@ -195,13 +139,6 @@
DEOSBASEAPI timerValue_t DEOSPALPPI windowTimerTimeRemaining(void);
/*
- See [PALSRD] for descriptions and definitions.
- Preconditions: [Must match descriptions in pal.sgm]
- - None.
-*/
-DEOSBASEAPI void DEOSPALAPI PALidleFunction(void);
-
-/*
See [PALSRD] for description.
Preconditions: [changes must be reflected in pal.sgm ]
- Must be called within a kernelCritical.
Index: HAL/srtiface.h
===================================================================
--- HAL/srtiface.h (revision 58030)
+++ HAL/srtiface.h (working copy)
@@ -1,7 +1,7 @@
/*
** MHdr-beg ***********************************************************************
**
-** Copyright DDC-I, Inc. 2012, 2013, 2014, 2016, 2017, 2018
+** Copyright DDC-I, Inc. 2012, 2013, 2014, 2016, 2017, 2018, 2019
** Copyright Honeywell Inc. 2008
** All Rights Reserved
**
@@ -25,7 +25,7 @@
#include <bootiface.h>
#include <deosuk.h>
-typedef void (CDECL *kernelEntryFunctionType)(PhysicalAddressV /*PDTAddr*/, BootInterfaceStructTYP *, coreIndex_t logicalCoreIndex);
+typedef void (CDECL *kernelEntryFunctionType)(BootInterfaceStructTYP *, coreIndex_t logicalCoreIndex);
/* The SVASReferenceTable is essentially an ReferenceTableEntry array with some
miscellaneous header information. The layout of the System VAS Reference
@@ -47,12 +47,14 @@
zero when we're done with it (zero is a token that signifies an old
entry. We can never have an address of zero since DEOS_RAMStart is
required to be > 0).*/
- PhysicalAddressV svas;
+ PhysicalAddress64 svas;
/* The kernel entrypoint address. 32-bit based on DEOS-RAM-32-RESTRICTION */
/* Depending on the architecture this is either a virtual or physical address. */
kernelEntryFunctionType kernelEntryAddress;
+ uint32_t reserved0; // TODO: make sure this is initialized.
+
} CrcFieldsStructTYP;
/* Compile time check to ensure there are no gaps in CrcFieldsStructTYP. If the sum of the parts
@@ -68,7 +70,7 @@
CrcFieldsStructTYP crcFields;
/* The 32 bit CRC of the crcFields. */
crc32_t crc;
- DEOS64_ONLY(uint32_t reserved0;)
+ uint32_t reserved0; // TODO: make sure this is initialized.
} ReferenceTableEntryStructTYP;
/* Compile time check to ensure there are no gaps in ReferenceTableEntryStructTYP. If the sum of the parts
@@ -84,7 +86,7 @@
FLASH reclamations due to SRT running out of room are infrequent but
small enough to keep the FLASH foot print reasonable and keep search
times managable. */
-enum {srti_numSVASEntries = 1023 } ;
+enum {srti_numSVASEntries = 1023 };
/* The version string header is 32 chars long, and for 64-bit alignment purposes must always be a multiple of 8. */
enum {srti_headerSIZE = 32 };
@@ -91,6 +93,7 @@
/* The following structure is defined by DEBOOT. */
typedef struct
{
+ // image file header goes in here, instead of all this.
char header[srti_headerSIZE]; /* Not required to be null terminated. */
PhysicalAddressV loadAddress; /* Physical address where the hyperstart image file should be stored. */
/* PhysicalAddressV (i.e., 32-bits on Deos32) is ok because of DEOS-RAM-32-RESTRICTION */
Index: api/testapiuk.h
===================================================================
--- api/testapiuk.h (revision 58030)
+++ api/testapiuk.h (working copy)
@@ -3,7 +3,7 @@
/*
** HHdr-beg ****************************************************************
**
- ** Copyright DDC-I, Inc. 2012, 2013, 2014, 2015, 2016, 2017, 2018
+ ** Copyright DDC-I, Inc. 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019
** Copyright Honeywell Inc 2004
** All Rights Reserved
**
@@ -42,10 +42,11 @@
/* Declare "fun" first to match older IPAL instances where the only value
available was the colstart return address. */
bootInterceptorReturnToBootHook fun; /* KernelBootHookInfo_fun */
+ // Either this gets changed to a PhysicalAddress64, or pdt is moved to boot interface, and the "pdt" goes away.
PhysicalAddressV pdt; /* KernelBootHookInfo_pdt */
BootInterfaceStructTYP *bootObj; /* KernelBootHookInfo_bootObj */
coreIndex_t logicalCoreIndex; /* KernelBootHookInfo_logicalCoreIndex */
- DEOS64_ONLY(uint32_t reserved0;)
+ // DEOS64_ONLY(uint32_t reserved0;) when 64-bit is needed, re-layout this structure.
/* testData is unused by the kernel, it is for use by test software. */
void *testData; /* KernelBootHookInfo_testData */
} KernelBootHookInfo; /* sizeof_KernelBootHookInfo */
@@ -79,15 +80,15 @@
raceVariable_t *QuotaParentSecureRaceVariable2Ptr;
raceVariable_t *SemaphoreReadyAllWaitersRaceVariablePtr;
raceVariable_t *AttachedMemoryObjectDetachRaceVariablePtr; /* Used to test DDD_DEOS_SHMEM_625 */
- uintptr_t bootObjectPtrPtr;
- uintptr_t bootReturnAddressPtrPtr;
+ KernelBootHookInfo *bootHookInfoPtr// was bootReturnAddressPtrPtr; Need to resolve why this was a "**".
+ // uintptr_t bootObjectPtrPtr; // Now part of bootHookInfoPtr.
/* The address of the kernel entrypoint in virtual address mode, i.e., ::coldstart() */
- uintptr_t coldstart;
+ // uintptr_t coldstart; Don't think this is needed anymore.
BOOL *checkIntegrityAtStartupPtr;
/* The portal memory base address (i.e. PORTAL_BASE_PAGE_ADDRESS). */
uintptr_t portalBasePageAddress;
/* Pointer to the SVAS Reference Table. */
- SVASReferenceTableStructTYP *srt;
+ // SVASReferenceTableStructTYP *srt; // Shouldn't be needed anymore. Flash write routines will be removed. Any other use is suspect.
/* A pointer to the number of pages of system RAM quota remaining. Note: applications should use systemRamQuota().*/
pageQuota_t *systemRAMQuotaPtr;
/* The number of RAM/FLASH pages (page size in bytes is defined by getSystemInfoDEOS()->dwPageSize) needed
Index: bootint/bootiface.h
===================================================================
--- bootint/bootiface.h (revision 58030)
+++ bootint/bootiface.h (working copy)
@@ -207,20 +207,19 @@
Deos kernel. */
/* Page aligned address of first (lowest address) page of Deos RAM. Must be > 0. */
- PhysicalAddressV DEOS_RAMStart;
+ PhysicalAddress64 DEOS_RAMStart;
- /* Number of pages of Deos RAM (including non-allocatable pages). Boot
- coldstart alterable. The altered value must be <= the DEOS_numRAMPages
- value specified in the hypstart instance. */
- pageQuota_t DEOS_numRAMPages;
-
/* Flash block aligned address of first block of Deos FLASH. Boot coldstart
alterable. The altered value must be >= the DEOS_FLASHStart value
specified in the hypstart instance. Must be flash block aligned. Must be
<= hyperstart image load address. Must be consistently specified for every
cold start. */
- PhysicalAddressV DEOS_FLASHStart;
+ PhysicalAddress64 DEOS_FLASHStart;
+ /* Number of pages of Deos RAM (including non-allocatable pages). Boot
+ coldstart alterable. The altered value must be <= the DEOS_numRAMPages
+ value specified in the hypstart instance. */
+ pageQuota_t DEOS_numRAMPages;
/* Number of pages of Deos FLASH (including non-allocatable pages). Must be a
multiple of FLASH blocks. Boot coldstart alterable. The altered value
@@ -333,6 +332,8 @@
*/
uintData_t architectureSpecificConfigurationWord0;
+ // TODO: rearrange fields in this structure to be more sensibly grouped, and make corresponding changes to boot.h
+
/* ----------------- Members for internal use by the Deos kernel. */
/* Used by the kernel to track initialization sequencing during the early phases of startup.
Use is archetecture specific.
@@ -340,13 +341,16 @@
uintData_t bootSequencingWord;
/* The following is reserved for future use. Boot must set this to 0 every power-on coldstart */
- uintptr_t unusedBD[5];
+ uintptr_t unusedBD[5]; // TODO: Add more padding? Verify alignment.
/* ----------------- Members specified by the Deos kernel, read by Boot. */
+ // TODO: move this "up". Pay attention to alignment.
/* If Deos is warm startable, the Page Directory Table pointer to load. */
- PhysicalAddressV warmStartPDTAddr;
- crc32_t warmStartPDTcrc; /* the CRC of warmStartPDTAddr. */
+ PhysicalAddress64 PDTAddr; // TODO: This becomes "the" pdt for coldstart or warmstart
+ // TODO: Should put note that all cores would have to wait on the kernel
+ // return before any could start the next boot cycle, otherwise readers of the
+ // pdtAddr could get confused.
/* ----------------- Members for internal use by the Deos kernel. */
DEOS64_ONLY(uint32_t reserved0;)
@@ -356,6 +360,7 @@
uintptr_t testValue2;
uintptr_t testValue3;
uintptr_t testValue4;
+ // TODO: Add more here.
/* The core index of the core that is currently booting. After all cores have
booted, this will be equal to the number of cores. This is used as a cross
@@ -367,7 +372,7 @@
Must be able to execute instructions. */
uintData_t kernelScratchpad[15];
-}BootInterfaceStructTYP;
+} BootInterfaceStructTYP;
/* Return the state of the booti_FLAGS_debugServicesHonoredRequestedBit (i.e. are we requested to set
libkfs
- cleanup
- moveKernelFile
- deleteKernelFile
- "simulatedflash" is ok for first iteration.
- kernelFileSystemInfo
- kernelFileSystemInfoEx
- create kfslib.cd.xml
- create UG for kfslib
- tests?
- update ftp and TDL.
- link against new library
- update their cd.xml to depend on new library
- figure out name of the library.
