OA supports KFS
OpenArbor features to support KFS
- For Desert Eagle D7c delivery (shipped on 1/16/2020): command line support only. ToDo (Kelly): send them the instructions Richard created.
- For Durants D5 delivery (code complete by 1/28/2020): create KFS, assign resources to KFS, composite is created with all KFS integrated into platform, right click and bifdump a KFS. Add a checkbox for "bifdump" to allow user to specify as part of the build
- Rfrost@ddci.com (talk) 20:50, 20 January 2020 (UTC) Is bifdump the name we want in the IDE? Should this be Dump Boot Interface File or something similar instead? bifdump at the cmd line is fine.
Tasks
| ID | PCR | Item | Priority | Assignee | Phase | Category | Comments |
|---|---|---|---|---|---|---|---|
| 1 | DDCI_PCR:3913 | Minimal File Systems tab's list box columns to be "Owned File System Name", "Storage Block Address", "Automount Path" | 1 | LCJ | Stable | BuildInfra | MFS' are created by definging a platform resource and owned file system. The platform resource (memory mapped resource name) is the MFS name. |
| 2 | PCR:13561 | The pi tool -k option needs to additionally report the automountPath for an owned file system in the json file. | 1 | GK | Stable | BuildInfra | The automount path setting should be displayed in the Minimal File Systems tab list. |
| 3 | DDCI_PCR:3913 | Document procedure to reserve physical memory for an MFS (automated later) | 1 | NH | Dev | BuildInfra | This will not automated for the upcoming release. The steps for creating an mfs be documented in the OA User Guide. Assumption for the short-term (and possible long-term?): all the xml updating to specify the platform resources for KFSen, and to set the search path and desired mounting for processes, will be the user's responsibility -- manual xml editing. |
| 4 | DDCI_PCR:3913 | Update loadlist.txt (and loadlist-absolute-paths.txt) to be prefixed with 'HYPSTART_' | 1 | LCJ | Stable | BuildInfra | |
| 5 | DDCI_PCR:3913 | LoadListManager compareFiles(DeosFtpWrapper service) should compare files for all MFS, not just hyperstart. | 1 | LCJ | Stable | BuildInfra | |
| 6 | DDCI_PCR:3913 | Add list(String path) function to DeosFtpWrapper() | 1 | LCJ | Stable | BuildInfra | List files for an mfs to use when comparing files. |
| 7 | DDCI_PCR:3913 | For debugging with MLD, the loadlist passed to MLD should be a complete list of all the MFS loadlists (not just HYPSTART). | 1 | LCJ/JK | Stable | BuildInfra | |
| 8 | DDCI_PCR:3913 | For debugging with Gdb, gdb-target-files directory should contain a copy of all the files from all the loadlists (all mfs') | 1 | LCJ | Stable | BuildInfra | |
| 9 | DDCI_PCR:3913 | Convert existing workspaces <kernel_FileSystem> attributes to the new format. | 1 | LCJ | Stable | BuildInfra | From: <kernel_FileSystem>TFS1,SCATHitmap,0x0000000040004000</kernel_FileSystem> To: <kernel_FileSystem>SCATHitmap,0x0000000040004000</kernel_FileSystem>. The automount path will be updated when the pi tool with the -k option is executed when loading the table. Project's .options files that have a <kernelFile kfs=<old mfs name>... </kernelFile> will have to be updated to the new mfs name. |
| 10 | Update ftp/vfile/kfslib to support MFS directory listings | 1 | CP | Stable | BuildInfra | be able to list files in the old familiar legacy KFS, the hypstart one, and also what files are in the other KFSen, so we can tell if there are files there that need to be updated or added or deleted. OA will need a list of KFS'. OA will need ftp to provide a listing of a specified KFS. Currently IT tool gives some indication for building. | |
| 11 | PCR:13564 | Network to automount and own all memory mapped MFS resources | 1 | GK/LCJ | Stable | BuildInfra | GK to generate a file, LCJ to use the file |
| 12 | Support for MFS' when there is no loadlist support | 99 | TBD | BuildInfra | ** To support the debuggers in the "no loadlist" scenario OA will need a way to query the target and ask what is the search path for a given process, including the details on the platform resources where its KFSen are mounted, |
''Legend''
ID: Just a number to be able to quickly refer to an item during a meeting, etc.
PCR: Link to the PCR is one has been written or TBD if there is no PCR
Item: Short description of task to be worked
Priority: A number 1 - 99; 1 is a must have for April; anything greater than 1 can be deferred
Assignee: The person responsible for doing the work associated with the component.
Phases:
- Blank - Not yet started
- Dev - In Development
- Test - Component unreleased and ready to test
- Delta - A Delta baseline must be established
- Audit - One (or more) SQA Audits must be performed (e.g., a Verification and/or Software Conformity Audit)
- Stable - You are done! All steps (including test report) in the Deos Software Release HowTo or OpenArbor Development HowTo have been followed.
Category: One of the following: BuildInfra || DockerImage || DeosComponent || OpenArbor || TestInfra || WorkStep || TBD
Comments: Free form text.
Customer Use Cases
OA MFS Use Case AL01
Create / delete MFS
OA will provide (eventually) an interface to create and delete MFS file systems. The end result for a new MFS is:
A platform resource. How to get the start address is TBD. Nominally adding an MFS will require updating either the settings for Deos RAM pool 0 range 1, or perhaps some other more sophisticated means. Discussion of said sophistication ongoing between AL and BSP team.
<memoryMappedResource
name = "kfs-01"
startAddress = "0x4000"
lengthInBytes = "0x10000"
accessRights = "r"
allowMultipleOwners = "yes"
cacheMode = "writeBack"
resourceOverlapList = ""
>
</memoryMappedResource>
When an MFS is created an option should be provided to select which (or all) processes the new file system should be automounted:
<ownedFileSystems>
<ownedFileSystem
resourceName = "kfs-01"
automountPath = "/maps"
></ownedFileSystem>
Any such processes should also get an increase in mountQuota.
All state updates should be stored in .xml files, e.g., .cd.xml, .pd.xml, .pi.xml, or perhaps some dog awful .fp.xml.
Q) [lcj] Will the other manual steps be automated by the create MFS tool? The OA User Guide has several more manual steps that are not mentioned above:
In the application project, add a dependency on 'ansi' In the application project's .pd.xml file
add a usedFeatureSet entry for libansi in the processTemplate, set the mount quota to 3 in the processInstance, set the fileSystemSearchPath
In the platform project's .pi.xml fle:
modify the memoryMappedResource's allowMultipleOwners=yes
Assign files to an MFS
OA already does this and stores the result in a .cd.xml file.
Integrate application into a platform
Presumption is no OA support needed for integration. Said integration will be provided by integ tool.
Detect when an MFS loaded onto a target needs to be updated
Modify the ftpserver.pd.xml (or more likely lwip.pd.xml, and possibly via a .fp.xml) so that ftp/lwip has mount quota and appropriate ownership for all memory mapped resources containing a file system, all automounted as /mfses/KFS_PLATFORM_RESOURCE_NAME, where KFS_PLATFORM_RESOURCE_NAME is the value of the memoryMappedResource name attribute. When OA wants to know what the contents of a KFS are, it get a dir listing of /mfses/KFS_PLATFORM_RESOURCE_NAME. It will use normal load list management tricks to discover if there are any changes (differing integrity keys, or list of files). If there are differing files, then the make(1) generated .dkfs file for the file system should be FTP put to /proc/res/KFS_PLATFORM_RESOURCE_NAME.
Note that the download mode .pi file will have to specify "rw" for the memoryMappedResource accessRights.
It is possible that in a future release Deos will support putting/deleting individual files in an MFS, but currently that capability does not exist.
LFS file system updates should work as before.
Debug application in one or more mfs'
For MLD: create a loadlist.txt style file containing kernelFile= lines listing the non-directory target name and the fully qualified host path name for all files in the LFS and all MFS file systems.
For GDB: create a directory containing the contents of each file system (LFS and all MFSes) where the directory contains all the host files (symbolic links would be acceptable on linux and may be acceptable on Cygwin). When invoking GDB specify the list of such directories as the directory search path.
It would be a good idea at some point to verify that there are no duplicate file names across the LFS and all MFSes.
Design notes
2021-06-17 telecon
- Need ability to identify all the mount points for all processes.
- Probably a new output from "pi -k".
- Suggested constraints:
- All mount points across all processes must be consistent:
- Not all processes need to mount all file systems.
- Executable image files (.exe or .so) must be unique in all file systems.
- This constraint is imposed by OA, partly because of the way image building is performed, but mostly to simplify building debugger config files.
- Need to note in OA documentation that kernel does not have these restrictions.
- FTP will mount all file systems as /RESOURCENAME or similar and ignore the actual mount points.
- How to detect changes in addresses of MFS resource.
- Need to document how to create MFS resources.
- Newer BSPs have deos ram pool 0 range 0 small, and add pool 0 range 1 as "rest of RAM". This simplifies the allocation of physical memory.
- To make job even easier, it would be nice to have some math/special tokens in platform resources (e.g., "b immediately follows a", or perhaps "follows after alignment to N").
- FTP (LwIP) needs to own all platform resources and automount all file systems.
- This enables ftpserver to be able to show the integrity keys of all files.
- Add ability to write to MFS.
- This reduces complexity of OA, and improves download performance.
KFS Use Case meeting with Durants occurred on 1/31/2020
- OA knows the same dependencies as the registry: Currently, there's a switch in the integ tool that provides the list of MFSs to OA
- Updated loadlist capability:
- OA provides a file structure view of the MFSs: Currently the bifdump tool provides the list of files
- OA identifies files that have changed in an MFS, to enable the user to reload the changed files
search paths are currently hard coded, but should be able to support runtime updates from the user
Aviage: Demonstration of current capability from Mike H
The example workspace and video provided by MH can be found here: \\nx3000\openarbor\Integration\mfs
video: mfs-example.mp4
example workspace: mfs-exampe
