OpenArbor cdproc

cdproc Info

Look here (Cdproc) for information about cdproc. There is also a teams chat

Goal

The overall goal is to have OpenArbor's generation of makefiles in java code be replaced by executing cdproc-makefile for building platform projects. Application projects are not part of the initial scope. This seems, initially, to be a daunting task. A phased approach might be more doable. So the initial plan is to replace OpenArbor's generation of the mfs-config.mk with the execution of cdproc-makfile to generate the mfs-config.mk file.

Release Notes

2025-08 Release

1. Cdproc will generate a platform project's project-config.mk file.

2025-04 Tsunami

1. Selection of component variants is now handled only in the Platform Project "Complete Integration" dialog. Specifically:
   1. Executable (and metadata?) projects no longer have an "active variant".
   2. The development-support compoonent is no longer included by reference when a new platform project is created. Instead it is copied by inclusion.
      • This was done to resolve the prior problems associated with attempting to select the Production variant of the kernel or LwIP.
   3. Executable projects now provide an explicit list of the variants they supply.
2. The set of registries defined in a platform is now controlled by the "registries" component, which defaults to platreg and platregd for backward compatibility.
3. There is a new MFS wizard to assist with the generation of MFS kernel file systems.
4. Building of MFSs no longer generates spurious warnings.
5. Files included in a platform project now exclusively only include ones from the output/desk directory of referenced projects.

Tasks

Issues spanning teams:

1. ABC 7.0.0 problems
2. How should the OA "Boot Image File Generation" Deos platform property dialog affect cdproc?
   1. How is makeboot handled for various platforms that support it?
3. OpenArbor's componentDescriptor.xsd needs to be updated to include bootModuleFile.
   1. See: https://deos.ddci.com/scm/Deos/products/desk/cdproc/branches/mainline/code/openarbor-content/
4. Still need to finish:
   1. IOI
   2. IST
   3. 653
   4. pciconfig
   5. deosname
   6. MFS/LFS handling.
5. cdproc modular boot support is incomplete
   1. How should boot modules appear in the GUI?
6. Will we need a project upconvert?
7. Will soon need testing support.
8. AL to review all platform project OpenArbor property pages with OA team to make sure all "menu" items are implemented.
9. platform project output dir contains a qemu-aarch64.cd.xml file (generated by OA?) and the cdproc generated project-config.mk references it via an absolute path.

OpenArbor Tasks

Post August

OA Code Cleanup

Improve change notification handling in OA. Don't ignore change notifications for files in a dependent project's output/desk folder (ProjectChangeListener), and cleanup resourcesChanged() in DDCIProject.
DDCI_PCR: 5464
DDCI_PCR: 5471

Files Lists in OpenArbor GUI

Not sure what they are called, but in OpenArbor under "Complete Integration" and "Deos Component" there are lists of files contained in some of the cd.xml elements. It would be nice, i.e., not critical for August, if there were lists for bootModuleFile and bootFiles.

We previously discussed that, perhaps, cdproc should provide a list of the elements that are "public" and that OA could then list all such lists in a generic fashion. If this is the preferred mechanism, and it is non trivial to get the two specific lists added, then we might consider deferring this request until the next DDS release.

Specify composite archive elements

Have a field for "KFSs to be included in Composite Archive". It could even be auto populated if someone creates MFSs in the workspace.

August

loadlist-absolute-paths

Done

OA adds this to output/Makefile:

 FLAGS.elfchk += -complete -fileListFile=$(PROJECT_DIR.qemu-aarch64)/output/loadlist-absolute-paths.txt
 all: qemu-aarch64.elfchk

But there is no rule to ensure that the loadlist-absolute-paths.txt exists before elfchk is run. Aaron believes this is a longstanding latent defect.

Instead suggest adding to output/Makefile:

 FLAGS.elfchk += -complete -fileListFile=loadlist-absolute-paths.txt
 qemu-aarch64.elfchk : loadlist-absolute-paths.txt
 all: qemu-aarch64.elfchk

postIntegrationElfchkProperty

Done

The postIntegrationElfchkProperty element is being added to platform's cd.xml, but there is no postIntegrationElfchkProperty in the componentDescriptor.xsd

Since this is being handled by OA in the project output/Makefile, there is no need for this element in cd.xml.

obsolete FLAGS.deos_653_xml

OpenArbor generated Makefile FLAGS.deos_653_xml specification of "-p 1" is no longer required.

bootModuleFile

Done

Need to add bootModuleFile element to OA's componentDescriptor.xsd and DEOS_PLATFORM_CORE.xsd files. Or alternately have OA start using the xsd provided by cdproc (in /desk/etc/xsd)

See https://ddci.zapto.org/scm/Deos/products/desk/cdproc/branches/mainline/code/openarbor-content/componentDescriptor.xsd.proposed

Selective Build

Done

Selective build adds an "ignore" attribute in .options which causes, for example, FILES.platreg to be commented out.

Aaron suggests we remove this capability from platform projects. Not sure who needs to be consulted.

[lcj 25-jul] Spoke with RLF and the Selective Build tab can be removed from platform project options.

names.dat in cd.xml

Done

OpenArbor is adding kernelFile names.dat to the .cd.xml generated in platform project's output/desk. This is no longer required.

Boot Image pane

Done

Remove the command boxes and change the labels:

 Hypstart filename: current default
 Makeboot bif_in:   current default

bifdump

Done (OA 13.1.0.r110744) lcj

Boot Image pane

execute bifdump causes changes to project-config. Instead can this be done in project output/Makefile?

On second thought, is this a development environment issue, or a formal build need?

There is also an executeBifdump element added to platform's cd.xml, but there is no executeBifdump in the componentDescriptor.xsd

Since this is being handled by OA in the project output/Makefile, there is no need for this element in cd.xml.

# all : boot <--- move this line to output/Makefile and have it commented/uncommented by the Generate Boot Image checkbox

# boot : bifdump <--- add this line to output/Makefile and have it commented/uncommented by the Generate bif dump checkbox

validate xml in place

Done (OA 13.1.0.r110744) lcj

Integration Tool pane

Validate XML in place is an OpenArbor concept. Currently OA causes changes to the project-config.mk file. Instead can this be placed in the project output/Makefile?

There is also a validateinplace element added to platform's cd.xml, but there is no validateinplace in the componentDescriptor.xsd

Since this needs to be handled by OA in the project output/Makefile, there is no need for this element in cd.xml. Move the # Platform project Validate in place: lines to output/Makefile

Change default bifdump to false

Done (OA 13.1.0.r110824) lcj

Execute bifdump currently defaults to "yes". This is a poor default for QEMU. Suggest the default be changed to "no".

Enable copy basecon.makeboot and .hyp files

bootimage.hyp is obsolete. It could be remove completely if basecon.makeboot and basecon.hyp files could be copied from the /desk into the local xml directory.

cdproc-makefile PROJECT_DIR not PROJECT_DIR.proj-name

Done (OA 13.1.0.r110824) lcj

When OpenArbor calls cdproc-makefile the command:

 cdproc-makefile '--var=PROJECT_DIR.qemu-aarch64

should be:

 cdproc-makefile '--var=PROJECT_DIR=

Rationale: The PROJECT_DIR.proj_name variables expand into ../../proj_name instead of "..", however the COPY_FILE_IF_CHANGED rules use $(PROJECT_DIR). The difference causes make rules to not match. At least those references need to match.

Duplicate COPY_FILE_IF_CHANGED rules

Done (OA 13.1.0.r110824) lcj

OpenArbor is creating output/Makefile rules that have duplicate targets:

   platreg-pia-xml-to-update += $(PROJECT_DIR)/xml/basecon.pia.xml
   $(eval $(call COPY_CHANGED_FILES,$(platreg-pia-xml-to-update),validated-xml.platreg,all))
   ...
   platregd-pia-xml-to-update += $(PROJECT_DIR)/xml/basecon.pia.xml
   $(eval $(call COPY_CHANGED_FILES,$(platregd-pia-xml-to-update),validated-xml.platregd,all))

The highlighted text generates two rules with the same target, which confuses make(1).

install-rule.mk update

Done (OA 13.1.0.r110824) lcj

The OpenArbor/openarbor-common copy of the install-rule.mk file needs to be updated. AL: needs to fix a defect in the build-utils copy of said file, then OA should get the updated copy.

Update documentation

Update OA's component descriptor documentation to refer to cdproc's version.

June

ID	Task	Who	Estimate	Status	Comments
17	Support Automated tests for Test Cases	KT	40
	Total		@sum(column)

cdproc-makefile Tasks

ID	Task	Who	Estimate	Status	Comments
19	Review diffs of OA generated project-config.mk / cdproc-makefile Makefile	??	0	Done	TODO: pciConfig support: TODO: copy for edited files ignored (e.g. basecon-resources.pia.xml) OA magic for ioi-api and ring-buffer cds and libioiapi-release.fp.xml various other TODOs in cdproc generated Makefile
20	Create an mfs example to be delivered in /desk/example		0	Deferred
22	Support for multiple files (e.g. exe's) with the same name in different mfs' (see PCR:16422 for details)	AL/GK	0	Deferred	OA test: com.ddci.openarbortests.deos.MinimalFileSystem.multiple-exes
23	Complete cdproc-makefile for project-config.mk	AL/GK	0	Done	At least IOI, pciconfig, vfile, dvms, a653, hypstart, deosname, and new boot.
24	Support for mulitple remote targets	AL	0	Deferred	OA test: com.ddci.openarbortests.deos.DeosGdbLaunchConfigurations.deosGdbMultipleRemoteTargets
	Total		@sum(column)

Test Failures

2025-08-07

Started informal testing of all QEMUs, and adding IMX8QM results here: OpenArbor_13.1.0_Test_Plan_and_Informal_Report

2025-07-23 (zhangmen dds with unreleased oa and desk-cdproc)

ID	Task	Who	Estimate	Status	Comments
1	*** FAIL com.ddci.openarbortests.compileRunDebug.Compile.demo653	LJ	0	Done	re-run with names.dat patch. [jul-29 lcj] re-ran test and fails as expected due to warnings [lcj 31-jul] test failed. Error: No rule to make target 'platreg/ioi/platreg-demo_config-ioi.cfg', needed by 'desk/platform/qemu-aarch64/etc/platreg/platreg-demo_config-ioi.cfg' There is no ioi config file for 653 projects unless the 653.xml file contains sampling or queueing ports. The test should NOT be trying to install the *-ioi.cfg file, or the companion .fp.xml file. [lcj 04-aug] Re-ran test
2	*** FAIL com.ddci.openarbortests.compileRunDebug.MulticoreIntegration	AL	3	Done	On qemu-aarch64, after modifying bootimage.hyp -maxCoreIndex=3, and basecon.pia.xml numberOfCores = "4", an update target load results in "Waiting to boot core index 1 last is 3" message displayed on the emulator. Is that expected? Did a hypdump on hypstart.dfks and saw that coreIndex was 3. AL has patch, Tests should now pass, but see #Enable_copy_basecon.makeboot_and_.hyp_files for a (deferable) preferred solution. [lcj 31-jul] Test passed
3	*** FAIL com.ddci.openarbortests.compileRunDebug.Run.vfile_logbook_partition1	AL	0	Done-ish	video display missing content run vfile or 653 example. sudo -E apt-get install deos653-dvms new example deos653/vfile_logbook_workspace Result is: dvmsInitialize ERROR sent message to Chris Pow. [lcj 31-jul] Test failed Error: No rule to make target 'platreg/ioi/platreg-config-ioi.cfg', needed by 'desk/platform/qemu-aarch64/etc/platreg/platreg-config-ioi.cfg'. Stop. See "no ioi config file" description in #1 [lcj 04-aug] Test failed. Done: Passed was not displayed in the video stream as expected Works on qemu-arm. This is a known component error. See DDCI_PCR:5281 attachment.
4	*** FAIL com.ddci.openarbortests.compileRunDebug.SpacesInPaths	AL/LJ	0	Done	determine reason for test failure rerun test on OALinux02 reran test on jul 25 and it passed
5	*** FAIL com.ddci.openarbortests.compileRunDebug.SuperMakefile	AL/LJ	0	Done	make install from workspace level, outside of OA, is getting undefined var WORKSPACE_DIR. : with ubuntu-kismet-dev:2025-07-25, create platform project qemu-arm, exit OA, then at workspace level doing the following does not cause any warnings: make --warn-undefined-variables reran test on jul 28 and it passed
6	*** FAIL com.ddci.openarbortests.compileRunDebug.VfileIntegration	LJ	0	Done	reproduce the failure updated and reran test on jul 29 and it passed.
7	*** FAIL com.ddci.openarbortests.deos.CopyForEditing	LJ/AL	0	Done	This is due to copyToAppbin and copyToEtc issue. LJ to move the exported files rules to output/Makefile. Change committed, test pending [lcj 31-jul] test passed. [lcj 4-aug] Test failed AL guessing this is export of copy for edited files [lcj 5-aug] Test passed
8	*** FAIL com.ddci.openarbortests.deos.DeosGdbLaunchConfigurations.deosGdbMultipleRemoteTargets	AL	0	In Work	The test is attempting to modify the Makeboot command: textbox to add "-tap-host-ip-mask=192.168.19.16/31", but the Makeboot command: textbox has been removed. See item 2 for an alternate solution [lcj 30-jul] Need more discussion on what the test should do, and what should be done for the alternate solution. AL needs to chat with AR.
9	*** FAIL com.ddci.openarbortests.deos.ExternalDebugging.noLoadList	AL/LJ	0	Done	presume this would be fixed by wiki task #loadlist-absolute-paths [lcj 25-jul] These test the functionality of 'no load list' which means no platform project. So the presumed fix does not apply for this functionality. [lcj 29-jul] Updated and reran tests, passing now
9.5	*** FAIL com.ddci.openarbortests.deos.ExternalDebugging.noLoadListMfs	AL/LJ	0	Done	[lcj 29-jul] Updated and reran tests, still failing [lcj 31-jul] Test passed
10	*** FAIL com.ddci.openarbortests.deos.MakeExport.photon_integrated	LJ/AL	0	Done	Needs the exported files rules (#7) [lcj 31-jul] Test passed [lcj 4-aug] Test failed AL guessing this is export of copy for edited files [lcj 5-aug] Test passed
11	*** FAIL com.ddci.openarbortests.deos.NewProjectValidation.checkNonProjectFolderInWorkspace	LJ	0	Done	WORKSPACE_LOC undefined reran test on jul 28, and it's still failing [lcj 29-jul] Problem identified, awaiting OpenArbor code change [lcj 30-jul] AL committed a fix [lcj 31-jul] Test failed [lcj 1-aug] Updated test. Test passed
12	*** FAIL com.ddci.openarbortests.deos.Sybil	AL/LJ/GK	0	Done	probably need co-debug. [lcj 30-jul] ioiConfig: Error #1: Produce item formatting (NormalCounter) for process (partition1) uses invalid formatting function name (libhybrid653.so:writeSamplingPortInIOIForm) ioiConfig: Error #2: Produce item formatting (NormalCounterWithCRC) for process (partition1) uses invalid formatting function name (libhybrid653.so:writeSamplingPortWithCRC) [lcj 31-jul] GK is helping to debug the issue. AL fix committed to cdproc. [lcj 31-jul] Test passed
13	*** FAIL com.ddci.openarbortests.other.PerformanceTest	LJ/GK	0	Done	Aaron to talk to GK., test is assuming the existence of platreg-demo_config-ioi.cfg Since there are no sampling or queueing ports in existence there are no IOI files, in previous versions of OA, there would have been a dummy IOI CFG file, we believe the test is still assuming the dummy file exists (i.e., the old behavior), and the test must be updated to reflect the fact that there will be no IOI file created in this situation. [lcj 31-jul] test updated, need to rerun test. [lcj 31-jul] Test passed
14	*** FAIL com.ddci.openarbortests.projectOptions.GccLinkOptions.cpp_outputs	LJ	0	Done	--whole-archive not being removed? [lcj 25-jul] reran test and it passed
15	*** FAIL com.ddci.openarbortests.projectOptions.other.DeosOptionTabs.bootImageTabTest	AL/LJ	0	Done	Known failure. Missing cdproc feature has been implemented [lcj 30-jul] Test updated. Needs to be rerun. Requires fix for 4.1.1.10 [lcj 31-jul] Test passed
16	*** FAIL com.ddci.openarbortests.projectOptions.other.DeosOptionTabs.integrationToolTabTest	LJ/GK	0	Done	Need to debug. Lisa to talk to GK: failed to have evidence of hello-world.ai.xml file being validated. [lcj 31-jul] Test passed.
17	*** FAIL com.ddci.openarbortests.projectOptions.other.DeosOptionTabs.publicHeadersTabTest	LJ	0	Done	[lcj 30-jul] Test passed
18	*** FAIL ccom.ddci.openarbortests.other.MinimalFileSystem.multiple-exes	AL	0	In Work	[lcj 30-jul] See item #22 in cdproc-makefile Tasks table
19	*** FAIL com.ddci.openarbortests.deos.IntegrationDetailsViews	AL/GK	0	In Work	[lcj 1-aug] Test failed. The test adds the platform project's .options file to the load list. Creating HYPSTART_loadlist.txt make[1]: *** No rule to make target '.options', needed by 'all'. Stop. Is that no longer permitted? [lcj 04-aug] Test Failed AL ran the example. .options is added and that part of the test passes. The FAIL is for names.dat, but that file is also on the target. Will need help determining what the failure is. [lcj 5-aug] Test failed. names.dat is not being 'exported', and the other exported files should have the exported dir path in the kernel files list (not $(OUTPUT_DIR))
20	*** FAIL com.ddci.openarbortests.deos.StatusMonitorDebugShouldNotUpdateTarget	LJ	0	Done	[lcj 1-aug] Test failed. The test adds the platform project's .project file to the load list. Creating HYPSTART_loadlist.txt make[1]: *** No rule to make target '.project', needed by 'all'. Stop. Is that no longer permitted? [lcj 04-aug] Test passed
21	*** FAIL com.ddci.openarbortests.other.MinimalFileSystem.mms-configuration	LJ	0	Done	[lcj 1-aug] Test failed. cdproc-makefile: Warning: Assuming MFS-IOI.kfs.xml will be built cdproc-makefile: Error: No variable 'OUTPUT_DIR' referenced by $(OUTPUT_DIR)/MFS-IOI.kfs.xml AL: Lisa, I'm not certain, but I think this is because the project_dir is not in a --directory. I.e., I think this will be fixed with the latest OA change. [lcj 04-aug] Test passed
22	*** FAIL com.ddci.openarbortests.other.RenameProjects	LJ	0	Done	[lcj 1-aug] Test failed. Test update needed, passed locally [lcj 5-aug] Test passed
23	*** FAIL com.ddci.openarbortests.compileRunDebug.SelectiveBuild.selectiveBuildForFiles	KT	0	Done	[lcj 1-aug] Test failed. Test update needed. [lcj 5-aug] Test passed
24	*** FAIL com.ddci.openarbortests.compileRunDebug.SelectiveBuild.selectiveBuildForFiles	KT	0	Done	[lcj 1-aug] Test failed. Test update needed. [lcj 5-aug] Test passed
25	*** com.ddci.openarbortests.gdb.GdbVariables.Deos653	LJ	0	Done	[lcj 1-aug] Test failed. Reproduce the problem. Error: No rule to make target 'platreg/ioi/platreg-dbg-deos653-config-ioi.cfg', needed by 'desk/platform/qemu-aarch64/etc/platreg/platreg-dbg-deos653-config-ioi.cfg'. Stop. See "no ioi config file" description in #1 [lcj 04-aug]Test fails as expected
26	https://deos.ddci.com/svn/DDCI/support/zhangmen/2425-project-layout-example/	LJ	0	In Work	[lcj 04-aug]Test this customer workspace
27	All rtems-653-configuration tests	LJ	0	Done	[lcj 04-aug]Test failed integrating rtems-653-configuration into a platform project results in ioiConfig reporting 24 errors cdproc unrelease awaiting test results [lcj 06-aug]Integrating rtems-653-configuration into a platform project builds without errors.
	Total		@sum(column)

Questions and Concerns

OA Component Descriptor xsd

AL>In the OA componentDescriptor.xsd there are some things that should be fixed:
The attribute "component" applies to many types (and thus to many elements). "component" should be specific to the "selected-variant" element.

The element "selected-variant" currently has the attribues "component" and "optional".
selected-variant's attributes should be:

<attribute name="component", type="string"/>
<attribute name="targetArch" type="tns:targetArch"/>
<attribute name="variant" type="tns:variant"/>

I am explicitly excluding "registry" since I do not know how to have registry specific variants in the same KFS.
I am also excluding "optional" because I can't think of what "optional" would mean in this context.

The above allows, for example, the lwip.cd.xml to contain:

 <selected-variant variant="diagnostic" component="ansi">instrumented</selected-variant>

Thus selecting the debug variant of lwip, would also select the debug variant of ansi.

I'm ok with OA not implementing the selected-variant semantics of targetArch and variant. cdproc does that now, and cdproc will supply that semantics to OA in the future.

Attribute "kfs" should be eliminated from all elements.

I'm not sure which elements should support "optional". "optional" currently applies to many elements. Most I could imagine some use, but not for parentPlatformName and registry. If the original intent was that "optional" only apply to "depend", I'm also fine with restricting "optional" to just the "depend" element.

Platform Projects

• openarbor.options vs cd.xml to define platforms
  • A BSP is something that is a minimal configuration of items required for the platform. Being able to edit the BSP is not a good idea.
  • Currently an openarbor.options file can define a parent platform. Needing to edit two openarbor.options files seems bad.
  • Perhaps when creating a platform project, the person should specify a new name then depend on the platform.

• How is the LFS content described?
  • Is it in hypstart.kfs.xml?
  • Does a bsp provide that file?

AL> cdproc integration with MFS+hypstart is currently not implemented. I had a request for comments out to the group for the kfs.xml schema and never got back to it. I forget exactly where I left off, but I think default logic is in place for the LFS/hypstart. I'm not sure which component should (eventually) provide the LFS' kfs.xml file, and whether it should have a hypstart-like name, or if it should be called basecon.kfs.xml, or ???. My expectation is that its content would include:

 <kernelFileRegex scope="default">.*</kernelFileRegex>

AL> The most immediate issue is that I wanted to ensure we could test cdproc-makefile against OA's generated makefiles (ideally after OA started sorting the lists in the generated makefiles) so we could detect errors before making cdproc content changes.

AL> Then there are a variety of related issues, e.g., can we change deos-name to have a names.dat kernel file? Does that break OA? How do we specify that a component's kernelFile specification is for a file that is expected to be generated, e.g., the aforementioned names.dat or vfile-config, etc. It probably is easier to answer those questions if we don't have to update both cdproc and OA, or alternately that we do so at the same time. Like GK noted, there are a few TODO items remaining.

• Does a platform project contain one kfs.xml file for each mfs?

  AL> There is one kfs.xml file per MFS. Where the xml file lives is TBD, but currently unconstrained. I'm not wedged on that answer though.

  • Is this correct? kfs.xml files must reside in a platform project (nowhere else)?
• Is there a tool for processing a .kfs.xml file?

  AL> Currently cdproc handles them directly. Note that cdproc-query reports where it finds files, including kfs.xml files.

  • For other xml files (e.g. .pd.xml, .fp.xml) there is a corresponding invocation of a tool (e.g. pd or fp) to validate the file.
  • Is there something similar for .kfs.xml files?

    AL> No. My mental model of .kfs.xml files is that they are more like .cd.xml files than integration tool files.

NOTE: In OA 12.5.0, basecon-resources.pia.xml's validated xml is not being copied back to platform project's xml/basecon-resources.pia.xml (when copied for editing).

In OA 12.5.0 a platform project has several generated makefiles in the platform project's directory in a workspace

1. Makefile
2. project-config.mk
3. mfs-config.mk
4. output/Makefile

The build of a platform project is executed via '/bin/python3 /desk/desk-environment.py make --warn-undefined-variables install'

User Actions

In OA 12.5.0, a user has to perform the following steps to define an mfs

1. Define a memoryMappedResource or use an existing one, like SCATHitmap (currently done in basecon-resources.pia.xml)
2. Define an ownedFileSystem (currently done in a fp or pd xml file)
3. Assign kernel file(s) to the mfs (currently done in the project properties Kernel Files tab)

<kernelFile kfs="MFS1">$(PROJECT_NAME).exe</kernelFile>
   

Proposed:

1. Define a memoryMappedResource or use an existing one, like SCATHitmap (currently done in basecon-resources.pia.xml)

   AL> My expectation is that a .pia file would declare the platform resource, e.g., for a foo.kfs.xml there would be an associated foo.pia.xml

2. Define an ownedFileSystem (currently done in a fp or pd xml file)
3. Assign kernel file(s) to the mfs using the new kfs.xml file

• Since the kfs= filter will be removed in the project, how will OA know how to display the LoadList Manager view content? OA now needs to parse kfs.xml files?
  • Or can cdproc provide (1) a list of all the mfs' defined and (2) a list of all the mfs' defined and the kernel files in each?

    AL> cdproc-query provides that information.

• Is it possible that the kfs.xml file could contain all the info from all three user actions?

  AL> Currently "no".

  • Or is the purpose of the kfs.xml file to replace the action of assigning the kernel file(s) to an mfs (or mfses)?

    AL> Currently assignment only.

    AL> There was much discussion of this topic at GeekFest_2024, but I don't see notes for these decisions.

Error Conditions

• How are errors reported by cdproc-makefile and cdproc-query? OA will likely need an error manager to report errors in the Problems View.

Selective Build

• Does cdproc-makefile handle the <ignore> resource name </ignore> attribute?

Optional Components

In OA, via DDCI_PCR:4614 and DDCI_PCR:4611, a dependency can be designated as 'optional'. See screenshot: media:qemu-aarch64-optional-dependencies.png

• Does cdproc-makefile handle the <optional>component name</optional> attribute?

Perform makehash after building

OA generates makefile rules in the platform project's output/Makefile when this option is specified.

• Does cdproc-makefile handle this?

Perform elfchk after building

OA generates makefile rules in the platform project's output/Makefile when this option is specified.

• Does cdproc-makefile handle this?

cdproc-makefile Execution

executing cdproc generates the following files:

1. Makefile
2. workspace-config.mk
3. mfs-config.mk

• The Makefile appears to be a replacement for the platform project /output/ Makefile?
  • why is platform project / project-config.mk not generated?
  • why is platform project / Makefile not generated?
  • workspace-config.mk needs to reside in the top-level directory of the workspace.

AL> We should talk. It is more accurate to say that the makefile generated by cdproc-makefile is more like project-config.mk combined with output/Makefile

AL> The combination is more a compromise to the fact that I had to get something working than an ideal solution (there are notes in the generated makefile, and in the cdproc-makefile source)

AL> A complication is that I very much want to be able to support, for example, building hello-world for several different architectures and including it in any number of platform projects. I'm not sure if such behavior should be directly supported by cdproc-makefile or via autoconf, so I wasn't sure which makefiles cdproc should generate.

• Does cdproc need to be executed from the platform project's output folder?

  AL> Currently "yes". If we add a switch to generate a directory structure and multiple makefiles, then the answer changes.

Since a Makefile is generated, I'm not sure I understand how we can do this work in phases If OA calls cdproc to generate the mfs-config.mk - the Makefile and workspace-config.mk files is also generated Seems like a 'all or nothing' situation.

AL> The intent is that cdproc would generate all the makefiles. However, for this particular phase of the OA/cdproc project, I think we could update OA to generate the kfs.xml files and (at least temporarily) have cdproc only generate the mfs-config.mk file, or perhaps have OA call cdproc-makefile in a temp dir and extract the mfs-config.mk file.

cdproc-makefile command line options

--debugComponent=<component name>

• cd.xml files contain the <debugComponent> attribute, so why is this command line option needed?

  AL> We might be able to remove it. By the same logic, --extra-component is not needed. However, in both cases, without those switches it would be necessary to create a new cd.xml file with the associated content. I'm not morally opposed to that solution, but it does make command line use of cdproc more cumbersome.

--desk-root OA searches in this order:

1. the effective search roots provided by the projects in a workspace
2. search roots in the Windows | Preferences DDC-I Search Path (can be applied for all workspaces, or current workspace)
3. the installed desk

• Seems like cdproc-makefile defaults to (3) is that correct?

  AL> Correct. My expectation was that OA would provide the search roots to cdproc via --desk-root, e.g, --desk-root=dir1 --desk-root=dir2 ... --desk-root=/desk causing cdproc to search the directories in the order given. I'm open to suggested clarifications for the cdproc-makefile help on this.

--extra-component(s)

• What is this for?

  AL> See answer for --debugComponent.

--output This is used to change the name of the makefile to generate (defaults to Makefile)

--unique-prefix --prefix-wrapper

• How do these two options apply to OA's invocation of cdproc-makefile?

  AL> Probably not at all. They are experimental switches intended to support maintainer (notably kernel) test procedures.

cdproc-query Execution

Do you have an example of how to use this script?

AL> cdproc-query works just like cdproc-makefile, but accepts another parameter which specifies the kind of information to report. If you replace "cdproc-makefile ..." with "cdproc-query ... all" cdproc will report lots of stuff, including where it found various files.

AL> My expectation is that cdproc-query will eventually have an option to generate JSON or something OA can more easily parse and that OA will get data from the cd.xml files via this interface.

Test Cases

Existing Automated Tests for MFSes

There are several existing OA automated tests that may need to be updated

• ExternalDebugging - This test has a test case for a project that contains an MFS definition and is external to the test workspace
• ExternalLoadList - This test has a test case for a project that contains an MFS definition and is external to the test workspace
• HyperstartFiles - This test has a test case for a project that contains an MFS definition and is external to the test workspace
• IntegrationDetailsViews - This test has a test case for a project that contains an MFS definition and is external to the test workspace
• StatusMonitorDebugShouldNotUpdateTarget - This test selects/expands MFS nodes in the Target Manager View
• MinimalFileSystem - This is the test for a single MFS and multiple Mfses
• DeosOptionTabs - This is the test for the assigning of an MFS, etc
• CheckLoadList - This test should have some checks for MFSes?
• DeosIntegrationScope- This test should have some checks for MFSes?
• Launcher loadMFS() - this defines the subset of tests that are MFS related

KFS Definition Scenarios

ID	KfsFile Attibute Exists	Kfs File Exists	KfsFile RegEx Valid	MemoryMappedResource Defined	Owned File System Defined	Expected Result	Comments
1	True	True	True	True	True	No Errors, MFS is properly integrated
2	True	True	True	True	False	Error missing OwnedFileSystem definition for MFS
3	True	True	True	False	True	Error missing Memory Mapped Resource definition for MFS
4	True	True	False	True	True	Error invalid KernelRegex specified
5	True	False	True	True	True	Error missing .kfs.xml file for MFS
6	False	True	True	True	True	Error missing <kfsFile> attribute definition

• Two .kfs.xml files with the same name
• A .kfs.xml file with an invalid kernelRegex (file does not exist in the integration scope)
• A kernel file assigned to multiple mfs' (e.g. ansi)
• Provide an example in the /desk/examples directory for defining MFS'?

Include Development Support

• The Include development support checkbox has moved to the DeosQuickstartWizard and NewPlatformProjectWizard. A new test scenario needs to be added for this change.

OpenArbor cdproc Team Meetings

OpenArbor/cdproc Team Meeting Minutes
Jan 23, 2025
Invited: Aaron Larson, Lisa Jett, Kenny Tope, Chuck Frayman, Gary Kindorf, John Dorman
Attendees:TBD

A weekly meeting to discuss and implement OpenArbor's usage of cdproc.

Discussion Topics

• Demo of progress so far
  • Move of 'Active Configuration' selection from Project Options Build Configuration tab to Platform Project's Components view.
  • Bug fix for cdproc-makefile and cdproc-query
• Differences in cdproc-makefile and OA generated makefiles
  
  • workspace-config.mk
    
  • cdproc-makefile Makefile vs OA project-config.mk
    
  • mfs-config.mk
    

Meeting Minutes

media:oa-cdproc-team-meeting-01162025.png
media:oa-cdproc-team-meeting-01072025.png
media:oa-cdproc-team-meeting-01092025.png

2025-02-13

cdproc status

• --var command line switch is implemented
• selected-variant element is implemented
• variant element is currently ignored.
  • Should this have been available-variant?
• optional element is NOT implemented

cd xsd status

• Need element and attribute documentation. Is this in OA UG?
• element and attribute fixes status is TBD

variant names, change maintainer names

From	To
release	production
debug	diagnostic
instrumented	instrumented_abc
instrumented_optimized	instrumented (-production?)

kernelFile "generated" attribute:

• TBD

cd.xml files that need to be updated:

• names: add kernel-file names.dat
• development-support: replace debugComponent with selected-variant

build-utils status

• • cd.bangvar.xml "implemented"
  • desk-private "implemented"
  • variant renames NOT implemented
    • Needs component specific updates
    • Needs howto updates, possibly plans too.