OpenArbor Makefile Design
Estimate Summary
| Task | Estimate | Comments |
|---|---|---|
| Workspace | @sum(wstot) | |
| Application Projects | @sum(appvartot) | |
| Platform Projects | @sum(pptot) | |
| Automated Testing | @sum(attot) | |
| UserDocumentation | @sum(doctot) | |
| Workspace Conversion Issues | @sum(wscontot) | |
| Files To Deliver With OA | @sum(deltot) | |
| Total | @sum(column) | Orig: 159 |
Sandbox
Here you will find some sample executable and platform projects used for testing new make rules, etc.
https://deos.ddci.com/svn/DDCI/products/openarbor/openarbor/branches/improve-build/sandbox/
Implementation of new Makefiles
In an OpenArbor workspace, there are several makefiles at various locations within the workspace and project directories.
In a general sense, the existing generated config.mk file is being "split up" into other files, and new make commands/variables are being added to provide a more flexible, user-friendly make system that is closer in design to the Deos make system.
Application Projects
Makefile
This file is generated when the project is created.
- See MakefileGenerator::generateMasterMakefile(DdciProject project)
- The same template is used to generate both an application project & platform project Makefile. The only difference is the BUILD_DIRS variable is different for a platform project.
Builds the project
The BUILD_DIRS variable indicates which target\variants will be built:
- target is determined by target selected on the Project Options | General tab
- Consider building for all installed targets instead?
- Consider building for all installed targets instead?
- variants checked on the Project | Configurations tab)
The output/arch/variant dirs are created
make is executed in each of the output/arch/variant dirs
Template: Makefile.mk (replace existing template)
project-config.mk
For non-platform projects: included by the output/arch/variant dir Makefile
defines VPATH for all directories that contain buildable files
defines make variables required for selective building for all the file extensions in the project
defines the dependency and clean rules for each file extension
Template: ExecutableProjectConfig.mk
user-config.mk
if it exists, included by the output/arch/variant dir Makefile.
This file is created by users, to configure PRE_LINK or POST_LINK make rules, or any other configuration type stuff.
Makefile
A Makefile will be generated for each target/variant combination specified in the Application Project Makefile.
defines make variables for the default and user-specified compile and link options for the target/variant.
Template: ProjectTargetVariantMakefile.mk
Tasks
| ID | Who | Priority | Task | Estimate | Comments |
|---|---|---|---|---|---|
| E1 | LCJ | 2 | The Project Options | Deliverables tab is inaccurate (missing 653 or ioi loadlist items, duplicated items) | 8 | |
| E2 | LCJ | files in output/target/variant/deos653 dir cannot be opened in the DeosComponents view | 4 | ||
| E3 | LCJ | 2 | Check exported files for PowerPC when FPU type is specified to "Classic" | 2 | |
| E4 | JK/LCJ | 1 | missing support for building projects containing files that are "links" to other files in the workspace, or external to the workspace | 16 | |
| E6 | JK/LCJ | 1 | Compiling an individual file, directory. Compiling a list of files/and or directories. | 16 | Compiling .sx, .cc and .s files fail For .s: x assembler -c ../../../get5error.s -o get5error.o make: x: Command not found For .sx and .cc, no compile command is executed. |
| E8 | JK/LCJ | 1 | Build the pattern for specifying asm, compile, link options for C/C++ folders. | 16 | DDCI_PCR:3608 |
| E10 | JK/LCJ | Generalize ability to specify name of binary config file output for 653, IOI, IST. | 16 | DDCI_PCR:3619 | |
| E11 | JK | 1 | Instrumented build failure for auto-tune example | 4 | |
| E12 | JK | 1 | Add user-config-suffix.mk | 24 | DDCI_PCR:3621 |
| E13 | JK | 1 | Add namelist/version-script | 16 | DDCI_PCR:3637 |
| E14 | JK | 1 | resource links for source and header files in the same folder issue | 16 | DDCI_PCR:3672 |
| E15 | JK | 1 | resource link renames a file or folder | 16 | DDCI_PCR:3672 |
| E16 | LCJ | 1 | generated makefile variable names need updating | 8 | DDCI_PCR:3662 |
| E17 | JK,LCJ | 1 | support ai.xml files | 16 | DDCI_PCR:3641 |
| E18 | JK | 1 | add optional debug-logging to makefiles | 16 | DDCI_PCR:3620 |
| Total | @sum(column) | Orig: 94 |
Ada Specific Tasks
| ID | Who | Priority | Task | Estimate | Comments |
|---|---|---|---|---|---|
| E5 | JK | Review ada-deos-interfacing example makefile structure to flesh out tasks required for Ada support in the makefile build system | 12 | DDCI_PCR:3717 The latest tostones (Ada) DDS is here: \\nx3000\ship\dds\windows\approved\DDS-tostones2-deos-handies-20180223 | |
| E7 | JK/LCJ | Build the selective build pattern for Ada files and folders. | 4 | DDCI_PCR:3717 | |
| E9 | JK/LCJ | Build the pattern for specifying asm, compile, link options for Ada files and/or folders. | 8 | DDCI_PCR:3717 | |
| E19 | LCJ | Review and test support for editing Ada Files. | 8 | DDCI_PCR:3717 | |
| E20 | JK | Design and test makefile updates for building Ada projects using Deos | 40 | DDCI_PCR:3717 | |
| E21 | LCJ | Implement building Ada projects using Deos | 40 | DDCI_PCR:3717 | |
| E22 | LCJ | Implement automated tests for building Ada projects | 40 | DDCI_PCR:3717 | |
| E23 | LCJ | Update OA User Guide for Ada Support | 16 | DDCI_PCR:3717 | |
| Total | @sum(column) | Orig: 94 |
Platform Projects
Makefile
The BUILD_DIRS variable is set to output (since a platform project does not have variants), and make is executed in the output directory.
See application project Makefile section for implementation details. The only change required to the application project version is the setting of the BUILD_DIRS variable.
BUILD_DIRS=output
project-config.mk
For platform projects: included by the output/Makefile
Contains makefile variables and targets for building a platform project.
Template: PlatformProjectConfig.mk
Makefile
Builds the platform project
Contains the options for the platform project
Template: PlatformProjectOutputMakefile.mk
Tasks
| ID | Who | Priority | Task | Estimate | Comments |
|---|---|---|---|---|---|
| P1 | [TBD] | Allow users to specify -names.dat files from other components. | 16 | A platform project no longer gathers -names.dat files from dependent projects and append those files to the platform project's names.dat file. A better design is to include a -names.dat file in a project's component description (.cd.xml). | |
| Total | @sum(column) | Orig: 16 |
Workspace
Makefile
Builds all the projects in the workspace from the command line. (no changes are required for PCR 3465)
workspace-config.mk
This file is generated when the workspace is created, and regenerated when a project is added to / deleted from /renamed in the workspace.
Included by each project's project-config.mk file
Defines root paths for DDS tool locations (desk, SCORE, TRAC, etc)
Defines variables for each project in the workspace using relative paths
- The root paths and other workspace project variables are currently generated by MakefileGenerator::collectToolHomeDeclarations(...) called by MakefileGenerator::generateConfigFile(...).
- These definitions now need to be placed in workspace-config.mk.
- The tool home locations suffix _ROOT to replaced with _DIR
- ABS_PROJECT_PATH replaced with PROJECT_DIR
- ABS_PUBLIC_ROOT replaced with INSTALL_PREFIX
- DEOS_ROOT replaced with DESK_DIR
- SCORE_ROOT replaced with SCORE_DIR
- TRAC_ROOT replaced with TRAC_DIR
- PYTHON_ROOT replaced with PYTHON_DIR
- GCC_ROOT replaced with GCC_DIR
- OPEN_ARBOR_UTILS_ROOT replaced with OPEN_ARBOR_UTILS_DIR
- GCC_RTEMS_ROOT replaced with GCC_RTEMS_DIR
- hardcoded paths to workspace replaced with WORKSPACE_DIR.
- New piece to be included in workspace-config.mk:
Template: WorkspaceConfig.mk
Tasks
| ID | Who | Priority | Task | Estimate | Comments |
|---|---|---|---|---|---|
| W1 | LCJ | Review and cleanup unused code related to the old makefile build system and check for any functionality that needs to be implemented in the new build system. | 8 | ||
| W2 | JK | Write a script to generate a project(s) with 1000+ files for testing | 40 | DDCI_PCR:3622 Provide some profiling data | |
| W3 | LCJ | Building for multiple variants | 32 | DDCI_PCR:3535 The new makefile system handles this, but OA will need a change to the ComponentDescriptor class to maintain a list of build variants. | |
| Total | @sum(column) | Orig: 48 |
Automated Testing
| ID | Who | Priority | Task | Estimate | Comments |
|---|---|---|---|---|---|
| 0 | |||||
| A1 | LCJ | 1 | Update the MakeExport test | 2 | DDCI_PCR:3551 Done. |
| A2 | LCJ | 1 | Analyze the failure of the SelectiveBuild test | 2 | |
| A3 | LCJ | Review existing tests for completeness with regard to build functionality | 2 | ||
| A4 | LCJ | Add test for adding, removing and modifying a source code file. | 4 | ||
| A4 | LCJ | Add test for adding, removing and modifying an include file. | 4 | ||
| A5 | LCJ | Add test for adding, removing and modifying a dependency. | 4 | ||
| A6 | JK | Add optional debug-logging to makefiles | 24 | DDCI_PCR:3620 | |
| Total | @sum(column) | Orig: 16 |
- Test to verify each supported file extension is tested with a compile command.
User Documentation
| ID | Who | Priority | Task | Estimate | Comments |
|---|---|---|---|---|---|
| D1 | LCJ | 1 | Document overall OA build system for users. | 4 | |
| D2 | LCJ | 1 | Document use of user-config.mk | 2 | |
| D3 | LCJ | 1 | Provide a list of make VARIABLES for users | 2 | |
| D4 | LCJ | Perform makehash option (exe projects): User is required to create a PROJECT_NAME-names.dat file. | 1 | ||
| D5 | LCJ | 1 | Update User Guide section on building externally (Section 3.2.4) | 1 | |
| Total | @sum(column) | Orig: 10 |
Files to Deliver With OA
| ID | Who | Priority | Task | Estimate | Comments |
|---|---|---|---|---|---|
| Deliver the make-utils files with the OA RCP | 0 | Done | |||
| Total | @sum(column) | Orig: 4 |
The restructuring of the OA build system requires some supporting files to be delivered within the OA installation directory.
- Add the make-utils dir and it's contents to the workspace dir
- Modify OA's makefile rule for creating RCPs to include the new "make-utils" directory and it's contents. (See flexnet dir as an example).
- The files are now configured here:
https://deos.ddci.com/svn/DDCI/products/openarbor/openarbor/branches/improve-build/workspace/make-utils/
make-utils Directory
This new folder will contain supporting makefiles referenced by the generated makefiles in the user's OA workspace.
deos-rules-prefix.mk
Contains makefile rules that must be included before project specific variables are defined.
deos-rules-suffix.mk
Contains makefile rules that must be included after project specific variables are defined.
functions.mk
Contains useful functions for the makefiles to use.
install_rule.mk
Defines macros for exporting projects
User Workspace Conversion Issues
| ID | Who | Task | Estimate | Comments |
|---|---|---|---|---|
| Convert existing Makefile in an application project | 0 | Done | ||
| Convert existing Makefile in a platform project | 0 | Done | ||
| Total | @sum(column) | Orig: 8 |
The existing config.mk file in each project's output directory can be deleted.
The existing Makefile in each project's directory will require "conversion" since it can currently be modified by users.
- Any lines that are not generated by the plugins will need to be moved to the new user-config.mk file
- Any references in the user-config.mk file to modified make variables (See list in workspace section) will need to be replaced with new variable values.
- The .options files will require conversion for deprecated variables as well.
Deos Shared Object Example Projects that "Link with Intermediate Archive"
If an example is at a version lower than 15, the project converter (from 14 to 15) will add set the "Link with an intermediate archive" option to "true", because prior to version 15 the default value was true. This is done to prevent user's project settings from being altered. Many of our Shared Object project's .options files are at a version lower than 15. We may want to consider replacing those with updated .options files. Also, two of our example project's have the option set to true in their .option file, and they are at version 16. So.. those should also be updated.
| Example | Version | Comments |
|---|---|---|
| deos653\custom-io\custom-io-library | 12 | |
| deos653\hybrid-io\hybrid-653-library | 12 | |
| deos653\hybrid-io\hybrid-ioi-library | 12 | |
| ioi\ioi-simple\ioi-simple-data-formatter | 12 | |
| kernel\auto-tune\auto-tune-library | 12 | |
| kernel\shared-library\application-loaded-library | 16 | has the option set explicitly in it's .options file |
| kernel\shared-library\startup-loaded-library | 16 | has the option set explicitly in it's .options file |
| training\printx\printx | 12 |
Execution of make in OpenArbor
| Scenario Description | Steps to Reproduce | make cmd | OA code implementation |
|---|---|---|---|
| Compile/Assemble selected source code files (one or more) | (1) Disable Project | Build Automatically. (2) Right-click a source code file (i.e. hello-world.cpp) and select Compile/Assemble |
make --keep-going --warn-undefined-variables <path to project's output/target/variant folder>/hello-world.o | Builder.java internalCompileResources(...) L231 |
| Clean | (1) Project | Clean (2) Clean all projects / or one or more projects using checkboxes on Clean dialog |
make --keep-going --warn-undefined-variables clean | Builder.java performClean(DdciProject project, IProgressMonitor monitor) L475 |
| Building QEMU emulator | On the Target Manager View, click the "Build Emulator" or "Launch Emulator" toolbar buttons | make --keep-going --warn-undefined-variables boot | Builder.java buildTargetAndPopupOnError(....) L320 |
| Building a project | (1) Right-click a project and select Rebuild Project (2)Automatic build (i.e. source file modification) |
make --keep-going --warn-undefined-variables [--jobs=8, --output-sync=target] | Builder.java internalBuild(.....) L392 |
| Exporting a project | Right-click a project and select Export | make export | ExportDeliverablesWizard L79 |
| Build a list of macro names to their definitions | (1) Right-click a project and select Rebuild Project (2)Automatic build (i.e. source file modification) |
make --keep-going --warn-undefined-variables -Rrqp | Builder.java getQueryVariablesOutput(DdciProject project) L502 DDCI_PCR:2623 Ensure recursive macro check always runs. |
| Create gdb-target-files directory in a platform project | Debug with gdb | make --keep-going --warn-undefined-variables gdb-target-files | DeosLaunchGdbConfigurationDelegate generateGdbTargetFiles(...) |
