Work Flow
Reference BSP
Prerequisites
These are assumptions that if not satisfied will substantially impact the time estimates given below.
- Documentation has been received for the hardware including:
- Hardware startup guide
- Processor technical reference manual
- Memory map (could be in the form of a FDT (flattened device tree)
- Hardware is installed/accessible on the farm
- Emulator is attached and functional
- Dev-kit components exist for each of the platform specific components being initialized. Component examples are:
- processor specific startup code such as /scm/Deos/products/bsp/dev-kit/branches/mainline/boot/code/architecture/arm
- interrupt controller specific code such as /scm/Deos/products/bsp/dev-kit/branches/mainline/pal/code/interrupts/arm/gic
- pal timer specific code such as /scm/Deos/products/bsp/dev-kit/branches/mainline/pal/code/timers/armv7-generic-timer
Boot
- Collect documentation: Uboot code, quick start guide, platform & SOC reference guide, ICUD, schematic, contact name for person who developed the boot loader - after receiving all documentation, the development schedule begins
- Create an index file that points to each of the relevant documents (some of which may be located in https://deos.ddci.com/scm/Deos/docs/reference-data)
- Digest documentation sufficiently to develop boot - steps 2-3 (along with H/W configuration) take ~14 hours
- Commit Infrastructure to SVN (cloned or template)
- Updated with platform specifics based on the hardware manuals (the following are examples, not a complete list):
- May need to choose different dev-kit files (and update makefile accordingly)
- Specify cross component constants in constants.py (values that span boot/PAL/config components or are used in multiple places making bangvar replacement useful)
- Customize the entry.s if necessary
- Search and replace the reference BSP name for the custom BSP name
- Set version number of components back to 1.0.0
- Update the addEntities.bangvar.ent (make special note of document numbers as they must be unique per product).
- Boot builds
- Configure first stage boot loader (U-Boot launches deos boot)
- Before making changes in uboot:
- Copy off the current environment variables
- Save the existing boot command to another name so that it can be restored
- Document the steps taken to configure U-Boot (or alternate boot loader) as you go into the User Guide.
- Before making changes in uboot:
- Launch kernel on a single core - PAL depends on this milestone; steps 4-7 take ~35 hours (average)
- Launch kernel multi core (if required) - takes 0-35 hours
PAL
- Digest documentation to develop the PAL - takes ~14 hours
- Commit Infrastructure to SVN (cloned or template)
- Updated with platform specifics based on the hardware manuals (the following are examples, not a complete list):
- Search and replace the reference BSP name for the custom BSP name
- Set version number of components back to 1.0.0
- Updated with platform specific timer (Some info may need to be pushed into the constants.py in boot)
- Updated with platform specific interrupt controller - steps 1-3 can be performed independent of boot
- Builds - steps 2-3 take ~35 hours
- Deos ticks (assumes low level tooling; eg, lauterbach) - step takes ~24 hours (incorporates boot)
- All Interrupts Handled (timer and network) - highly variable: 0 - 35 hours (implement interrupts as described in interrupt controller document to give customer full capability)
Config
- Infrastructure in place (cloned or template)
- Get the output of the registry files by running source platform in OpenArbor so that you get the latest updates
- Update with platform specifics (This should mostly be contained in constants.py in boot)
- Network specifics
- Number of cores
- Open Arbor data
- Interrupt numbers configured per HW
- Build - steps 1-4 takes ~24-40 hours
BSP Common
- Update all BSP Documentation (includes BSP UG + network, release notes) - after BSP is fully functional; 35 hours
- Test BSP - takes ~24 hours
- Release; takes ~16 hours
- CCB
- Create deos2cygwin scripts
- final test of the BSP
- generate test-report
- post on ftp site
- make stable
Bare Metal BSP
Desired design approach is to keep the reference BSP in its current configuration and replace the existing boot loader with a Deos Boot Loader. The Deos Boot loader would call the Deos Boot.
- Collect up documentation including (if possible):
- Reference source code from the boot loader (uboot) or BSD source code for the platform
- Contact name for organization that developed the reference source code (e.g. NXP or NAII)
- quick start guide
- platform & SOC reference guide
- ICD for specific components (example for flash device)
- schematic (seldom necessary but perhaps for GPIOs)
- Capture platform specific documents in the platform/docs directory. Processor docs should be located in https://deos.ddci.com/scm/Deos/docs/reference-data
- Create an index file that points to each of the relevant documents
- Review documentation
- Put Infrastructure in place:
- Create Boot Loader Component in Bugzilla
- Create PCR for initial Boot Loader Development
- Create and commit a clone (using svn copy to retain file history) of another boot loader component as starting point
- Search and replace the platform name
- Set version number of component back to 1.0.0
SW
- Take control from the reset vector
- Initialize H/W (order needs updating; list is just a sample):
- memory controller
- clocks
- GPIO
- SerDes
- exception handlers
- secondary core(s) reset vector(s)
- network low level init (may include PCI)
- PAD registers
- Flash
- Initialize customer requested H/W devices:
- PCIe bus(s)
- CAN bus
- FPGA
- other FLASH device
- SPI bus
- I2C bus
- Specific GPIOs
- Time Triggered Ethernet (TTE)
- Update boot/Config/PAL components if impacted
- Update UG to indicate boot loader exists and how to get the target running using this mechanism, retaining reference BSP method.
- Follow steps in BSP Common
Verification Phase 1
Getting Started
- Collect up documentation including (if possible):
- Reference source code from the boot loader (uboot) or BSD source code for the most similar platform
- Contact name for organization that developed the reference source code (e.g. NXP, NAII or customer BSP team if one exists)
- Memory map for custom hardware
- quick start guide from the customer
- platform reference guide
- SOC reference guide
- Processor manuals
- Network Device Manual
- PHY documentation
- Architecture manuals including EABI
- ICD for specific components (example for flash device)
- schematic (seldom necessary but perhaps for GPIOs)
- Capture platform specific documents in the platform/docs directory. Processor docs should be located in https://deos.ddci.com/scm/Deos/docs/reference-data
- Create an index file that points to each of the relevant documents (some of which may be located in https://deos.ddci.com/scm/Deos/docs/reference-data)
- Review documentation
- Put Infrastructure in place:
- Create BSP Components in Bugzilla
- Create PCRs for initial BSP development (one for each boot/pal/config)
- Create and commit a clone (using svn copy to retain file history) of the reference BSP under the new BSP name. Notes:
- If this is custom hardware for any customer other than Honeywell it should be located in the DDCI Subversion Repo.
- If the reference BSP was a bare metal BSP the boot code and boot loader code should be merged into one boot component.
- Create new UG Document number and update it in the UG
- Search and replace the reference BSP name for the custom BSP name
- Set version number of components back to 1.0.0
Design
Consideration must be made as to if initialization should happen in boot, pal or a device driver. Factors that influence decisions include:
- If the kernel needs it, it must be done in boot (e.g. memory controller)
- Testing is easiest to perform on a user mode driver, hard in the PAL, hardest in boot.
- If it is hard to postpone, it can be done in boot
- If initialization is only for the customer, could it be done in palColdStart()/palWarmStart()? Keep in mind this code will be rerun on mode changes. This could impact performance requirements.
- If initialization is only for the customer, is it possible to use a pal extension library to give the customer control over configuration. The customer often wants changes late in the game, so giving them control helps reduce rework after verification is complete.
- If a user mode driver is going to use it, can the driver do the initialization? Don't initialize it in the boot/pal if it isn't necessary.
- PCI:
- Do the minimum init that the customer requires. The certification authority does not like devices to be initialized and unused.
- On some hardware (X86), PCI was not memory mapped and accesses were through I/O ports. To access it, the customer needed a runtime library to prevent processes interfering with one another. In this case a PRL was the best solution.
SW Updates
Each component included from the dev-kit should be evaluated to determine if code can be used unmodified:
- Some components will state that it cannot be used in a certified environment. Others will not have been evaluated yet.
- If any of the following conditions exist:
- Dead code will exist when run on the platform (code contains conditionals that could not be hit on real hardware)
- Code performs a behavior that would cause inconsistent timing (example would be muxed timer code)
- Comments suggest that author did not understand or agree with the implementation
- Code does not match the platforms documentation (yes, this means you have to read the manual for this component)
- then clone the module making it platform specific. Strip out any unused or problematic code for the specific platform. Also consider looking for another BSP that has already been verified that uses this same component.
- If code is able to be used in its current form, the dev-kit version of the files can have trace tags, requirements, test cases(maybe) and test procedures(maybe) added directly to the dev-kit. One example of where this might work would be in the low level timer code (not common timer) specific to the platform hardware. This will save effort on the next BSP verification.
The following need to be added to the BSP:
- Take control from the reset vector
- Initialize H/W (order needs updating; list is just a sample):
- memory controller
- clocks
- GPIO
- SerDes
- exception handlers
- secondary core(s) reset vector(s)
- network low level init (may include PCI)
- PAD registers
- Flash
- Initialize customer requested H/W devices. Examples:
- PCIe bus(s)
- CAN bus
- FPGA
- other FLASH device
- SPI bus
- I2C bus
- Specific GPIOs
- Perform other customer specific requirements. Example:
- Hyperstart selection algorithm
- Memory tests
- Initialize Watchdog
The BSP developer(s) should become familiar with the entire set of hardware. A determination should be made if each device should be enabled or disabled. Some customers have a requirement that all unused hardware (such as peripherals in the SOC) be disabled if not in use. Understand the customers needs in order to produce appropriate requirements for the platform.
Read the Requirements checklist before you start developing the requirements. Keep these in mind as you write the requirements. Refer to them regularly.
Requirements should be developed for all code. Requirements are to be developed from the hardware manuals, kernel requirements or customer requirements, not by reverse engineering the source code.
In some cases, detailed documentation is not available in the manual and the hardware vendor directs you to reference code as source of the initialization method. In these cases, consider if requirement can be written sufficiently high level as to explain the intent and be testable, but not provide the low level details of the initialization process. For example, a requirement for initializing the interrupt controller could be written in two (or more) ways:
- The pal shall initialize the interrupt controller per table 1. (Table 1 contains 10 steps) The tester would have to test that steps 1-10 were each performed in order.
- The pal shall initialize the interrupt controller to enable the interrupts described in table 1. (Table 1 contains 255 interrupts) The tester would have to test that each of the 255 interrupts worked.
Remember that the tester should be able to read your requirements and have sufficient information to test. Every branch in the code and every possible return code should be evident by requirements, thus resulting in a test. If they can't write a test for it, there could be a structural coverage hole.
Add trace tags to code for each requirement. While adding trace tags, make sure that all code is based on a requirement. This could be iterative with adding the requirements.
Consider how code will be tested. Test points may need to be added to induce failures or ensure code was executed in a specific order. Test points will need to be documented in the requirements using NI_* tags.
Consider if platform has secure firmware (arm), system management mode (x86), hypervisor (all architectures) or other firmware (such as network devices) that could get control during runtime. Can the unknown SW/FW affect time or space partitioning? For most systems, this is unacceptable behavior and must be prohibited. Consider the customer requirements to determine if these components need to be replace by a verified component.
Read the Code checklist and review your own code against it to make sure you are meeting the checklist.
Confirm the component is ready to energize the status files for Requirements and Code.
- Update Config component:
- Changes made to the memory map should be done in constants.py and added into the registry
- Evaluate if additional memory regions should be marked as reserved based on impact to time/space partitioning
- Do any customer requirements lead to registry changes?
- Update UG
- Update Release notes (each component impacted)
- All BSP Documentation updated (includes BSP UG + network, release notes) - after BSP is fully funtional; 35 hours
- Test BSP - takes ~24 hours
- Test suite depends on the customers requirements. Go through the reference BSP tests and consider if each apply to this BSP. Add tests specific to this platform.
- Ideally, someone else would have been developing some formal tests during this time. Run as many formal tests as possible.
- Can we run the OpenArbor automated tests on this platform?
- Release; takes ~16 hours
- CCB
- Create deos2cygwin scripts
- final test of the BSP (generate test-report)
- post test report on ftp site and move component to stable
- Make sure components can be installed using the cygwin installer
- Notify DDCI management Verf Phase 1 is ready for delivery to customer
Verified
Prerequisite: Everything from Verification Phase 1 should be complete.
The DDC-I verification process is well documented, so this section will just give you helpful hints.
Your new favorite links are to:
Create the Component Wiki
A Component Wiki is the flow chart for the verification process. If one does not exist, create a component Wiki for each of the BSP components based on the last verified BSP. Example Harrys_Boot. Each of the BSP components should be referenced from the higher level Program page. Example Harrys_Program.
The Product Wiki page just created contains a list of activities to be performed. Update the Assignment column with the engineer expected to do each task. It may be more than one engineer. If it is still unknown, set it as TBD. When an engineer becomes available, they should update it with their name so other people know it is in progress. Tasks marked SQA will require coordination with the current DDC-I SQA engineer.
Note that a grey bar is used to designate tasks above the line should be completed before tasks below the line commence.
StatusSummary Script
The component wiki will point to the instructions to energize the status files. But from either the component wiki (specific to your component) or the Review Game (can specify the component), there is a link to statusSummary. The statusSummary script will tell you the status of the review process as well as provide the following very useful scripts for updating status files:
- The "Requirements (verbose)" links will run cmpStatus2src.py and tell you which requirements files are different since the version listed in the requirements status file.
- The "Code (verbose)" links will run cmpStatus2src.py and tell you which Code files are different since the version listed in the Code status file.
- The "Tests (verbose)" links will run cmpStatus2src.py and tell you which test files are different since the version listed in the tests status file.
- The "Analysis (verbose)" links will run cmpStatus2src.py and tell you which analysis files are different since the version listed in the analysis status file.
- "Check status file consistency with SCM." will run matchSrc2Status and give a list of differences between the current status files and what is in subversion.
- "Add Authors to the following display" will show who is eligible to perform reviews based on the commit history. Example From here, if you can:
When you are ready to perform a review and have committed the status file to put a file in review, you can use this page to generate the review status report by:
- Expand "Add Authors to the following display"
- Select the checkbox next to files that are marked as in review by you
- Expand "Show Uplink Files" if you are reviewing anything other than requirements
- Select the upstream requirements documents that the file you are reviewing trace to
- Select "Generate Review Summary Report"
Adhoc Testing
- The Boot contains the User Guide for the entire BSP. When testing the board for first release, the instructions to create a USB key (on a new key) or hyperstart image should be followed explicitly to ensure that the instructions are intuitive and complete. The entire UG should be reviewed in its htm format in the desk to ensure all content is accurate for this board and all links are active (and work from the desk).
- Create a Hypstart image using Open Arbor and instructions in $(PLATFORM)-bsp-user-guide.htm. Make sure it runs, and can be used in the steps below.
- Make sure a hypstart and makeboot command can be executed on the command line and in OA. Steps to do this should be documented in the quick start section of your BSP UG.
- Build and load the images for the release variants of boot and pal to run the tests below. Do the same for the debug variants.
- Following the OA UG instructions for the hello-world example load the target, test mode changes, videostream, status monitor, and the mld and gdb debuggers.
- Additional examples recommended are cffssimple, ioi-simple, and pal-extension. If a Sata MAL is supported, the cffssimple example should also be run with that specific supported MAL.
- Follow all the steps in the ext-clock-sync example should be done to ensure that feature is working and the clock is consistent.
- Ping flooded the target from 3 separate consoles. Ensure that after ping flood is terminated the network resumes operation (`ping -f <ip addr>`).
- Boot typically owns and defines the Platform Specific Information object. This can be tested with the psio-print bsp example.
- As a general rule and best practice, clear the TSC at initial power on. This will be the starting point for total elapsed time for the system.
- The tick rate should be verified against a wall clock. This should be evolved into a formal test, but at present, a minimum test is to capture the number of ticks that occur in five minutes and compare that against the tick rate set up in the registry. For example, most reference systems ship with systemTickInUsec = "12500", which is 80Hz. So, using a stop watch, capture the number of ticks that occur in 5 minute. If this number differs substantially from 24000, you probably have a problem with your tick calculation.
- If the PAL is required to support 653, run one of the demo653 examples and verify correct behavior.
- If the PAL is required to support WATs, run the timemap-WAT-process example and verify that the windows in the timemap match (within a margin of error) the correct timing for the start time, length and frequency documented in the windowActivations table of the timemap-WAT-process.pd.xml.
- If a dev-kit is produced, the instructions in the UG should be followed and the desk in the output folder should be diffed against the delivered desk to ensure applicable content is built and binary files are identical.