Deos Multicore Verf

From DDCIDeos
Jump to navigationJump to search

This project is to verify Deos for Multicore. This includes the first verified baseline for ARM. It is generally not expected that the functionality of the components will be extended except as needed to work and be verified for multicore.

The only customer program needing the first verification is the Harrys_Program. See the program page for the SOW and hardware information. Other customers for the 2nd follow on verification with additional features include Desert_Eagle_Program and Durants3_Program. The goal is for components that are complete during the 1st verification to remain complete for the 2nd verification. Exceptions will include the Kernel where features cannot make it to the first verification within the schedule needed.

Current Status

The plan is to use component mainlines for development in connection with the initial multicore verf. The Multicore architectures are: arm, ppc and x86.

Reference BSPs will include:

  • qemu-arm
  • qemu-ppc
  • qemu-x86

Note: The distros for these BSPs are configured for single core; their configurations can be updated to emulate two or more cores.

Other BSPs for hardware platforms will be needed for Run-For-Score and must therefore be updated appropriately for Multicore. BSPs and hardware platforms supporting true multicore will be required e.g. for new kernel tests. Currently these BSPs include:

  • minnow-turbot (E3800)
  • t10xx-mc
  • zcu102 (Durants2) & s32v234 (Harrys)

Any other BSPs are TBD.

The first verification will be ARM only, so components which are being updated in the 2nd verf do not require RFS on x86 or PPC.

Be a Maintainer

Steps to establish a Multicore environment.

  1. Install experimental Indie Cygwin packages:
    • deos-maintainer-tools

High level Issues

We expect that almost all components providing Deos binaries will be updated and re-released to support Multicore. The most pervasive issue is to ensure proper use of data shared between several threads of execution.

Infrastructure Update

The following table has the current status.

Meaning of the columns:

  • Sequence : Lower numbered tasks need to be complete before higher numbered tasks. Tasks with priority 1 are done. Tasks with priority 1xx are for 2nd verification. Tasks with priority '999' can be done "later".
  • GCC 7.3 & new build :
    1. Builds using GCC 7.3 (so that compiler is C/C++-11 memory model aware).
      1. Eliminate compiler warnings for level E components, higher design assurance require cost benefit trade-off.
    2. Stuff that should have been done as part of arm port:
      1. Component has been ported to commonBuildSystemInterface=5 and compiles for all architectures.
      2. Makefiles match canonical format.
      3. The package's deos2cygwin 'run_update_deos_libs.sh' postinstall link should be removed.
      4. Standard content bootstrap files have been removed ref Common-Makefile-HowTo.
      5. Fix code/makefile.mk error from MIPS port. Some makefile.mk files put $(DESK_PREFIX)/include before $(DESK_PREFIX)/$(HOST_ARCHITECTURE)/include. That is backwards.
      6. Add support for "make QUIET=@" to code/makefile.mk (optional, and applies to all makefiles).
      7. README.txt follows new conventions.
      8. Components that install headers validate C conformance with deos-validate-header-conformance.
      9. If in doubt then compare with recently verified components.
  • Multicore Updated? :
    1. Adapted to new kernel APIs and integer types.
      • Nice to have: consider 64 bit architectures (e.g. in choice of integer types).
    2. Analyze and update for new C/C++-11 memory model:
      1. This is much harder than you think! If you find it is easy, you are not thinking hard enough!
      2. Declare global variables to avoid cross-core interference (e.g. on separate cache lines).
      3. Use locks or memory fences for global state where necessary.
      4. Verified components need change impact analysis.
      5. Document interference patterns (in UG).
  • Informal Test :
    1. The tests have been ported to new test infrastructure.
    2. The regression runs, perhaps with some explainable errors.
  • Cygwin Released : The component has been at least "unreleased" with feature complete version.
  • Tests Run?  : All the tests developed and pass on on all targets.
  • Done : All verification activities have been completed.


Infrastructure Update Status

Component Sequence GCC 7.3 & new build Multicore updated? Informal test? Cygwin Released? Tests Run? Done DAL Assignee Comments
abc-tool 2 Yes N/A Yes Yes Yes No TQL-5 TBR Updates & CAN files for GCC 7.3
ansi 3 In Work In Work Yes No No No DAL-A JON Global variables/needs verf'ed startup. Confirm Desert_Eagle_Program updates will make it into 1st verf.
bsp-dev-kit 99 No No N/A N/A N/A No DAL-E Doesn't the dev-kit have requirements/tests for verified bsp. Needed for harrys BSP?
cffs-examples 1 N/A N/A Yes Yes Yes Yes
cffs-server 40 No No No No No No DAL-A
cffs-server 140 No No No No No No DAL-A Desert_Eagle_Program requires filename length update (Assume not in time for harrys)
cffsapi653p2 150 No No No No No No DAL-A Verify to A for DesertEagle and Durants3
cffsdump 40 N/A N/A N/A N/A No No
cffsdump 140 N/A N/A N/A N/A No No Desert_Eagle_Program requires filename length update (Assume not in time for harrys)
cffsi7sata 999 No No No No No No DAL-A
cffsmalram 40 No No No No No No DAL-E
cffsmpc5200ata 999 No No No No No No DAL-A
cffsnorflash 999 No No No No No No DAL-A DO-178C "ify"
cffsp1013sata 999 No No No No No No DAL-E
cffssataflash 999 No No No No No No DAL-E
codetrace 1 N/A N/A Yes Yes Yes Yes Unqualified dev tool
ctxtime 80 No N/A No No No No
deos-training 90 N/A N/A No No N/A No
deos653-config-win32 1 N/A N/A Yes Yes Yes Yes
deos653-config-win32 130 N/A N/A No No No No
deos653-cvt 50 No N/A No No No No TQL-4
deos653-cvt 150 No N/A No No No No TQL-4
deos653p1-examples 1 N/A N/A Yes Yes Yes Yes
deos653p1-runtime 30 Yes No No No No No DAL-A ARM vfpu3-d16 or vfpu3-d32?
deos653p1-runtime 130 No No No No No No DAL-A See Desert_Eagle_Program and Durants3_Program for additional features
deosbook 1 N/A N/A Yes Yes N/A Yes Note: There is a new unrelease
deosbook-devel 1 N/A N/A Yes Yes N/A Yes Note: There is a new unrelease
deosname 1 N/A N/A Yes Yes Yes Yes DAL-E gcc/multicore updated really N/A?
desk-docs 1 N/A N/A Yes Yes Yes Yes
desk-examples 1 N/A N/A Yes Yes Yes Yes
desk-python-tools 1 N/A N/A Yes Yes Yes Yes
elfchk 1 N/A N/A Yes Yes Yes Yes
ftpserver 1 N/A N/A Yes Yes N/A Yes DAL-E gcc/multicore updated really N/A?
gdb-cross-debuggers 1 N/A N/A Yes Yes N/A Yes
gdbserver 1 N/A N/A Yes Yes N/A Yes DAL-E gcc/multicore updated really N/A?
gnu-language 3 Yes Yes In Work No No No DAL-A ANSI may require due to Durants_Program updates for TLS.
hypdump 1 N/A N/A Yes Yes N/A Yes
hypstart 1 N/A N/A Yes Yes N/A Yes
hypstart 103 N/A N/A No No N/A No multiple kernel file system updates
inetd 1 N/A N/A Yes Yes N/A Yes DAL-E Lots of compiler warnings. gcc/multicore updated really N/A?
integ-tool-command-line 1 N/A N/A Yes Yes Yes Yes
integ-tool-command-line 102 N/A N/A No No No No
ioi-api 20 No No No No No No DAL-A
ioi-config-tool 1 N/A N/A Yes Yes Yes Yes
ioi-cvt 50 No N/A No No No No TQL-4
ioi-examples 1 N/A N/A Yes Yes Yes Yes
ioi-ringBuffer 20 No No No No No No DAL-A
ist-application 1 N/A N/A Yes Yes N/A Yes DAL-E can gcc/multicore updated be N/A?
ist-arinc653-examples 1 N/A N/A Yes Yes N/A Yes
ist-config-win32 1 N/A N/A Yes Yes Yes Yes No CVT needed since IST is DAL-E
ist-ioi-examples 1 N/A N/A Yes Yes N/A Yes
kernel 3 Yes Yes Yes Yes In Work No DAL-A RLR/AL
kernel 103 Yes Yes No No No No DAL-A See Desert_Eagle_Program and Durants3_Program for additional capabilities
kernel-examples 1 N/A N/A Yes Yes N/A Yes
kernel-multicore-examples 30 No No No No N/A No
kernel debugger library 1 N/A N/A Yes Yes N/A Yes DAL-E
link_deos/strip_deos 1 N/A N/A Yes Yes N/A Yes
makeboot 1 N/A N/A Yes Yes N/A Yes
makelib 1 N/A N/A N/A Yes N/A Yes
makereg 1 N/A N/A Yes Yes Yes Yes
lwip 1 N/A N/A Yes Yes N/A Yes DAL-E gcc/multicore updated really N/A. New version to support RTEMS coming but can be independent of ARM/Multicore verf.
mailbox-transport-library (MTL) 30 Yes No No No No No DAL-A DO-178C "ify", Tests still broken
mailbox-transport-library-examples 1 N/A N/A Yes Yes Yes Yes
math (unified) 20 No No No No No No DAL-A JON needs verf'ed startup. Desert_Eagle_Program features will be included in initial verf
math-x86 120 No No No No No No DAL-A Maybe we can MC analyze existing baseline? May be 999 if 2nd verf is only arm/ppc.
qemu-vm 1 N/A N/A Yes Yes N/A Yes Keep unless we want to use host cores
qemu-arm-boot 2 Yes Yes Yes Yes N/A No DAL-E TBR
qemu-arm-config 2 Yes N/A Yes Yes N/A No DAL-E
qemu-arm-pal 2 Yes Yes Yes Yes N/A No DAL-E TBR
qemu-ppc-boot 2 Yes Yes Yes Yes N/A No DAL-E TBR
qemu-ppc-config 2 Yes N/A Yes Yes N/A No DAL-E
qemu-ppc-pal 2 Yes Yes Yes Yes N/A No DAL-E TBR
qemu-x86-boot 2 Yes Yes Yes Yes N/A No DAL-E TBR
qemu-x86-config 2 Yes N/A Yes Yes N/A No DAL-E
qemu-x86-pal 2 Yes Yes Yes Yes N/A No DAL-E TBR
registry-cvt-win32 50 No N/A No No No No TQL-4
registry-cvt-win32 150 No N/A No No No No TQL-4
serial-io-driver 1 N/A N/A Yes Yes Yes Yes DAL-D Can gcc/multicore updated be N/A? Is ARM variant needed?
socket-api-library (SAL) 30 Yes No Yes Yes No No DAL-A DO-178C "ify". Multicore analysis was performed and there are two issues with static local variables in gethostbyname and gethostbyaddr APIs. Plan is to re-verf SAL for multicore after merge of lwipd version back to mainline.
socket-examples 1 N/A N/A Yes Yes Yes Yes
startqemu 1 N/A N/A Yes Yes N/A Yes
startup-gcc 2 Yes No Yes Yes Yes No JON Any new functionality? Resolve static and so issues.
static-video-library 1 Yes N/A Yes Yes N/A Yes DAL-E
status-monitor 80 No No No No N/A No DAL-E Working version, but new functionality desired
system-video-streams 1 N/A N/A Yes Yes N/A Yes DAL-E can gcc/multicore updated be N/A?
tap-windows 1 N/A N/A Yes Yes N/A Yes
tardebug 1 N/A N/A Yes Yes N/A Yes
tdl-custom-library 70 No No No No No No DAL-E A program will have a specific custom-library (i.e. tdl-custom-hosmer-library)
tdl-examples 70 No No No No No No
tdl-integrity-library 70 No No No No No No DAL-A
tdl-main 70 No No No No No No DAL-A
test-infrastructure 2 No N/A No No N/A No ARM v5 rework (uses STREX/LDREX, v5 requires SWP).
test-utilities 1 Yes N/A Yes Yes N/A Yes
tftp-server-library 70 No No No No No No DAL-A Only installed with tdl-main
trace32-extension 999 N/A No No No No No Raise sequence when Durants3_Program purchases
traceaid 2 N/A N/A Yes No No No TQL-5 Need 1.2.0 stable
virtio-net 1 N/A N/A Yes Yes N/A Yes DAL-E can gcc/multicore updated be N/A

TODO: The following list should be evaluated. Harrys looks like it uses TDL.

The following are not considered for updating:

  • Bundles, BSPs (see above), packages described as obsolete (e.g. with version 0.0.0)
  • GCC, MIPS, OpenArbor, RTEMS, SCORE and TRAC packages
  • Various network drivers (apm-cale, apm-cale-prl, dtsec, emac, emac-prl, etsec, etsec-surcouf, fec-earnhardt, fec-earnhardt-prl, fec_andretti, fec_harrys, i210, lance-pnp, marvell-egiga, network, network-fcc-ethernet, network-pro100, network-standard-apps, pro1000, rtl-bouton, rtl81xx, rtl81xx-prl, sungem, tsec-durant, tsec-durant-prl, xgmac, xilinx-gem)
  • Various host packages (ddci-desk, deos-subversion, desk, graphviz, msvcr80, msvcr90, pyPdf)
  • AFDX (afdx-driver, afdx-driver-config, afdx-driver-cvt, afdx-examples, afdx-library, afdx-library-cvt, afdx-tr)
  • ARINC664 (arinc664-examples, arinc664-library)
  • Deos653 other than part 1 (deos653p4-runtime, deos653p4-examples)
  • dev-flash-c90
  • imageapi
  • internet-api-library
  • ioi-pipcBuffer, ioi-rerouter
  • iostream
  • mpc5676resci
  • produce
  • serial-driver
  • static-socket-library

Deos653 P1 Runtime

The runtime has the following TODO in the ARM floating point portion of the 653 process context switch. 

    // TODO: Kernel bases d16-d31 on CPACR field CPACR_D32DIS. Runtime cannot read CPACR.
    //       We cannot use exception during startup to test since ARM manual indicates expected behavior can be UNDEFINED instruction, or NOP.
    //       For now we need to document only supported on platforms with v3 float 16 regsiters
    add      r12, fromStatePtr, #contextSwitchState_floatingPointContext+floatingPointContextState_d
    vstm     r12, {d0-d15}
    // vstm   r12, {d0-d31}  // build-utils currently compiles/assembles with-mfpu=vfpv3-d16 so only 16 registers are provided

OpenArbor currently matches build-utils with vfpv3-d16 option.


I'm open to any options here. I can think of:

  1. We make this a documented limitation
  2. Runtime requires a PRL
  3. Add an interface to the user mode callable PAL APIs to support the runtime
  4. Kernel exposes FPU type in getSystemInfoDEOS or another API

It seems like GDBserver would also need to know if d16-d31 exist, but it may have other options if debug services honored is set.

Decision (32 floats always available) recorded in PCR 10588 comment 20

Retrieved from "https://ddci.zapto.org/ddciWiki/index.php?title=Deos_Multicore_Verf&oldid=56084"