Deos 64-bit Port

From DDCIDeos
Jump to navigationJump to search

This project is to enable the DDS and embedded components to support 64-bit.

At this time the primary work is on the Kernel_64. The full set of PCRs shows all the work currently in progress.

Get started

You can get early feedback for coding issues by compiling a component using a 64-bit compiler. At present you may not be able to link depending on what libraries the component requires.

To build:

  1. build a kismet docker image using bdu64
  2. common/build-utils/build install # To verify the component builds before making changes.
  3. Update to latest build-utils: common/build-utils/update-externals && svn up common
    1. Review the list of changes from update-externals looking for things that need to be updated.
  4. Update configure.ac to current conventions.
    1. vfile or ftpserver are good examples.
  5. Modify makefiles to replace references to
    1. MORE_FILES_TO_CLEAN += foo which should be replaced with "CLEAN_FILES += foo"),
    2. MORE_CODE_DEPENDENCIES+=foo which normally should be replaced with a direct dependency: "code : foo".
    3. MORE_DOCS_DEPENDENCIES+=foo which normally should be replaced with a direct dependency: "docs : foo".
  6. For source code:
    1. Replace references to setNextLibraryStartAddress and getNextLibraryStartAddress with adjustNextLibraryStartAddress
    2. Resolve compiler warnings.
  7. for XML docs, including config file XML:
    1. Replace encoding="ISO-8859-1" with encoding="UTF-8"
  8. common/build-utils/build install # To see the issues you have to fix

Docker/WSL

svn up ~/scm/Deos/maintainer-tools/docker/branches/mainline  # to get any updates to bdu64 script
bdu64

run-docker -it --rm ubuntu-kismet-dev

/OpenArbor/OpenArbor -data ~/OpenArbor/junk-64-bit
# Create platforms for all  5 architectures
aarch64 arm ppc x86 x86_64
    can ping, run ftp, video streams, and status monitor.
    I'm not certain, but I think I saw a red diagnostic message on the
    video stream from ftpserver on ppc, maybe arm also.
    GDB 64-bit executables are currently just stubs.  Debugging on 64-bit is not not supported yet.

Cygwin

Cygwin is highly experimental at this point.

Download setup.exe Cygwin 2020 broken per Aaron 9/25/23 use 2018 here: Download setup.exe

Since most Kismet packages are currently unreleased the guidance is to install Test (i.e., unreleased) versions of everything. On first install you MUST install the Test version of desk-python-tools or the install will be utterly broken. 

setup --local-package-dir=c:/cygwin/packages --wait --root=c:/cygwin/2022-kismet
  • Select Test in the Select Packages screen
  • choose package deos-maintainer-tools
  • gcc-aarch64-elf
  • qemu-aarch64-[boot,configuration,pal]
  • gcc-x86_64-elf
  • qemu-x86_64-[boot,configuration,pal]
  • When you choose Next, make sure that package desk-python-tools is 4.23.0-8 or newer.

There are two known problems:

  • Setup reports errors when running post install scripts for OpenArbor (two messages).
  • When OpenArbor runs desk-environment.py, the Cygwin path for desk-environment.py is used (i.e., not just /desk/desk-environment.py) and that is confusing the desk-environment.py script.

Notable Changes

  1. Support for 64-bit
  2. Cygwin 2022 Ubuntu 2022 (Cygwin update is pending, was 2018, Ubuntu was 2021)
  3. gcc 11 (was gcc 7)
  4. .exe files and libpal.so are now linked relocatable by default by both openarbor and build-utils unless compiled with gcc 7.
    1. PAL interceptor is thus currently broken. See PCR:15060.
  5. ANSI: printf "z" length specifier (size_t), 64-bit printing on 32-bit, vfile printf interface See PCRs:14989,12087.
    1. TODO: vfile: fdprintf() PCR:14997
  6. build-utils has been updated to no longer use THUMB when generating ARM ELF images.
    1. Still need to update OpenArbor accordingly.

All DDCI supplied .exe files are now relocatable. Most build-utils generated .exe files are not linked with --pic-executable so they can be used as LwIP apps.

Lessons and Issues

  1. Interfaces that take an [unsigned] "int" buffer size are problematic. It makes logical sense that buffers could be restricted to 2 or 4GiB, but then using sizeof() causes type issue that require casts (bad). If unsigned is used and the function return interface says "negative numbers represent errors, positive are byte counts", then the return value is signed and there is the issue of restricting the output to max signed.
  2. All DDCI supplied .exe files are now relocatable and .exe files linked with --pic (not --pic-executable.
    • this would enable address space randomization later
    • This would make updating kernel address space layout easier (not likely critical).
    • Unsure what performance hit this would entail, if any.
    • --pic not --pic-executable enables .exe files to be used as network apps.
  3. The TLS quota computed by the IT for 64-bit targets is not sufficient to cover the TLS needed. Not sure if this is because the TLS quota specified by each user is less than is needed or if the IT is doing something wrong. One concern I have is I did not see a way to specify required alignment. For now vfile has been hacked to request more than is needed.
  4. GCC 11 issues a narrowing error for vfile's ioctl.h.
  5. hard float vs soft float ABI warnings.
  6. On aarch64 (at a minimum) the kernel does not appear to be handling user mode exceptions correctly.
  7. There is some confusion about how to handle lib dirs and linker scripts. Ryan and Aaron talked about this on 2023-06-30 and have a proposal:
    1. For GCC 11, OA should not look for a linker script.
      • We'll depend on making everything compiled and linked relocatable, even executables.
    2. Eliminate references to dlladdr.mak. Still need:
      • PAL_DLL_BASE_ADDRESS = 0xF7F80000 # Unless we fix the pal interceptor.
      • Need to ensure all ELF files are linked -shared
      • Should be able to put the above in build-utils.
  8. The linker now warns about having a mix of ELF files with soft float vs hard float ABIs
  9. The kernel is linked as executable, not as a shared object.
  10. Need to resolve build-utils handling of TLS for 64-bit architectures.
  11. QEMU does not support floating point for 86_64. Specifically QEMU only supports mmx when using an accelerator (e.g., kvm), and on x86_64 the ABI specifies that floating point values are passed in MMX registers. The options are:
    1. Just say that x86_64 does not support float.
    2. Try to get QEMU for x86_64 to work in an accelerator.
  12. The changes to ANSI to get 64-bit printf() caused a circularity in the ELF dependencies between ANSI and gnu-language. There are hacks in place now, but: We need to find a way to remove the dependency
  13. Getting gdbserver to support 64-bit is a project. It would be better to get the update to gdb 13 done first.
  14. Need to resolve cygwin warnings: 
    missing a package hint for distribution version 'cygwin-20220401'

Standard Apps Virtual Memory Management

The standard apps executables have been changed to be relocatable. There has been an ongoing issue with multithreaded virtual address space management inside lwip and the apps. Aaron did the following grep to find likely problematic issues: 

 grep -H --color --exclude-dir=.svn --exclude-dir=.cache --exclude=*~ --exclude=TAGS --exclude='*.drawio*' -nirE 'virtual|library.*addr' ~/scm/Deos/products/{network/lwip,vfile/core,ftpserver,status-monitor/status-monitor,video/system-video-stream}/branches/mainline/{code,common} | grep -vE '/reference-xml/|/build-utils/|getLibrarySymbolAddress' | sed -e 's/ virtual //g' | grep -iE 'virtual|library.*addr'


A first pass showed the following to be problematic, although there are likely others: 

dirs=$(echo ~/scm/Deos/products/{network/lwip,vfile/core,ftpserver,status-monitor/status-monitor,video/system-video-stream}/branches/mainline/{code,common})
grep -H --color --exclude-dir=.svn --exclude-dir=.cache --exclude=*~ --exclude=TAGS --exclude='*.drawio*' -nirE '(set|get)NextLibraryStartAddress|virtualQuery' $dirs

/home/alarson/scm/Deos/products/vfile/core/branches/mainline/code/dal-e/vfilesupport/dev-vm.cpp:110:  if ( virtualQueryEx(vm->procHandle,(const void*)vm->start,&mbi,sizeof(MEMORY_BASIC_INFORMATION)) == sizeof(mbi) &&
/home/alarson/scm/Deos/products/vfile/core/branches/mainline/common/vfile-kfs-accounting/lib-kfs.cpp:168:  simulatedPVA = (PersistentVirtualArea*)pageAlignUp(getNextLibraryStartAddress());
/home/alarson/scm/Deos/products/vfile/core/branches/mainline/common/vfile-kfs-accounting/lib-kfs.cpp:195:  setNextLibraryStartAddress((void*)simEnd);
/home/alarson/scm/Deos/products/status-monitor/status-monitor/branches/mainline/code/command.cpp:802:     void*         nextLibAddress = getNextLibraryStartAddress();
/home/alarson/scm/Deos/products/status-monitor/status-monitor/branches/mainline/code/command.cpp:834:     setNextLibraryStartAddress(nextLibAddress);
/home/alarson/scm/Deos/products/status-monitor/status-monitor/branches/mainline/code/command.cpp:3538:      //if not present, call virtualQueryEx and if it is free adjust currentVirtualAddress by the size of the free region. 
/home/alarson/scm/Deos/products/status-monitor/status-monitor/branches/mainline/code/command.cpp:3542:        size_t status = virtualQueryEx(targetHandle, (void *)currentVirtualAddress, &currentRangeMemoryInfo, sizeof(currentRangeMemoryInfo));
/home/alarson/scm/Deos/products/status-monitor/status-monitor/branches/mainline/code/command.cpp:3545:          r.reportErrorString("virtualQueryEx() returned 0.");

Development distribution status as of 2023-10-04: 

grep -ri 'this is a development distribution' /desk/help | sort

/desk/help/qemu-x86_64-boot-release-notes.htm:    THIS IS A DEVELOPMENT DISTRIBUTION.  All FILES ARE FOR DDC-I
/desk/help/qemu-x86_64-configuration-release-notes.htm:    THIS IS A DEVELOPMENT DISTRIBUTION.  All FILES ARE FOR DDC-I
/desk/help/qemu-x86_64-pal-release-notes.htm:    THIS IS A DEVELOPMENT DISTRIBUTION.  All FILES ARE FOR DDC-I

Component Status

Note: The following are just the apps believed to be needed for standard apps. It is not a complete list of all components needing 64-bit porting activity.

Specifically it does not include:

  • Obsolete: afdx (afdx-driver-cvt, afdx-library-cvt, etc), cffs(cffsdump, cffsapi653p2)
  • CVTs: deos653-cvt, ioi-cvt, kfs-cvt, registry-cvt
  • No 64-bit dependencies: codetrace, deosbook, deosbook-devel, review-process-scripts, traceaid, x9-client
Component Priority Compiles? Unreleased? Docbook5? GCC 11? Tests Run? Assignee Comments
ansi 1 Yes 4.10.0 Yes Yes Yes FA/DED PCR:13822. Substantial effort will be needed for verf on aarch64 (ref PCR:13822#c8). Compiles on GCC 11, but numerous warnings remain, many related to const.
DED 2024/03/15 Release 4.10.1: Tests run on qemu-arm, qemu-aarch64 and qemu-ppc, but don't all pass (PCR:15322 - test-infrastructure issue). One test hangs on qemu-x86_64, and others fail. (PCR:15510, PCR:15552). See problems.mk.
abc-tool 9 No No TBD N/A No JD
crittime-tool 9 No No TBD N/A No RLR
Cygwin-2022 2 No No No No No AL Columns make no sense
build-utils 1 N/A N/A N/A N/A No AL Need to verify updated compiler switches work for TLS. Need to verify CRCs match on Linux and Cygwin.
deos653-config 1 Yes 1.28.0 Yes N/A Yes GK
deosname 1 Yes 6.11.0 Yes Yes Yes Informal AL PCR:14012 Aaron ran tests on ppc, other architectures not tested. David ran the tests on aarch64 and all pass.
DED 2024/03/25 Release 6.11.0: Tests run on all qemu platforms.
desk-python-tools 1 Yes 4.23.0 Yes N/A Yes AL
deos653p1-runtime 1 No x86_64 9999.2.0 Yes No x86_64 Yes RLF PCR:14627 Compiles for aarch64 with warnings. Tests need more aarch64 porting. qemu-arm has regressions over jupiter. Adding x86_64 support is a TODO (requires development of x86_64 specific HAL)
deos653p1-examples 1 Yes Yes Yes Yes Yes informal RLF/MS
deos-rtems-introspection 2 No No TBD No No OAR
deos-time 2 Yes 4.1.0 Yes Yes No FA PCR:15178 All API functions informally tested to work. Note: qemu-x86_64 will crash due to the a qemu bug with the FPU. Tests are not working at this time due to pal-interceptor related issues.
DED 2024/03/12 Release 4.1.0: Tests tpt000 and tpt001 need pal-interceptor to run (PCR:15540). Test tpt010 needs to be updated to not need ipcTickCounterPointer (PCR:15541). All of the tests need 64-bit updates to build for qemu-aarch64 and qemu-x86_64 (PCR:15542). See problems.mk.
deos-training 1 Yes 3.5.0 Yes Yes Yes Informal KT DDCI_PCR:4047
dvms-config 1 Yes 1.3.3 Yes N/A Yes CP
dvms-core 1 Yes 2.2.0 Yes Yes Yes CP/SS DDCI_PCR:4796 Assuming "complete enough" since released in Deos_Sales_Kismet_1A
DED 2024/03/30: dvms tests do not build for kismet.
dvms-examples 1 Yes 2.0.7 Yes Yes Yes CP/SS Assuming "complete enough" since released in Deos_Sales_Kismet_1A
dvms-exfat 1 Yes 2.1.0 Yes Yes Yes CP/SS DDCI_PCR:4824 Assuming "complete enough" since released in Deos_Sales_Kismet_1A
dvms-journal 1 Yes 1.1.0 Yes Yes Yes CP/SS DDCI_PCR:4835 Assuming "complete enough" since released in Deos_Sales_Kismet_1A
dvms-mmc-sdhc 1 Yes 2.1.0 Yes Yes Yes Informal Will H DDCI_PCR:4887 Many, many warnings produced by gnu-language-atomics external required by PRL.
dvms-ram 1 Yes 2.1.0 Yes Yes Yes CP/SS DDCI_PCR:4873 Assuming "complete enough" since released in Deos_Sales_Kismet_1A
fec-andretti 1 Yes 2.4.0 Yes Yes Yes Informal/Partial Will H PCR:15190 PCR:15191 fec_andretti-dev-kit testing skipped as BSP dev-kits are not ready.
ftpserver 1 Yes 11.5.0 Yes Yes Yes Informal CP PCR:13876
gdbserver 64-bit 2 No No Yes No No SD/RLR PCR:15053 32-bit only at this time. Strongly suggest update to gdb 13 be done first.
gnu-language 1 Yes 1.5.0 Yes Yes Yes AL/DED Circular library dependency with ANSI needs to be resolved. Ryan's latest kernel update appears to have resolved the makefile toggling needed for testing between 32-bit and 64-bit. All tests pass for qemu-aarch64 and qemu-arm.
DED 2024/03/14 Release 1.5.0: Tests run on qemu-arm, qemu-aarch64 and qemu-ppc. Atomic and TLS tests fail on qemu-x86_64 (PCR:15517,PCR:15518). See problems.mk.
hypstart 1 Yes 10.0.0 Yes Yes No&Yes RLR installed python test scripts run but simple regression does not work because of platforms moving to pia
inetd 1 Yes 8.1.0 Yes Yes Yes Informal RLF PCR:14061 Starts, but otherwise untested on 64-bit.
integ-tool-command-line 1 Yes 3.62.1 Yes N/A Yes GK
imx8qm-64bit 1 Yes 5.3.0 Yes Yes Yes RLR Assuming "complete enough" since released in Deos_Sales_Kismet_1A
ioiapi 1 Yes 4.3.0 Yes Yes Yes GK PCR:15097 Assuming "complete enough" since released in Deos_Sales_Kismet_1A
DED 2024/03/30 Release 5.0.0: No tests run (PCR:15596).
ioi-config 1 Yes 4.0.0 Yes N/A Yes GK
ist-arinc653-examples 1 Yes 1.4.1 Yes Yes Yes GK
ist-application 1 Yes 2.3.0 Yes Yes Yes GK
ist-config 1 Yes 1.4.1 Yes N/A Yes GK
ist-ioi-examples 1 Yes 1.6.0 Yes Yes Yes GK
kernel 1 Yes 11.0.0 Yes Yes No RLR See Kernel_64 for detailed status.
x86_64 Kernel crashes when FP operation performed.
kernel-examples 1 Yes Yes Yes Yes Yes FA/NH Assuming "complete enough" since released in Deos_Sales_Kismet_1A
lwip 1 Yes 4.7.6 Yes Yes Yes Informal JK PCR:13958 Tested with ftp.
math 1 Yes 3.1.0 Yes Yes Yes DED tests run and pass on all qemu except x86_64. Assume this is the float issue.
DED 2024/03/15 Release 3.1.0: Tests run and pass on qemu-arm, qemu-aarch64 and qemu-ppc. Some tests fail on qemu-x86_64 (PCR:15555). See problems.mk.
mailbox-transport-library(MTL) 2 Yes 2.7.0 Yes Yes Yes DED DED 2024/03/07 Release 2.7.1: Tests run on all qemu platforms available, but fail on qemu-x86_64 (PCR:15524). See problems.mk.
network-standard-apps 1 N/A 1.5.9 Yes N/A Yes informal RLF
pal-ext-stub 1 Yes Yes Yes Yes Yes PZ pcr:15151
pci-config 2 No No TBD No No GK Changes needed. Pointers are not 64 bit aligned. could use size_t instead of uint32_t, rearrange fields, or add padding. https://deos.ddci.com/svn/DDCI/products/bsp/celestial/branches/mainline/code/pal/pci-config/getPciConfigFile.h
qemu-aarch64-boot 1 Yes 1.0.0 Yes Yes Yes Informal NH SW release howto steps 5.2--5.6, 5.8--5.9 complete. TODO: 5.1 (ccb), 5.7(test), and 5.10(make stable) NOTE: CCB must include component ".Multiple". qemu-arm is now also in this same state.
qemu-aarch64-configuration 1 Yes 1.0.0 Yes Yes Yes Informal NH SW release howto steps 5.2--5.6, 5.8--5.9 complete. TODO: 5.1 (ccb), 5.7(test), and 5.10(make stable)
qemu-aarch64-pal 1 Yes 1.0.0 Yes Yes Yes Informal NH SW release howto steps 5.2--5.6, 5.8--5.9 complete. TODO: 5.1 (ccb), 5.7(test), and 5.10(make stable)
qemu-x86_64-boot 2 Yes 1.0.0 Yes Yes Yes Informal PZ
qemu-x86_64-configuration 2 Yes 1.0.0 Yes Yes Yes Informal PZ
qemu-x86_64-pal 2 Yes 1.0.0 Yes Yes Yes Informal PZ
qemu-vm 2 Yes No No No No RLR Need to add mmio "DLL" to qemu-vm on Linux. multi-core qemu-aarch64 and qemu-arm probably don't work.
ring-buffer 1 Yes 5.0.0 Yes Yes No GK DED 2024/03/28 Release 5.0.0: No tests run (PCR:15580). See problems.mk.
rtems-mini-environment 1.5 No No ? No No AL/RLF Would like Sept release to goto OAR to start RTEMS port. Need something ASAP so that we can get it to OAR.
rtems-socket-adapter (integrate OAR code) 2 No No ? No No DEG? Would like Sept release to goto OAR to start RTEMS port.
socket-adapter(for 653) 2 No No ? No No Needed for sockets653 example. Partners like RTI for their TSS.
socket-api-library (SAL) 2 Yes 5.1.0 Yes Yes Yes JK AL ran tests ran on aarch64, but tests have many compiler warnings.
DED 2024/03/12 Release 5.1.2: Tests run on qemu-arm, qemu-aarch64 and qemu-ppc, but don't all pass (PCR:15529 - regress from previous run). Additional tests fail on qemu-x86_64 (PCR:15530). See problems.mk.
socket-examples 2 No No TBD No No FA
startup-gcc 1 Yes 7.8.0 Yes Yes Yes MS MS 2024/03/15 Release 7.8.0: Tests run on qemu-arm, aarch64, and X86_64. The tests do not build on qemu-ppc for the current compiler version (11.2.0.2) due to problems with the embedded ASM commands within the test procedure, and have not been run since the Harry's verification (2019). PCR 15564 was created to investigate/resolve this problem.

Additionally, there is a LARGE number of inline warnings generated by the supporting test infrastructure. This is new, so what changed for Kismet?

static-socket-library 1 Yes N/A Yes Yes Yes Informal JK PCR:13942. static-socket-library is not delivered to customers.
static-video-library 1 Yes 7.5.0 Yes Yes Yes Informal RLF
status-monitor 1 Yes 11.4.1 Yes Yes Yes Informal RLF Changes committed to PCR:13983.
system-video-streams 1 Yes 8.2.0 Yes Yes Yes Informal RLF See PCR:13979.
test-infrastructure 1 Yes Yes Yes N/A Yes Informal AL DED - Needs to be released. PCR:15322 needs to be resolved.
trace32-extension 2 No No Yes No No RLR
uart-imx 1 Yes 1.1.0 Yes Yes Yes informal Will H PCR:15183 Done.
vfileconfig 1 Yes 1.8.0 Yes N/A Yes GK
vfile 1 Yes 7.1.0 Yes Yes Yes FA PCR:13874 Assuming "complete enough" since released in Deos_Sales_Kismet_1A
DED 2024/03/19 Release 7.2.2: Tests run on qemu-arm, qemu-aarch64, qemu-ppc, and qemu-x86_64 but don't all pass, and some don't build. (PCR:15566, PCR:15567, PCR:15568, PCR:15571). See problems.mk.
vfile-api653p2 1 Yes Yes Yes Yes Yes SS/JMK DDCI_PCR:4881 aarch64 only for first release
vfile-api653p2-examples 1 Yes Yes Yes Yes Yes SS/JMK
virtio-net 1 Yes 1.5.0 Yes Yes Yes Informal AL PCR: 13959

Legend:

  • Priority
    • 1 Must be done by "End of September 2023"
    • 2
    • 9 By verf
  • Unreleased
    • Mark Yes when component is unreleased.
  • Docbook5
    • Yes when the component's documentation uses DeosBook 1.0.20 and DocBook5
  • GCC 11
    • Yes when the component has been compiled with GCC 11.
  • Tests Run
    • Yes: Regression suite runs and passes or failures noted in problems.mk. Since this is not a verified release, not all tests are required to pass, but we should understand failures.
    • Yes Informal: For Level E, component "seems to be working".


Retrieved from "https://ddci.zapto.org/ddciWiki/index.php?title=Deos_64-bit_Port&oldid=61776"