Deos 32+bit Physical Address Project

From DDCIDeos
Jump to navigationJump to search

Project to add greater than 32-bit physical address support to Deos.

Background

The need is to support 36-bit physical address platform resources for the APM86190 for the Goodrich SIS Customer. The current strategy is to add Deos support for 64-bit physical addresses, with the #DEOS-RAM-32-RESTRICTION. All Kernel Boot, API and PPI interfaces will be binary backward compatible. APIs that use 32-bit physical addresses will either generate a deprecation warning, or be source code non-backward compatible (but be binary backward compatible).

Status

DDC-I received a "down payment" in early November 2011 to start work. The SOW has not yet been finalized.

Hypstart and hypdump have been updated.

Deos currently boots up to PALColdstart() on the 64-bit DeosAPM86290, and on pre-existing 32-bit platforms.

The Jan 18th deliverable is a non-verified release that must support DDS development. Verified components are due October 2012.

Jan 18, 2012 Component Status
Component Assignee(s) Cygwin "Unreleased"? Comments
Kernel Ryan and Aaron Yes Mostly code-complete (requirements not updated yet). PCR:5913
Integ tool Gary Yes 2.7.0 unreleased. Schema 9, kernel version 7.10.0
Hypstart/Hypdump Ryan and Aaron Yes Minor updates pending due to latest kernel changes. PCR:7319
DESK Python tools Ryan and Aaron Yes Minor updates due to hypstart/hypdump changes. PCR:7318
Boot Carlos (Avionics) No Flashed onto DeosAPM86290
PAL Carlos (Avionics) No
Network Driver Adina No

Implementor Notes

PAL development on this platform should be possible after installing the unreleased versions of the above components. Driver development is not yet supported because 64-bit platform resources are not yet available.

Some notable API changes/issues related to PAL development as yet undocumented:

New APIs: 

 mapPhysicalAddress64() (see diffs).
 platformResourcePhysicalAddress64() (see diffs).

Note also that virtualAttributes() and VirtualAttributesPPI() now support 64-bit architectures.

Kernel Implementation Notes

hyperstart either makes a 32 or 64-bit physical address image based on the processor sub-architecture. A single kernel executable supports both. The primary difference is that 32-bit physical address paging structures are as before (2-level 1024 4-byte entries in Page Directory and Page Tables), and 64-bit physical address entries are three level 8 byte entry tables (PDT2, PDT, and Page Table) having 4, 512, and 512 entries respectively. A whole page is consumed for the PDT2.

DEOS-RAM-32-RESTRICTION

Even the 64-bit solution will only support Deos RAM and FLASH with 32-bit physical addresses. The primary driver here is page table structures, but it also affects other physical structures such as RAM and FLASH free lists.

This restriction can be lifted in the future, but it makes it possible to meet the customer early delivery date, and we'd probably keep that restriction through their cert.

At this time, we don't think there is anything we can do to prepare for 64-bit virtual addresses. I.e., we're assuming 64-bit virtual is going to be not-backward compatible with 32-bit Deos.

PhysStartup

The kernel is entered in "physical" address mode. If the physical address is larger than 32-bits, this doesn't work, but given #DEOS-RAM-32-RESTRICTION there are no substantive issues.

Long term solution probably involves getting the kernel out of the file system business.

VAS

On 64-bit systems, VAS class instances change from a page containing 32-bit PDT entries to a page containing 4 64-bit PDT2 entries (we waste most of a page).

Terminology:

  • Page
  • PageTable
  • PageDirectoryTable
  • PageDirectory2Table (4 64-bit entries)
    • Is 32 byte aligned adequate for x86?
  • VAS32 refers to a 32-bit physical address space
  • VAS64 refers to a 64-bit physical address space

PageDirectoryTable is always present for VAS32. All 4 PageDirectory2Table in a VAS64 are always present.

Hyperstart determines whether the kernel will be VAS32 or VAS64.

Internally to the kernel all resource physical addresses will become 64-bit. Most remaining physical addresses will stay 32-bit, except those the physical addresses needed to support resources. Ref #DEOS-RAM-32-RESTRICTION.

The choice between 32 and 64-bit entries is encoded as a processor sub-architecture.

  • Some subarchitectures will always be 32-bit, some always 64, x86 may be only one that is both.

Portals were not on original estimate.

  • 32 vs 64-bit entries means we can only use every other entry. This may restrict the number of threads that can be created to ~2k.

RAM Class

Issue: Supporting more than 4GB of pages of Deos RAM could cause portal virtual address limitation issues.

Given #DEOS-RAM-32-RESTRICTION, no (significant) changes anticipated.

Exception handlers

  • Separate handlers for 32-bit and 64-bit.
  • #DEOS-RAM-32-RESTRICTION enables restriction to only 32-bit page table addressing.

Chapter will become dynamic (either 2MB or 4MB). This probably won't be a substantial issue for users, but it will affect a number of tests.

  • Issue: should chapter size be added to getSystemInfoDEOS()?

Backward compatibility

Issue: Is 32+bit Deos binary compatible with 32-bit Deos?
Issue: Is 32+bit Deos binary compatible with older 32-bit Deos?

  • Boot. Physical addresses in bootiface and SRT would require changes, * but given #DEOS-RAM-32-RESTRICTION no changes are anticipated.
  • PAL? Could.
  • API?
  • PhysicalAddressTYP:
    • Define 32 and 64 bit variants.
    • Either deprecate or remove PhysicalAddressTYP.

The following APIs which have potential issues for changing physical address size.

  • BIT: Given DEOS-RAM-32-RESTRICTION, all will be API backward compatible.
    • testPageTables()
      • Issue is the failedAddress member of the error log struct.
      • Given DEOS-RAM-32-RESTRICTION, this is backward compatible
    • testPageTablesEx()  ;; Not supported on 64-bit?
      • Issue is the failedAddress member of the error log struct.
      • Given DEOS-RAM-32-RESTRICTION, this is backward compatible
    • testPageOfRamCells()
      • Issue is the failedAddress member of the error log struct.
      • Given DEOS-RAM-32-RESTRICTION, this is backward compatible
    • testRamAddressDecodeLogic()
      • Issue is the failedAddress member of the error log struct.
      • Given DEOS-RAM-32-RESTRICTION, this is backward compatible
  • Platform Resources:
    • platformResourcePhysicalAddress()
      • The address parameter is currently (incorrectly) typed as void*
      • Issue: This is problematic for 32+bit support.
      • Proposal:
        • Introduce platformResourcePhysicalAddress64()
        • Have platformResourcePhysicalAddress() return an error code if high order phys address bits are set.
        • Rationale:
          • Cost is low (implementation and testing)
          • Many network drivers would be incompatible otherwise.
    • platformResourceDescription.startingPhysicalAddress
      • Used by PRLs.
      • Issue: This is problematic for 32+bit support.
      • The existing PRL drivers:
        • EMAC
        • RTL81xx
        • JBR may be working on a third.
  • Virtual memory services:
    • virtualAttributes() and the associated macros from pageattr.h
      • Virtual attributes would be backward compatible 32 to 64 on x86 since low order value is returned in EAX.
      • PPC is screwed since low order bits are returned in r4, not r3.
      • Proposal 1:
        • introduce virtualAttributes64() or vaEX()?
        • virtualAttributes() causes an exception if any high order address bits are set.
      • Proposal 2:
        • Cause virtualAttributes() to be 64-bit
        • Re-release PPC users (mostly network drivers), locking down old versions for Denali.
      • Proposal 3:
        • Change return type to be a struct where the words are ordered to be PPC backward compatible.

Suggest we go with proposal 2.

virtualAttributes is used by network drivers, kernel tests, and perhaps (ISSUE) PAL tests.

Network Drivers using virtualAttributes:

  • rtl81xx (level E part of PRL based driver, x86 is backward compatible, but PRL will be broken anyway)
  • EMAC (level E part of PRL based driver, used by ep440c, hotdish-aid, lutefisk-aid)
  • FCC (level A partitioned driver, reference is in LRPC library, used by rattler8280, ep8280, cmu-mkii-plus)
  • marvell-egiga (unreleased)
  • pro100 (x86 is backward compatible)
  • pro1000 (verified, but x86 will be backward compatible
  • SCC (No Cygwin package)
  • tundra-tsi109 (No Cygwin package)
  • Development Environment Services
    • getTestInfo(), the returned DEOS_TEST_INFO struct:
      • bootReturnAddressPtrPtr could be a physical address
      • Given DEOS-RAM-32-RESTRICTION, this is backward compatible
  • PAL
    • mapPhysicalAddress()
      • add mapPhysicalAddress64() with a new cache policy argument.
      • Rationale:
        • mapPhysicalAddress() is used lots of places
        • We've wanted to add the cache policy argument.
        • Minimal cost.
    • PALVideoMemoryAddress()
      • Proposal is to make this part of BIO and eliminate PAL function.
    • virtualAttributesPPI()
      • Change to be 64-bit return value
      • Change the name?
      • Rationale:
        • No known uses outside of test.

Add 64-bit support for:

  • Video Library
    • Desirable for debug/test
  • ANSI accept 64-bit args to funs, e.g., *printf()?
    • Not required

Programatics

What is our baseline?

  • 7.6.2?
    • Branching would force rework in exchange for lowered cert date risk.
    • What limitations or other PCRs fixed on mainline would need to be ported?
  • Mainline?
    • Any work would be a direct benefit to future work
    • We'd undoubtedly have to reduce cert scope.
      • No MIPS

Non-backward compatible changes, when would they be introduced? It would be kind of weird to say that 7.9.x is compatible with 7.6.2, but not 7.6.3, etc. How would we keep that all straight in our distributions?

Short term

There is a need to get the platform "pinging" in 11 weeks (not sure if that is calendar or work weeks), so somewhere around Feb 1st, but we don't have the SOW finalized yet.

Focus on getting platform resources 64-bit capable, assume RAM and FLASH stay in 32-bit range.

Things we could defer for first release:

  • BIT
  • Checksummer
  • Only do 64-bit variants?
  • Constrain RAM and FLASH objects to first 4GB

Plan PCR:5913

Phase 1: 64-bit page table entries, but only for 32-bit physical addresses

  • RR: Label and (for now) use mainline.
  • AL: pagetab (Easy)
  • AL: pageView in work
  • RR: VAS (clone*) Modest or worse
  • AL: checksummer (Easy)
  • AL: Startup (Easy)
  • RR: Exception Handlers (Either 86290, or for an existing CPU with an emulator which would be effectively throw away.)
  • AL: Hypstart (Modest)

Phase 2: Actually make use of 64-bit physical addresses

  • Integ tool
  • resource APIs
  • PRL struct
  • fallout from physaddrtyp removal/change.

Anticipated Restrictions

AL: all addrs in low 32-bits AL: SMO and platform resources an be in high memory, but size is still constrained to existing limits.

  1. Issue: Hyperstart image will have to be in first 32-bits of physical address space
    • The kernel is entered in physical address mode.
    • May be work around for some platforms, e.g., Book-E PPCs
  • Due to RAM free list virtual address restrictions, can only have 4GB of Deos RAM pages.

Issue: is execute from FLASH required? Issue: Need to make more consistent use of size_t in prep for virtual address space size change. Issue: virtualAllocDEOS() UG description mentions pagetables and the quotas required.

Retrieved from "https://ddci.zapto.org/ddciWiki/index.php?title=Deos_32%2Bbit_Physical_Address_Project&oldid=9843"