File System Project

From DDCIDeos
Jump to navigationJump to search

Description

Design notes for the infrastructure to manage multiple kinds of files in a Deos environment. See also https://ddci.zapto.org/Wiki/VFS_Project.

The resulting design should be compatible with, or preferably, form the basis for the POSIX/FACE_Project#File_System.

Complications

The file system will be used in many contexts and must support a broad range of use cases:

  • DAL A vs DAL E components at each level.
  • A file Server (something that implements a file interface) accessed directly within a process vs RPC to a file server process.
  • A process, e.g., a File server, may wish to use both local and RPC based files.
  • File ops, esp. reading and writing are handy in both user and supervisor mode.
    • E.g., video memory, test results, memory buffers, etc.
  • File ops will need to block, and the way blocking is implemented may vary depending on the type of scheduler being used.
    • POSIX/FACE vs RMA vs 653.

Namespace Management

Independently of specific file system types, how does Deos translate names to files?

  • In UNIX (I think also POSIX), the notion of a file having an identity tied to an inode is deep.
    • Utilizing inodes means there must be some entity managing the mapping from inode to related data.
    • Such a mapping agent would have to support a broad range of use cases.
  • How do directories interact with the file systems?
    • Similar to inode issue.
      • Either data is kept in inode and directory entry refers to inode, or
      • directory entry has data and aliases need to be managed somehow.

Mounting

Presumption is that file drivers will reside in a library. Each library will specify the file operations it supports. Mounting will create a relationship between a place in the file system and the driver.

  1. A library might have more than one driver in it.
  2. A library probably needs to be passed some initialization information, e.g., clock rates, resource names, etc. Where does that come from?
  3. Libraries should be able to be loaded into multiple processes.
  4. How do libraries do common activities, e.g., memory allocation?

Types of Files

Buffers

A FILE which is just a memory buffer would make snprintf() easy, and would also form the basis of performing buffered file writes, ala sockobj.

Video Memory

Using printf() to a VideoStream would be handy.

Devices

  • Serial ports
  • RAW access to physical media, e.g., disks, flash, etc.
    • The file system on the media could then use the RAW device file anonymously.

File System Files

Speculation is that We could acquire file systems that utilize block oriented file access and get higher order file systems with less cost, e.g., https://www.rtems.org/

File Operations

Devices and file systems must implement #Basic_File_Operations. A runtime library will use those basic operations to implement higher level operations in a file independent manner.

A complication is that not all basic file operations will be of an appropriate design assurance level.

The expectation is that basic file ops, perhaps just read, and write, will be direct calls to the FileOperations structure, and other functions may depend on some sort of library implementation. This separation means that basic I/O can be done in varying contexts without encumbering the caller.

Runtime File Operations

This level of operations utilize the #Basic_File_Operations and implement things like fgetc, getline, etc.

One complication is that, for cost reasons, the set of file operations is likely going to be separated into DAL A and DAL E functions. This means that we'll probably be able to re-use externally developed software for the DAL E portion, we'll likely need to take ownership of DAL A functions.

Basic File Operations

Device Drivers will implement basic file operations in a manner similar to Linux (ref https://static.lwn.net/images/pdf/LDD3/ch03.pdf#page=8).

A FileOperations structure will contain function pointers for read, write, ioctl, fseek, mmap, close, open, etc.

A FILE object will contain a pointer to the associated FileOperations, a current position, and a single void* for the FileOperations to use

Desired changes to other components

File system is a means of coordinating the interactions of many pieces of software, not surprisingly this coordination requires some underlying infrastructure. E.g., opening a file probably requires doing some memory allocation. Requiring that driver library loading all happen in the "main" thread is likely to be problematic. It may not be critical to get all of these things fixed at once, but some have already shown to be problematic, e.g., the LwIP apps part of Embedded_Tools_Footprint_Reduction_Project, and the serial driver for LwIP_PPP_Project.

Kernel

  1. Add adjustNextLibraryStartAddress(delta) as atomic replacement for setNextLibraryStartAddress(delta+getNextLibraryStartAddress())
  2. Make symbol lookups utilize all process images, not just "needed" list.
    • Current restriction to just "needed" is sane, so some means to override current behavior desired.
    • Rationale is because varying usage patterns would be easier to deal with if the symbol was context sensitive, e.g.,
      • Symbols could be RPC or local based on usage, e.g., socket send() if called from within LwIP would be local, from another process would be RPC based.
      • A library could use the malloc() function from the process if the process had an optimized version.

Issues

  1. Do we have 32-bit file offsets, or 64-bit?
  2. What relation to existing device configuration standards?


Retrieved from "https://ddci.zapto.org/ddciWiki/index.php?title=File_System_Project&oldid=23768"