Kernel Benchmark Tests Project
Description
The Deos team has been asked by Wayne King to perform some Deos kernel benchmark testing. This testing is to support Honeywell pursuing a Sikorsky customer funded program to provide an Epic-based platform to host Fly-By-Wire (FBW) and autoflight functions on the new CH-53K helicopter. This is a key opportunity in that business segment, with 156 of these helos to be fielded beginning in 2015 (first flight in 2011). Sikorsky issued a Request for Proposal (RFP) for the "Flight Control Computer" platform earlier this month, with response due sometime in September. A team of folks from Teterboro, Albuquerque, and Phoenix are working the RFP.
Although the applications being hosted on the platform are limited to Flight Controls, the platform itself is planned to be a militarized repackage of Epic hardware components with maximum re-use of core Epic concepts, software, and tooling such as Deos, Core Software, Integration Tool, ASCB-D, NICs, PDD, and ESCAPE. We need to make sure the Epic core systems team(s) responsible for these areas are properly involved in the RFP and (Qutation Estimate (QE) response. Because this platform is not a simple rollover of the Epic platform, and involves many folks outside the original Epic community, there is also high risk of changes being proposed from the Epic baseline which could impact our re-use of Epic software, tools, and cert assumptions without proper understanding of the additional Non-Recurring Engineering expenses (NRE) (as well as staffing impacts).
The results are also to be forwarded on to Amber Ahmann. She would like to evaluate them in relation to the SLS_Program
The source code for the tests, upated to work with the test infrastructure circa 2012 is now in SCM.
News
- 2006-09-18: Benchmarks delivered. Project complete.
- 2006-08-31: From Judith Krieg "I was told I will have a project number today....I will keep you posted."
- 2006-08-29: We are still awaiting Purchase Orders and Timesite codes for this project. From Locklin, Claudia "After this morning's telecom between Scott Esplin, Dave Nyberg and myself, the decision has been made to allow these Purchase Orders to be place. Once costs are transferred to this site, we will apply an estimated unallowable percentage of 15% (based on our 2005 costs realized in our Tech Pursuit pool) to calculate the unallowable costs that will be excluded from our B&P pool."
- 2006-08-25: From user:Stephen.smith@honeywell.com "Jeff Novacek, go me a *.hyp file and hypstart binaries from Tim Beagle and re-reloaded both boot and the flash and now we have joy (I can even rebuild the hypstart file and it works). So now I am off to working on what I wanted to work on at the beginning of the week. I just wanted to say thank you to those who helped."
- 2006-08-22: A request has been made to Wayne King for access to a Primus Epic(tm) Pentium-M module. We will use this computing module to test our benchmarks on.
- 2006-08-21: Pentium-M PAL source has been ported to DESK Agave and GNU/gcc build environment.
- 2006-08-17: Jeff Novacek is going to ship us a copy of the Pentium-M Module PAL source and Hardware Requirements Document.
Tasks
Initial Budget: $30,480
YTD: $36,195
Current ETC: $0
Current EAC: $36,195
| Milestones | Due Date | Estimated Delivery | Delivered | Percentage Complete |
|---|---|---|---|---|
| Implement Benchmark and execute on Epic Pentium-M | 31-AUG-2006 | 15-SEP-2006 | 13-SEP-2006 | 100% |
References
- Epic Pentium-M Module hardware data sheets can be found in the Deos SCM System.
Benchmark test cases
The following test will run 2 times:
- 1st time in a system with 10 processes (processes == partitions).
- 2nd in a system with TBD (some large number) processes.
Each process will contain 6 threads (tasks):
- 2 tick rate
- 1 at tick-1
- 1 at tick-2
- 1 at tick-3
- 1 at tick-4
During the running of these test the network and inetd will not exists to avoid unexpected interrupts. Once the tests complete they shall be created inorder to transmit the results out of the target.
Task Context Switch Time and Partition Context Switch Time
Define schedule before relationships between all tick rate threads such that threads in the same process run in succession. Each thread will read the TSC (directly not via the kernel API) write it into a shared memory object at a specified offset, increment the offset, then wait until next period. The difference in Successive TSC values is the "Task Context Switch Time".
This will be repeated until 1000 task to task switches and 1000 partition switches have occurred then the averages will be computed and reported.
Task Preemption Time
Define one of the threads at tick-4 (or maybe a tick-5 thread) to be a slack consumer, call this the "slow" thread. The slow thread will sit in a tight loop reading the TSC and writing it into shared memory. The first tick rate thread that runs after the system tick will read the TSC and the last TSC value written by the slow thread and store them into shared memory. The difference in TSC values is the "Task Preemption Time".
This will be repeated for 1000 system ticks then the averages will be computed and reported.
Interrupt Latency
Three interrupt modes to test:
- User Mode Interrupts
- Kernel Mode Interrupts
- Low Latency Kernel Mode Interrupts
The test for all 3 involve modifying the PAL to provide access to a timer. If the platform does not have an extra timer the thread timer can be multiplexed into 2 timers (this will have some impact on performance). A user mode app will then program the timer with a pseudo random amount of time and store the expected TSC when the interrupt will fire into shared memory. The interrupt handler will read the TSC and store it into shared memory (for KMIs and LLKMIs it will have to be stored into user readable memory). The difference between the thread written TSC and the Timer's TSC is the KMI and LLKMI latency. The difference in the written TSC and the ISR thread's TSC is the user mode interrupt latency.
This will be repeated for 1000 interrupts of each type then the averages will be computed and reported.
- Note
- This will requier modifications to the PAL for the platform used for benchmark testing :(.
GCJ: On the PowerPC MPC8250 I had to cascade the timers to support the 32-bit requirement for the system tick and thread timer. This trick used up all the timers. I do not know if the next generation MPC8270 has the same problem.
Semaphore Shuffling Time
Have one of the tick rate threads defined as slack consumer not in a schedule before relation block indefinite on a semaphore. Have a slower rate thread read the TSC and store it in shared memory then signal the semaphore. Then have the tick rate thread read and store the TSC in shared memory. The difference between the slow and tick rate TSCs is the "Semaphore Shuffling Time".
This will be repeated for 1000 times then the averages will be computed and reported.
Deadlock Breaking Time
There are no deadlocks in Deos. To ensure a lower priority thread never has a resource required by a higher priority thread Deos provides Mutexes which raise the priority of a thread for an integration defined amount of time. Instead the test will compute the amount of time it takes to lock and unlock a mutex. Have a Tick rate thread read the TSC then lock a mutex, read the TSC again, unlock the mutex and read the TSC. The difference in TSC values will be reported.
This will be repeated for 1000 times then the averages will be computed and reported.
Datagram Throughput
There are 3 forms of inter-process communication commonly used on Deos:
- mailboxes
- PIPC
- IOI
Each will be tested by having a tick rate thread send 4K messages to a thread in another process and measuring the time it takes the send function to run using the TSC.
AL: I wonder if this shouldn't be done for a couple of varying sizes of messages.
This will be repeated for 1000 times then the averages will be computed and reported.
Results
The results of the benchmarks that where delivered can be found here.
Timesite Administrivia
See Timesite_Codes
