OpenArbor 12.0.2 Test Plan and Report LPB D3
About
OpenArbor Windows/Linux 12.0.2
SVN Revision: r105601
SVN Branch Name: mainline
Test Schedule
| Scheduled Test Runs | Start Date | Due Date | Delivered | Percentage Complete |
|---|---|---|---|---|
| Formal | 01-18-2024 | 01-19-2024 | 01-19-2024 | 100% |
Introduction
The automated test suites for Eclipse, supports Eclipse 2023-06-R, which OpenArbor is built upon. The tests were run the on build mentioned above.
A list of PCRs corrected for this release can be found here: OpenArbor_12.0.2
See below for a list of machines that will be used to reproduce/fix test failures.
The test logs can be viewed at:
- \\nx3000\openarbor\TestResults\Release12.0.2
Legend
| Pass | The test passed. |
|---|---|
| Expected Deviation | The test fails and a fix is not going to happen due to technical restrictions or restraints beyond our control. |
| Known Failure | The test fails, but we have plans to fix the problem in a future release. |
| Fail | The test failed. |
| Not scheduled to be tested for this release. | |
| X | Scheduled to be tested for this release. |
| Not Supported. |
Test Machines
| Machine | OS | Comments | |
|---|---|---|---|
| OATESTLAP | Windows 10 (64bit) | ||
| OATEST | Windows 11 (64bit) | ||
| TESTW10E1 | Windows 10 (64bit) | ||
| TESTW10E2 | Windows 10 (64bit) | ||
| testlinux01 | Linux (64bit) | ||
| testlinux02 | Linux (64bit) | ||
| OALinux01 | Linux (64bit) | LCJ Linux development | |
| OALinux02 | Linux (64bit) | NH Linux development backup test machine |
|
| OALinux03 | Linux (64bit) | KT Linux development backup test machine |
|
| OATEST-W11VM | Windows 11 (64bit) | Unusable media:oatest-w11vm.txt |
|
| TESTW10E3 | Windows 10 (64bit) | VM, slow | |
| TESTW10E4 | Windows 10 (64bit) | Unusable media:testw10e4.txt |
|
| TESTW11E1 | Windows 11 (64bit) | Unusable media:testw11e1.txt |
|
| OATEST | Windows 7 (64bit) | Retired | |
| TESTW7P2 | Windows 7 (64bit) | Retired |
Executed Formal Test Runs
The automated OpenArbor Test Suites are executed for each entry in the table below. The "Test Suites" column will indicate what tests were run. The full set of OpenArbor tests are indicated by the name Launcher. A second test suite, CompileRunDebug is used to just test compilation, linking, running and debugging the example projects. The PlatformIntegration test suite is a small set of tests that exercise the more common operations customers use in interacting with the platform.
| Product | DDS Release | Timestamp | Target Architecture |
Available Boards |
Test Suites | Test Machine | Who | Formal Results |
Comments |
|---|---|---|---|---|---|---|---|---|---|
| Deos | DDS-tostones3-deos-jupiter-20240118 | 1705589484 | ARM | DeosZCU102 (zcu102) | loadMldTests | testw10e1 | KT | Known Failure |
Testing Notes
| ID | Priority | OS | PCR | Status | Affected Tests | Target | Screenshot | Comments |
|---|---|---|---|---|---|---|---|---|
| 1 | Windows | DDCI_PCR:4964 | Known Failure | BuildAll.ada-shared-library | DeosZCU102 (zcu102) | media:tostones-d3-zcu102-buildall-ada-shared-library-instrumented-abc-build-fail.txt media:tostones-d3-zcu102-buildall-ada-shared-library-instrumented-build-fail.txt |
'Instrumented' and 'Instrumented ABC compliant' build configurations cause the 'ada-shared-library' project build to fail. /desk/bin/link_deos /usr/local/cross-compilers/gcc-7.3.0 arm so -nostdlib -Map ./ada-shared-library.map -shared --whole-archive libada-shared-library.ada.a --no-whole-archive -o libada-shared-library.so -L. -L/desk/arm/appbin -l ada-shared-library.ada -l kernel -l adaroot -l adarts Traceback (most recent call last): File "<string>", line 1, in <module> IOError: [Errno 2] No such file or directory: 'index.txt' make[1]: *** [/OpenArbor/openarbor-common/compiler-rules.mk:261: libada-shared-library.hitmap-size-validated] Error 1 This example is the only Ada example that does not have C/C++ code, which apparently is what causes an index.txt file to be created when building an instrumeted build configuration. | |
| 2 | Windows | PCR:15397 | Known Failure | mld.Suspend mld.StepOperations.disassembly mld.TerminateWhileRunning |
DeosZCU102 (zcu102) | media:tostones-d3-zcu102-mld-suspend-fail.png | While in a debug session, these tests attempt to click the Suspend button to suspend a thread. The thread never suspends, causing a failure. Kenny confirmed failure also occurs when using the GDB debugger on C/C++ code. | |
| 3 | Windows | DDCI_PCR:4572 DDCI_PCR:4468 |
Known Failure | mld.stopPoints.GutterClicking | DeosZCU102 (zcu102) | media:tostones-d3-zcu102-gutter-clicking-fail.png | While in the debug perspective, the mld.stopPoints.GutterClicking test attempts to 'Toggle Breakpoint' from the gutter's context menu. The test fails as 'Toggle Breakpoint' is not enabled within the menu, seemingly due to the focus being on the incorrect place at the time of the attempt. When selecting 'Toggle Breakpoint' from the gutter context menu manually, the test passes.
The test should be altered to ensure the context menu selection is enabled and selected correctly. See the PCRs for more details on why the selection is not correctly occurring. | |
| 4 | Windows | DDCI_PCR:2608 | Known Failure | mld.windows.VariablesAndExpressionsViews.stringEditing | DeosZCU102 (zcu102) | media:tostones-d3-zcu102-VariablesAndExpressionsViews-stringEditing.png media:tostones-d3-zcu102-VariablesAndExpressionsViews-stringEditing-Debugger-Console.txt |
The test changes a variable value to a temporary value, which works correctly. When attempting to assign the original value back to the variable, the assignment fails, and an error is recorded in the debugger console. | |
| 5 | Windows | PCR:15397 | Known Failure | mld.stopPoints.MultiProcessDebugging.testMultiprocessUI | DeosZCU102 (zcu102) | media:tostones-d3-zcu102-testMultiProcessUI-thread-never-suspends.png | While the hello-world.exe thread is suspended, the test selects the thread, and places a bp at main() of hello-world.cpp. The 'Resume' button is selected, and the test expects the thread to suspend once the bp is reached. The thread never suspends, causing a failure. |
