OpenArbor 12.1.0 Test Plan and Report LPB D4
From DDCIDeos
Jump to navigationJump to search
About
OpenArbor Windows/Linux 12.1.0
SVN Revision: r106113
SVN Branch Name: mainline
Test Schedule
| Scheduled Test Runs | Start Date | Due Date | Delivered | Percentage Complete |
|---|---|---|---|---|
| Formal | 03-25-2024 | 03-29-2024 | 03-29-2024 | 100% |
Introduction
The automated test suites for Eclipse, supports Eclipse 2023-06-R, which OpenArbor is built upon. The tests were run the on build mentioned above.
A list of PCRs corrected for this release can be found here: OpenArbor_12.1.0
See below for a list of machines that will be used to reproduce/fix test failures.
The test logs can be viewed at:
- \\nx3000\openarbor\TestResults\Release12.1.0
Legend
| Pass | The test passed. |
|---|---|
| Expected Deviation | The test fails and a fix is not going to happen due to technical restrictions or restraints beyond our control. |
| Known Failure | The test fails, but we have plans to fix the problem in a future release. |
| Fail | The test failed. |
| Not scheduled to be tested for this release. | |
| X | Scheduled to be tested for this release. |
| Not Supported. |
Test Machines
| Machine | OS | Comments | |
|---|---|---|---|
| OATESTLAP | Windows 10 (64bit) | ||
| OATEST | Windows 11 (64bit) | ||
| TESTW10E1 | Windows 10 (64bit) | ||
| TESTW10E2 | Windows 10 (64bit) | ||
| testlinux01 | Linux (64bit) | ||
| testlinux02 | Linux (64bit) | ||
| OALinux01 | Linux (64bit) | LCJ Linux development | |
| OALinux02 | Linux (64bit) | NH Linux development backup test machine |
|
| OALinux03 | Linux (64bit) | KT Linux development backup test machine |
|
| OATEST-W11VM | Windows 11 (64bit) | Unusable media:oatest-w11vm.txt |
|
| TESTW10E3 | Windows 10 (64bit) | VM, slow | |
| TESTW10E4 | Windows 10 (64bit) | Unusable media:testw10e4.txt |
|
| TESTW11E1 | Windows 11 (64bit) | Unusable media:testw11e1.txt |
|
| OATEST | Windows 7 (64bit) | Retired | |
| TESTW7P2 | Windows 7 (64bit) | Retired |
Executed Formal Test Runs
The automated OpenArbor Test Suites are executed for each entry in the table below. The "Test Suites" column will indicate what tests were run. The full set of OpenArbor tests are indicated by the name Launcher. A second test suite, CompileRunDebug is used to just test compilation, linking, running and debugging the example projects. The PlatformIntegration test suite is a small set of tests that exercise the more common operations customers use in interacting with the platform.
| Product | DDS Release | Timestamp | Target Architecture |
Available Boards |
Test Suites | Test Machine | Who | Formal Results |
Comments |
|---|---|---|---|---|---|---|---|---|---|
| Deos | DDS-tostones3-deos-jupiter-20240325 | 1711402398 | ARM | DeosIMX8QM (imx8qm) | loadAda | oatestlap | KT | Known Failure | See Testing Notes |
| Deos | DDS-tostones3-deos-jupiter-20240325 | 1711402398 | ARM | DeosZCU102-1 (zcu102) | loadMldTests | oatest | KT | Known Failure | See Testing Notes |
Testing Notes
| ID | Priority | OS | PCR | Status | Affected Tests | Target | Screenshot | Comments |
|---|---|---|---|---|---|---|---|---|
| 1 | Windows | DDCI_PCR:4964 | Known Failure | BuildAll.ada-main | DeosIMX8QM (imx8qm) ZCU102-1 (zcu102) |
Media:lpb-jupiter-D4-windows-imx8qm-adaMain-Instrumented-build-error-3-27-2024.txt Media:lpb-juptiter-D4-windows-imx8qm-adaMain-InstrumentedABC-build-error-3-27-2024.txt Media:lpb-juptiter-D4-windows-zcu102adaSharedLibrary-Instrumented-build-error-3-28-2024.txt Media:lpb-juptiter-D4-windows-zcu102adaSharedLibrary-InstrumentedABC-build-error-3-28-2024.txt |
ada-main has build errors when built in Instrumented and InstrumentedABC configuration. Traceback (most recent call last): File "<string>", line 1, in <module> IOError: [Errno 2] No such file or directory: 'index.txt' 7 B := new Binary_Tree_Node'( I, Empty_Binary_Tree, Empty_Binary_Tree );
^
BINARY_TREE_NODE so allocator raises Storage_Error [4.8(5.4/3)] | |
| 2 | Windows | DDCI_PCR:4964 | Known Failure | BuildAll.ada-shared-library | DeosIMX8QM (imx8qm) DeosZCU102-1 (zcu102) |
Media:lpb-jupiter-D4-windows-imx8qm-adaSharedLibrary-Instrumented-build-error-3-27-2024.txt Media:lpb-jupiter-D4-windows-imx8qm-adaSharedLibrary-InstrumentedABC-build-error-3-27-2024.txt |
ada-shared-library has build errors when built in Instrumented and InstrumentedABC configuration.
in the library. Inspect the partitions and subprograms in the library to get a list of candidates | |
| 3 | Windows | DDCI_PCR:4964 | Known Failure | BuildAll.test-call-overloading | DeosIMX8QM (imx8qm DeosZCU102-1 (zcu102) |
Media:lpb-jupiter-D4-windows-imx8qm-testCallOverloading-Instrumented-Build-Error-3-27-2024.txt Media:lpb-jupiter-D4-windows-imx8qm-testCallOverloading-InstrumentedABC-build-error-3-27-2024.txt |
test-call-overloading has build errors when built in Instrumented and InstrumentedABC configuration. Traceback (most recent call last): File "<string>", line 1, in <module> IOError: [Errno 2] No such file or directory: 'index.txt' | |
| 4 | Windows | DDCI_PCR:5110 | Known Failure | Run.ada-main | DeosIMX8QM (imx8qm) DeosZCU102-1 (zcu102) |
Media:lpb-jupiter-D4-windows-imx8qm-adaMain-videoStream-3-27-2024.png | Video Stream is blank. No exceptions seen in Status Monitor. Update: the ada-main example requires an update to correctly output to the video stream. | |
| 5 | Windows | DDCI_PCR:5103 | Known Failure | AdaBreakpoints AdaTracepoints AdaCodeGeneration AdaCompileAdditionalOptions AdaCompileOutputs AdaRunTimeChecks |
DeosIMX8QM (imx8qm) DeosZCU102-1 (zcu102) |
The AdaBreakpoints and AdaTracepoints tests attempt to click ada-main->deos_main.adb, which does not exist. They should be updated to click ada-main->code->deos_main.adb. Tests passed after this was changed.
The AdaCodeGeneration, AdaCompileAdditionalOptions, AdaCompileOutputs, and AdaRunTimeChecks test uses a resource path that is also incorrect, causing a failure, and need to be changed from ada-main->deos_main.adb to ada-main->code->deos_main.adb. | ||
| 6 | Windows | DDCI_PCR:2608 | Known Failure | mld.windows.VariablesAndExpressionsViews.stringEditing | DeosZCU102-1 (zcu102) | media:tostones-d3-zcu102-VariablesAndExpressionsViews-stringEditing.png media:tostones-d3-zcu102-VariablesAndExpressionsViews-stringEditing-Debugger-Console.txt |
The test changes a variable value to a temporary value, which works correctly. When attempting to assign the original value back to the variable, the assignment fails, and an error is recorded in the debugger console. Media is from lpb D3 Report. | |
| 7 | Windows | PCR:15397 | Known Failure | mld.Suspend mld.StepOperations.disassembly mld.TerminateWhileRunning |
DeosZCU102-1 (zcu102) | media:tostones-d3-zcu102-mld-suspend-fail.png | While in a debug session, these tests attempt to click the Suspend button to suspend a thread. The thread never suspends, causing a failure. Kenny confirmed failure also occurs when using the GDB debugger on C/C++ code. Media is from lpb D3 Report. | |
| 8 | Windows | PCR:15397 | Known Failure | mld.stopPoints.MultiProcessDebugging.testMultiprocessUI | DeosZCU102-1 (zcu102) | media:tostones-d3-zcu102-testMultiProcessUI-thread-never-suspends.png | While the hello-world.exe thread is suspended, the test selects the thread, and places a bp at main() of hello-world.cpp. The 'Resume' button is selected, and the test expects the thread to suspend once the bp is reached. The thread never suspends, causing a failure. Media is from lpb D3 Report. | |
| 9 | Windows | DDCI_PCR:5109 | Known Failure | adaPath.AdaLibraryUnitPartition | DeosZCU102-1 (zcu102) | media:lpb-jupiter-D4-windows-zcu102-AdaLibraryUnitPartition-editor-open-correctly-3-29-2024.png | Test issue: There is a check to see if calculations.ads is opened in the editor. The file is correctly opened in the editor, but the test fails because the function that checks to see if the file is open uses the incorrect file path. | |
| 10 | Windows | DDCI_PCR:5108 | Known Failure | mld.stopPoints.BreatAtCPPMethod | DeosZCU102-1 (zcu102) | Test issue: The test launches a gdb session on the 'stacks' example, and then attempts to issue a 'go to main' command, which causes a freeze. After manually changing stacks.pd.xml breakAtStartup="yes", the test passes. | ||
| 11 | Windows | DDCI_PCR:5107 | Known Failure | mld.expression.ExpressionFormatting | DeosZCU102-1 (zcu102) | Test issue: The test creates Expressions in the debugger, and ensures they populate correctly in both the default and Verbose formatting. The test is failing when attempting to change the Expressions view to Verbose formatting. When the change to Verbose formatting is selected manually from the context menu, the test passes. |
