OpenArbor 10.4.8 Test Plan and Report LPB D1

About

OpenArbor Windows 10.4.8
SVN Revision: r103037
SVN Branch Name: 10.4.8

Test Schedule

Scheduled Test Runs	Start Date	Due Date	Delivered	Percentage Complete
Formal Run	05/08/2023	05/12/2023	05/12/2023	100%

Introduction

The automated test suites for Eclipse, supports Eclipse 2020-R-12, which OpenArbor is built upon. The tests were run the on build mentioned above.

A list of PCRs corrected for this release can be found here: OpenArbor_10.4.8

See below for a list of machines that will be used to reproduce/fix test failures.

The test logs can be viewed at:

\\nx3000\openarbor\TestResults\Release10.4.8

Legend

Pass	The test passed.
Expected Deviation	The test fails and a fix is not going to happen due to technical restrictions or restraints beyond our control.
Known Failure	The test fails, but we have plans to fix the problem in a future release.
Fail	The test failed.
	Not scheduled to be tested for this release.
X	Scheduled to be tested for this release.
	Not Supported.

Test Machines

	Machine	OS	Comments
	OATEST	Windows 7 (64bit)
	OATESTLAP	Windows 10 (64bit)
	TESTW10E1	Windows 10 (64bit)
	TESTW10E2	Windows 10 (64bit)
	TESTW10E3	Windows 10 (64bit)	VM, slow
	TESTW7P2	Windows 7 (64bit)	slow
	TESTW10E4	Windows 10 (64bit)

Executed Formal Test Runs

The automated OpenArbor Test Suites are executed for each entry in the table below. The "Test Suites" column will indicate what tests were run. The full set of OpenArbor tests are indicated by the name Launcher. A second test suite, CompileRunDebug is used to just test compilation, linking, running and debugging the example projects. The PlatformIntegration test suite is a small set of tests that exercise the more common operations customers use in interacting with the platform.

Product	DDS Release	Timestamp	Target Architecture	Available Boards	Test Suites	Test Machine	Who	Formal Results	Comments
Deos	DDS-tostones3-deos-jupiter-20230508	1683583004	arm	DeosZCU102-1 (zcu102)	Platform Integration loadAda loadMldTests	TESTW10E1	NH	Known Failure	See Testing Notes
Deos	DDS-tostones3-deos-jupiter-20230508	1683583004	arm	DeosIMX8QM (imx8qm)	Platform Integration loadAda	OATEST	NH	Known Failure	See Testing Notes
Deos	DDS-tostones3-deos-jupiter-20230508	1683583004	arm	QEMU-ARM (qemu-arm)	Launcher	OATEST	NH	Known Failure	See Testing Notes
Deos	DDS-tostones3-deos-jupiter-20230508	1683583004	ppc	QEMU-PPC (qemu-ppc)	Launcher	OATESTLap	NH	Known Failure	See Testing Notes

Testing Notes

ID	Priority	OS	PCR	Status	Affected Tests	Target	Screenshot	Comments
1		Windows		Expected Deviation	Debug.trasher Run.trasher Debug.worker Run.worker	All Targets	Media:TrasherWorkerBuildFailure2.png	Platform project fails to build when the Multicore-memory-pools example projects trasher and worker are integrated into the out of the box single core and/or single pool platform. The MulticoreIntegration test performs the BuildAll, Run and Debug tests for the multicore memory pools examples in a multicore/multi memory pool environment.
2		Windows		Expected Deviation	BuildAll.ada-shared-library	All Targets	Media:ada-shared-library-errors.txt	See Media link for errors when integrating ada-shared-main and enabling instrumented build configurations for ada-shared-library. No plans to fix.
3		Windows	DDCI_PCR:4733	Known Failure	deos.StatusMonitor	QEMU-ARM (qemu-arm)		Attempting to change Reset Pad values in Status Monitor causes the qemu-arm emulator to stop responding. The emulator must be restarted in order to be used again. Error message in the emulator says the Reset Pad values entered must be negative.
4		Windows	DDCI_PCR:2608	Known Failure	VariablesAndExpressionsViews	DeosZCU102-1 (zcu102) QEMU-ARM (qemu-arm) QEMU-PPC (qemu-ppc)		OpenArbor should deal with large strings in slices (see DDCI_PCR:2562) for both display and assignment.
5		Windows	DDCI_PCR:4734	Known Failure	Run.ada-main	All Targets		The test needs additional setup to provide output: compile everything PIC (properties -> Ada Compile, check Emit position-independent code and properties -> Project -> Ada, Additional make_partition options: -pic).
6		Windows	PCR:14663	Known Failure	deos.DdsInstallerTest	DeosZCU102-1 (zcu102) DeosIMX8QM (imx8qm)	Media:lpbd1-setup-log-filtered.txt	From AL: Here are the remaining items: mount: /tmp: Invalid argument configureScoreDoc.py: warning: Cannot find SCORE installation The "mount" issue is a longstanding problem. I could add a filter rule for it, but some solution is probably warranted, I just don't know what it would be. Whether missing SCORE is an error I assume is a customer specific issue. The message can certainly be filtered out unconditionally if there is some other way to ensure the content of the DDS is complete. The filtered output from the setup.log.full file's additional output lines were reviewed by AL and none are show-stoppers. The docbook4 output is due to the DDS running /etc/postinstall/deosbook-devel.sh and will be fixed in a future release.
7		Windows		Expected Deviation	CffsMal	DeosIMX8QM (imx8qm)		Mal file: [C:\oaTestHarness\workspace\DDS-list\DDS-tostones3-deos-jupiter-20230508\desk\etc\cffs-emmc-a.cd.xml] does not exist in the DDS desk\etc folder.
8		Windows	PCR:14500	Known Failure	Debug.blackboard_event Debug.cffs_demo Debug.custom-io-partition1 Debug.custom-io-partition2 Debug.demo653 Debug.except Debug.health_monitor Debug.hello-world-ist Debug.hybrid-io-partition1 Debug.ipc-partition1 Debug.ipc-partition2 Debug.ipc-partition3 Debug.ipc-partition4 Debug.ist-partition-1 Debug.ist-partition1 Debug.ist-partition2 Debug.mms_partition1 Debug.mms_partition2 Debug.mms_partition3 Debug.multi-wat-critical-process Debug.multi-wat-process2 Debug.multi-wat-process3 Debug.multi_partition1 Debug.multi_partition2 Debug.multicast-ist-ioi Debug.photon Debug.sockets653 Debug.stacks Deos653GdbTest gdb.LaunchGdbTest	QEMU-PPC (qemu-ppc)	Media:debug-interrupt-failed.png	Launching a debug session on the examples, and issuing a suspend results in an Interrupt Error dialog being displayed. Also manually tested QEMU-PPC with two different DDS' (celestial and deserteagle) and got the same error. To reproduce (1) integrate blackboard_event into qemu-ppc, (2) Updated Target Load (if needed), (3)debug the .exe with gdb, and (4) click Suspend. This is not a problem on qemu-arm.
9		Windows	PCR:14500	Known Failure	mld.Call mld.MultiThread mld.StepOperations.basic mld.basic.ResumeDebugging mld.commands.DebugCommands mld.commands.EnableDisableTrace mld.commands.Go mld.commands.SetBreakAtAddressCounter mld.expression.ExpressionFormatting VariablesAndExpressionsViews.stringEditing	QEMU-PPC (qemu-ppc)	Media:mld_command_queue_log_.txt	MLD fails to respond when specific commands are sent. OpenArbor displays 'Sending Command' in the Progress view, but never gets a response. This behavior was observed when issuing a Step Continue and GO TO FILE POSITION when debugging photon.exe.
10		Windows	DDCI_PCR:4051	Known Failure	ProjectConverter	QEMU-ARM(qemu-arm) QEMU-PPC(qemu-ppc)		Test update is needed.
11		Windows	PCR:14080	Known Failure	CacheTrasher	QEMU-ARM(qemu-arm) QEMU-PPC(qemu-ppc)	Media:cacheTrasher-utl-details.png	Update Target load times out when attempting to run the cache trasher example. The test waits 60 seconds for the Update Target Load and reconnection to the Load List Manager to complete It takes longer than that, which is not expected. Tried manually and had same result - time out waiting for the update target load to complete. I tried manually creating a pal-ext-stub project in the workspace, but still have a time out error connecting to the load list manager after the update target load. I put a copy of the workspace here: \\nx3000\openarbor\Integration\workspace-cache-trasher.zip