OpenArbor Development Docker

From DDCIDeos
Jump to navigationJump to search

This is how to setup a WSL development environment for developing OpenArbor, debugging OpenArbor, running the automated test system, and other project-management activities.

For a quick guide to building and installing OpenArbor, see OpenArbor Build.

See also OpenArbor Testing.

It is assumed that all command line operations will be done in a bash shell in a desk maintainer environment.


Get a Docker Capable Workstation

Helpful info is provided here: DESK_On_Docker_Project and here DDCI_Linux_Hints

If you are interested in setting up a Linux test machine:
In DESK_On_Docker_Project:
follow the steps under Install Docker, Update your .bashrc, Login to the Docker Server, and Build A Docker Image.
In DDCI_Linux_Hints:
follow the steps under Store password in clear text and Eclipse subversion plugin.

WSL

  1. On a windows workstation, install docker within a Windows Subsystem for Linux (WSL). See the WSL Readme.
    • WSL requires Windows 11 or Win10_22H2, both require specific patch levels.
    • The most recent versions of WSL require that the Windows Features "Windows Subsystem for Linux" and "Virtual Machine Platform" are enabled. Please see WSL2 notes

The OA team should use option #1 in the above instructions to "Use docker within the WSL distribution you just installed."

To shutdown WSL:

  1. start a cmd shell (cmd.exe)
    wsl --shutdown

Create a WSL shortcut

  1. Create a shortcut to cmd.exe in order to launch WSL
  2. Set the shortcut's Target to: C:\Windows\System32\cmd.exe /c wsl -d ubuntu --cd ~

Define Environment Variables

In your ~/.bashrc file, add these to lines at the end of the file: 

PATH=$HOME/scm/Deos/maintainer-tools/docker/branches/mainline:$PATH
export DDCIFLEX_LICENSE_FILE=~/all_prods.lic
export DDCI_FLEXNET_TOOLS_LOC=~/ddci_eclipse_test_harness_dev/dds-manager-e4/build/test-harness/64bit/linux/FLEXnet

Note: If you have already pasted these lines into your ~/.bashrc from section 'Update your .bashrc', in https://deos.ddci.com/ddciWiki/DESK_On_Docker_Project: 

export DDCIFLEX_LICENSE_FILE=the location of your license file
PATH="$PATH":~/scm/Deos/maintainer-tools/docker/branches/mainline
# Update prompt to include "dkr" when inside a container.
if [[ $- == *i* ]]; then #interactive
    PS1='\W\$ '
    if [[ -f /.dockerenv ]]; then
        dkr=$(</.dockerenv)  # /.dockerenv may be empty or contain the categorization of the environment, e.g., dkr or wsl.
        PS1="${dkr:-dkr} $PS1"
    fi
fi

then instead replace the 'export...' and 'PATH...' lines from this block with the mentioned lines.

Checkout the OA development plugin source code

Launch a terminal
Checkout the eclipse plugin code 

mkdir -p ~/ddci_eclipse
svn co  https://deos.ddci.com/svn/DDCI/products/openarbor/openarbor/branches ~/ddci_eclipse/branches
enter password for scoretest

(Note: if you accidentally create the dev workspace while in a docker container, launching OA will result in an error that the .metadata folder is read-only!)

Checkout the test harness source code

Launch a terminal
Checkout the test harness code

mkdir -p ~/ddci_eclipse_test_harness_dev
cd ~/ddci_eclipse_test_harness_dev
svn co https://deos.ddci.com/svn/DDCI/products/openarbor/utils/dds-manager/dds-manager-e4/ dds-manager-e4
enter password for scoretest!

(Note: if you accidentally create the dev workspace while in a docker container, launching OA will result in an error that the .metadata folder is read-only!)

Setup Mount Points

  1. cd ~/ddci_eclipse_test_harness_dev/dds-manager-e4
    ./setup-testharness-mountpoints

Copy all_prods.lic

cp /mnt/sf_Downloads/all_prods.lic ~

Build Test Harness in a Baseline Docker Image

In Docker, the test harness must be built and executed from within a docker container. So, first use the build-docker script to create a baseline dev image (e.g. build-docker kismet). Then run-docker on the created image to build the test harness and create the mount points needed for the test harness. These steps would need to be repeated only if the test harness code changes, or the baseline image was deleted.

  1. Launch a terminal
  2. Create the baseline image (optional)
    ~/scm/Deos/maintainer-tools/docker/branches/mainline/build-docker kismet
  3. Run docker on the baseline image
    run-docker --host-docker -v /mnt:/mnt --name=oa-test-harness-kismet -it --rm ubuntu-kismet-dev
  4. Install docker and subversion in the image
    sudo -E apt-get install --no-install-recommends docker.io libsvn-java subversion rename libwebkit2gtk-4.0
  5. Setup needed mount points
    cd ~/ddci_eclipse_test_harness_dev/dds-manager-e4
    ./setup-testharness-mountpoints
  6. Build the test harness
    svn up
    ./oaTestHarness.sh
  7. Save the updated docker container to a new image
    Open a second terminal
    docker container commit oa-test-harness-kismet oa-test-harness-kismet-dev

Launching Test Harness in a Baseline Docker Image

The test harness needs to be built when the source code for the test harness has been modified. Otherwise, it can be launched.

  1. Launch a terminal
  2. Run docker on the baseline image
    run-docker --host-docker -v /mnt:/mnt --name=oa-test-harness-kismet -it --rm oa-test-harness-kismet-dev
  3. Setup needed mount points
    cd ~/ddci_eclipse_test_harness_dev/dds-manager-e4
    ./setup-testharness-mountpoints
  4. Launch the test harness
    ./oaTestHarness.sh -n

Troubleshooting Notes:

  • If there is an error on test launch in the test harness that reports "To continue you should: docker container stop ddci-flexnet-server-linux-20230119-7181, docker container rm ddci-flexnet-server-linux-20230119-7181, Then re-run this script", the user should delete .../oaTestHarness/workspace/ddses/<dds-name>/license.lic for the dds they were launching with, and then re-launch the test.
  • Run 'make MAVEN_OFFLINE=""' from the .../oaTestHarness/workspace/branches/<branch>/workspace/ dir for the branch you are running tests with. This has been shown to resolve various errors.

Download and install Eclipse

In WSL or from a Linux VM, the Eclipse development environment is installed in /home/<username>/eclipse/2025-03-R (e.g. /home/scoretest/eclipse/2025-03-R). In order to debug a test, the Eclipse development environment must be launched from within a docker container for the DDS being tested

  1. Install the OpenArbor development environment
    Launch a terminal
    1. for WSL:
      Use windows explorer to copy the devkit from \\nx3000\openarbor\Downloads\Eclipse_2025-03-R\oadevkit_wsl_noble_2025-03_R.tar.gz to your Linux Ubuntu Home dir
      for a Linux VM:
      Copy the devkit from \\nx3000\openarbor\Downloads\Eclipse_2025-03-R\oadevkit_linux_noble_2025-03_R.tar.gz to your Linux VM Home dir
  2. Unzip the dev env
    for WSL:
    1. mkdir ~/eclipse
      cd ~/eclipse
      tar xvfz ~/oadevkit_wsl_noble_2025-03_R.tar.gz
    for a Linux VM
    1. mkdir ~/eclipse
      cd ~/eclipse
      tar -C /Eclipse -xf ~/oadevkit_linux_noble_2025-03_R.tar.gz

DDS docker image

  1. To debug a docker DDS, or a automated test:
    Use the OA Test Harness to install a docker DDS (e.g. dds-docker-sales-deos-kismet-20250424)
    Launch a terminal
  2. (Optional) add unreleased components to the image using a bdu64 script
    edit a bdu64 script (e.g. bdu64-sales25a) to contain the desired unreleased packages
    ./bdu64-sales25a --name=dds-docker-sales-deos-kismet-20250424 kismet customer-sales
  3. Run the docker image and allow the mount points defined on the vm to be available in the docker image
    run-docker --host-docker --name=dev-env -it -v /mnt:/mnt --rm dds-docker-sales-deos-kismet-20250424
  4. Install the OpenArbor development environment
    sudo -E apt-get install --no-install-recommends docker.io libsvn-java subversion rename bzip2 zip libwebkit2gtk-4.0
  5. Save the updated docker container to a new image
    Open a second terminal
    docker container commit dev-env dds-docker-sales-deos-kismet-20250424-dev
  6. Now from any WSL terminal you can run the development environment for the dds:
    ./run-docker --host-docker -it -v /mnt:/mnt --rm dds-docker-sales-deos-kismet-20250424-dev
  7. Setup DDCIFLEX_LICENSE_FILE
    export DDCIFLEX_LICENSE_FILE=/home/<username>/all_prods.lic
  8. Setup needed mount points
    cd ~/ddci_eclipse_test_harness_dev/dds-manager-e4
    ./setup-testharness-mountpoints
  9. Launch Eclipse to debug OA source and/or OA automated tests within the docker image
    ~/eclipse/2025-03-R/eclipse/eclipse

Workspace configuration

This should be contained properly in the provided default workspace, but is documented here in case of a problem

  1. Shortcuts are not yet supported. Use the following command to launch the OA dev environment: /Eclipse/2023-03-R/eclipse/eclipse
  2. Create a new workspace such as: /home/<username>/ddci_eclipse/workspace-<branchName>
    • Rationale: Branches come and go, and sometimes you just hose up your working copy. Keeping the workspace and the working copy separate makes it easier to switch branches or delete and re-download your SVN checkouts without having to worry about losing your workspace configuration.
  3. Import Existing projects into workspace
    1. File->Import...->General->Existing Projects Into Workspace
    2. Browse... to /home/<username>/ddci_eclipse/branches/mainline/workspace
    3. OK
    4. Finish
  4. Import the launch configurations.
    1. File->Import...->Run/Debug->Launch Configurations
      1. Set From Directory to /Eclipse/2021-12-R/eclipse/launches
      2. Check the launches checkbox.
        • The DDS Dev Launch is for debugging OpenArbor, the DDS Test Launch is for debugging tests.
    2. If this is your first launch, use Window->Preferences->Run/Debug->String Substitution to create the string substitution variables username, ddsBranchName, ddsConfigName and ddsName as per #Dds_Variables.
  5. In Window->Preferences->Java->Editor->Save Actions
    1. Check the 'Perform the selected actions on save' box
  6. Fix auto-props for both environments.
    1. In your DESK maintainer environment, edit /etc/subversion/config; change the line that says enable-auto-props = no to enable-auto-props = yes. You will also have to uncomment the preceding [miscellany].
    2. In Eclipse, go to Window->Preferences->Team->SVN and change the configuration location at the bottom to point to the /etc/subversion/ location of your maintainer environment, i.e., the result of the shell command cygpath -wa /etc/subversion/
  1. To start OpenArbor: Run->Run Configurations...->Eclipse Application->DDS Dev Launch.

DDS Variables

The imported launch configurations use several environment variables to allow developers to easily switch between multiple installed DDS' when both developing and/or usring the automated test system. It is possible to test multiple bsp's for an installed DDS. The developer should only need to specify the ddsBranchName, ddsConfigName and ddsName. When a DDS is installed using the automated test system (OpenArbor_Testing), the DDS is installed at C:\oatestharness\workspace\DDS-list\$(ddsName). A workspace for that DDS is created at C:\oatestharness\workspace \ddses\$(ddsName\$(ddsConfigName)\workspace.

  1. String substitution settings: Media:linux-debug-testharness-settings.png
Retrieved from "https://ddci.zapto.org/ddciWiki/index.php?title=OpenArbor_Development_Docker&oldid=72941"