By: Vince Asbridge | March 4th, 2020 | SANBlaze
If a test running under SBCert becomes unresponsive to the "Stop" button on the SBExpress Manager page, the test may be stranded. It may, for example, be waiting on a condition that will never be satisfied. This condition indicates an unexpected occurrence that the script did not anticipate and handle correctly.
Use these steps to diagnose and correct the issue.
Step 1: Make a note of the SBExpress box/port/controller/namespace the test is assigned to, along with the test number:
In the above example, the SBExpress box is 1, port is 0, controller is 100, namespace is 1, and the test number is 1, as outlined in red above.
Step 2: Be sure the test is stranded (not simply waiting on a condition) by examining the log of the running script over time. To do this, select the test name link to access the log:
In the above example, a user would click on the NVMe_FormatNVMSecureErase_Verification.sh link, and the status of the test displays:
Observe by the timestamps shown that the test is indeed not making progress.
Step 3: Press the "Pause" button and wait. If the test does not pause, and is not making progress, follow this procedure to abort the test without disrupting other testing:
Observe the name of the test or cut/paste the name of the test.
Open an SSH session to the IP address of the system. Log in as:
Username: vlun
Password: SANBlaze
Access root privileges (note that your admin may have changed the password).
su
SANBlaze
Use ps to determine the Process ID (PID) of the stranded test:
# ps -ef |grep -v grep |grep NVMe_FormatNVMSecureErase_Verification.sh
root 10125 1 8 16:53 ? 00:00:00 /bin/bash /virtualun/webs/web/rest/sanblazes /1/ports/0/targets/100/luns/1/tests1 /NVMe_FormatNVMSecureErase_Verification.sh started
The first number on the line is the PID; use the kill -9 command to abort the
stranded test:
# kill -9 10125
Verify the stranded script is gone:
# ps -ef |grep -v grep |grep NVMe_FormatNVMSecureErase_Verification.sh
The SBCert test manager will move the state of the test to "Warning" and testing will continue in order. The test manager will take at least a minute to move the stranded test to the "Warning" state.
Step 4: Once the test is moved to the Warning state, subsequent tests will resume.
Note that tests in the "Warning" state will respond to "Clear."
Comments