Starliner faced “catastrophic” failure before software bug found | Ars Technica
▻https://arstechnica.com/science/2020/02/starliner-faced-catastrophic-failure-before-software-bug-found
Boeing, NASA, and U.S. Army personnel work around the Boeing CST-100 Starliner spacecraft shortly after it landed in December
"If it had gone uncorrected it would have led to erroneous thruster firing."
During its quarterly meeting on Thursday, NASA’s Aerospace Safety Advisory Panel dropped some significant news about a critical commercial crew test flight. The panel revealed that Boeing’s #Starliner may have been lost during a December mission had a software error not been found and fixed while the vehicle was in orbit.
The software issue was identified during testing on the ground after Starliner’s launch, said panel member Paul Hill, a former flight director and former director of mission operations at Johnson Space Center in Houston. The problem would have interfered with the service module’s (SM) separation from the Starliner capsule.
“While this anomaly was corrected in flight, if it had gone uncorrected it would have led to erroneous thruster firing and uncontrolled motion during SM separation for deorbit, with the potential for catastrophic spacecraft failure,” Hill said during the meeting.
Starliner’s December test flight had to be cut short due to a well-publicized timing error that delayed the spacecraft’s service module from performing an orbital insertion burn. This caused the thrusters on board the service module, which provides power to Starliner during most of its mission, to fire longer than expected. As a result, the spacecraft did not have enough fuel to complete a rendezvous with the International Space Station, a key component of the test flight in advance of crewed missions.
At Thursday’s meeting, Hill revealed the second issue related to software and thruster performance publicly for the first time.
However, as part of reporting on a story about Starliner software and thruster issues three weeks ago, a source told Ars about this particular problem. According to the source, Boeing patched a software code error just two hours before the vehicle reentered Earth’s atmosphere. Had the error not been caught, the source said, proper thrusters would not open during the reentry process, and the vehicle would have been lost.
In a response to a query about this in mid-January, a #Boeing spokesperson confirmed to Ars that software uploads were sent to Starliner “near the end of the mission.” However, the spokesperson then downplayed the gravity of the situation, saying, “The final upload before landing’s main purpose was to ensure a proper disposal burn of the Service Module after separation and had nothing to do with Crew Module reentry.” Because this made the issue sound not serious, Ars omitted it from the published story.
But the public remarks by Hill on Thursday appear to underscore the seriousness of the issue, and the safety panel recommended several reviews of Boeing. “The panel has a larger concern with the rigor of Boeing’s verification processes,” Hill said. “As a result, the panel recommends that NASA pursue not just the root cause of these specific flight-software anomalies but also a Boeing assessment of and corrective actions for Boeing’s flight-software integration and testing processes.”
The safety panel also recommended that NASA conduct “an even broader” assessment of Boeing’s Systems Engineering and Integration processes. Only after these assessments, Hill said, should NASA determine whether the Starliner spacecraft will conduct a second, uncrewed flight test into orbit before astronauts fly on board. (Boeing recently set aside $410 million to pay for that contingency).
Finally, before the meeting ended, the chair of the safety panel, Patricia Sanders, noted yet another ongoing evaluation of Boeing. “Given the potential for systemic issues at Boeing, I would also note that NASA has decided to proceed with an organizational safety assessment with Boeing as they previously conducted with SpaceX,” she said.