Validation of a COTS EHM Solution for the JSF Program

This paper presents results from a recent study conducted under the JSF program to evaluate and validate the COTS reasoner, TEAMS-RT, selected by Pratt and Whitney for onboard diagnosis of faults in the JSF F135 engine. The approach selected was to compare the TEAMS-RT reasoner's computational performance and diagnostic accuracy against a proven Pratt & Whitney's (P&W) legacy reasoner. The model was auto-generated from high quality information on LRC/module failure modes and rates from the FMECA data, and the relationship between individual failure modes and the built-in fault codes issued by the engine's electronic controls. The data provided by P&W contained fault-test dependency and failure rate data in a spreadsheet format, which was used to generate the TEAMS-RT model. P&W also supplied a rich set of test cases that were used to exercise the TEAMS-RT EHM solution. The objectives of the study were two-fold. The first objective was to use the test cases to validate the automatic model generation process and the diagnostic accuracy of a COTS reasoner (TEAMS-RT) against a proven legacy diagnosis system currently in use by the air force. It was, therefore, important to demonstrate that for the same test cases, the TEAMS-RT results were nearly identical to the ones produced by the reference reasoner. Thus, a secondary goal was to tweak the TEAMS-RT algorithm to mimic the performance of the reference reasoner. The second objective was to investigate ways to improve on the reasoning accuracy beyond that of the reference reasoner. The results show a slight improvement in diagnostic accuracy and substantial improvement in computational performance by the TEAMS-RT reasoner. Simulation results and implementation details are presented in the paper.