2016-02-27 By Robbin Laird and Ed Timperlake
During a recent visit to Pax River, we had the chance to discuss the F-35 test approach and way ahead with Andrew Maack. F-35 Integrated Test Force (ITF) chief test engineer.
Although we were at Pax River, the F-35 ITF is rooted in an integrated team at Edwards AFB as well, and the two teams, although in different physical locations, are part of the same team.
In 2014, Maack received a Department Navy award (2013 Test & Evaluation (T&E) Lead Tester Award) for his performance and accomplishments.
Maack would be the first to tell you, that the team deserves the credit but the words released by the Navy about the award tell a different story than so often grabs the headlines about the F-35 program.
“This award recognizes a member of the T&E workforce who has excelled at the planning, management and oversight of T&E activities as the Lead Tester for the largest programs,” Capt. Erik “Rock” Etz, the former director, Test & Evaluation (T&E) F-35 Naval Variants, explained. “Andrew’s leadership of the ITF and contributions to the F-35 program have been exceptional, particularly during the dynamic flight test tempo that we all experienced last year.”
Etz underscored Maack’s skillful leadership of the more than 920-member ITF team during the execution of safe and efficient execution of F-35 critical flight test events and his direct contributions to Fleet operations as he wrote the nomination for the award.
PAX RIVER F-35 ITF Year in Review Montage from SldInfo.com on Vimeo.
“Andrew’s exceptional efforts, incredible depth of technical knowledge, and meticulous attention to detail resulted in the safe completion of pivotal test milestones of the F-35 System Development and Demonstration (SDD) phase,” Etz added. “
In August 2013, he led the highly successful F-35B Developmental Test (DT) II sea trials aboard the USS WASP, providing clear direction for a 220-member Pax River ITF team that worked in concert with the more than 900-member crew of the USS WASP to verify F-35B capabilities at sea.
With safety of test as his primary focus area of effort, Andrew ensured that a high-paced tempo of flight test operations was executed in a safe manner that guaranteed the well-being of all embarked personnel and preservation of each of the participating uniquely-configured test aircraft (BF-01 and BF-05).
Each aircraft and team provided complementary execution paths to ensure that the aggregate body of collected data would provide the most robust data package for certification of Low-Rate Initial Production (LRIP) aircraft for the U.S. Marine Corps Initial Operational Capability (IOC) and fleet at-sea operations.
Testing was executed in areas of performance (including minimum performance Short Takeoffs), flying qualities (including high crosswind short take-offs (STOs) and vertical landings), loads, alternative approaches, first at-sea night operations, mission systems instrument carrier landing system (ICLS) approaches, in-motion alignments, propulsion (first at-sea tests for In-Service Release F-135), among other flight test disciplines.
The DT-II achieved the highest rate of execution yet displayed on the program (more than 90 sorties generated in just over two weeks, with approximately 220 test points for score).”[1]
Maack has many years of experience not just on the F-35 program but on other test programs, and it was from his total comprehensive experience at testing that he discussed with us the F-35 program.
He made several points, which we will focus on below.
First, developmental testing goes on throughout the life cycle of a combat air program. He noted that developmental testing with the F/A-18 was ongoing and, with a software upgradeable airplane such as the F-35, developmental testing would not end until the program did.
Second, the F-35 program is more complex than programs before it, and they had to craft a test approach that fit the complexity of the aircraft and program itself. The integration of both Edwards and Pax to conduct ongoing synergistic testing was a key point in that direction.
Third, the program is one of “spiral development” in which combat F-35 Type/Model/Series (T/M/S) airplanes emerge throughout the process to operate as effective combat assets, even while the developmental testing for all three types of F-35s continue. Put bluntly, the F-35B in the hands of the Marines is a fully combat-capable asset that will evolve over time.
Fourth, the airplane is the most integrated combat jet ever built, in terms of systems, fusion, and software. And, even though integrated, it is a very robust system in which cascading issues from any particular component or system really has limited effect on the overall F-35 air combat system. This robustness is found in the integrated systems’ ability to continue functioning in the presence of component failures. Thus, it is a safe combat aircraft.
Fifth, to put the team together required a different cultural approach, whereby specialists must act as integrators across the airframe and avionic components of the airplane. This is clearly different from legacy programs.
Sixth, the decade of putting together the unique and innovative approach to shaping the F-35 fleet is laying down the foundation for the decade ahead in which the services and partners generate the combat experience — which will then lead to further innovations and developmental testing.
Put bluntly, if you waiting for the end of developmental testing come back in 30-40 years.
Meanwhile, the F-35 fleet will have reshaped air combat operations.
Question: How integrated is the test force here and at Edwards?
Answer: Very. We talk and work together every day. A lot of times, people want to draw a distinction between Pax and Edwards, and that is really not applicable to this program. What is unique about the F-35 program really is the scope and scale of what we are doing.
It is not only different from what we have done before, but frankly there are some very real-world throughput issues. By having both Pax and Edwards integrated, we can expand the test space.
Let me give you an example. The data generated and communicated by the F-35 is unprecedented. This means that from a testing point of view we run up against telemetry limits. What we can use in DoD-related telemetry from the planes in flight is really pushed into a very tight box. We are limited to four or five airplanes at a time as the max that we can operate for the kind of bandwidth we are pulling off of F-35s at any given time.
The test data from what we call the orange instrumentation on the plane — specialized flight test instrumentation like pressure transducers and load strain gauges and things like that — are part of the significant bandwidth we transmit. And, when combined with bus data, it is indeed a very large bandwidth compared to our legacy programs.
Question: Developmental testing is not a one-off to get the planes out the door until the next model series arrives. What is the role of developmental testing in the life cycle of a combat airplane?
Answer: Put simply, developmental testing is for the life cycle of the airplane. People would be surprised, if they do not follow the business, at how much developmental flight-testing goes on today on the F/A-18. There are continuous needs generated by new weapons, or communication requirements, especially with the F-35 coming to the fleet.
With the F-35, clearly this type of developmental testing is for the life cycle. There is no reason to expect otherwise.
Question: It is clear that you are having to adjust the test culture to deal with the F-35.
How would you describe the change?
Answer: One aspect is clearly with regard to testers themselves. With legacy aircraft, you tended to have various specialists — a hydraulics expert, or flight controls expert, or a radar expert.
On the F-35, unlike anything that I’ve dealt with before — and I’ve been in test and evaluation for 30 years now — you find you certainly still have specialists in those areas. However, it’s not satisfactory to only know that system because it is so interactive across the airframe from a software standpoint, from a controls standpoint, that it really requires everyone on the team to be an integrator. You need to be an integrator across all the different disciplines on the airplane, and that’s been a challenge.
Question: In this past decade, you have established your basic test approach, which really is built around spiral development.
And this decade is laying down the foundation for the next where the combat experiences of the service and partner fleet become folded into the next round of development of the aircraft.
How would describe the process that you have established?
Answer: We have been developing three different variants of the F-35. It was clear that the services would need the aircraft prior to some notional finished aircraft.
And so the program was intentionally put into a spiral development type of mode in which there were going to be defined blocks. And each block represents a combat-ready variant of the aircraft, or of a particular model of the aircraft.
We defined all the different capabilities of the airplane that were going to be shaped over time. The end point, so to speak.
And then we divided them up into blocks in which there were going to be useful war fighting capabilities. Those were provided at incremental blocks to the airplane.
At the same, there was a flight envelope established as an expectation for each of those block developments.
All of our test planning was scoped around being able to clear — for flight sciences — a given envelope to meet a milestone promise date for the particular block in question. And, from the mission system side, developing those missions systems and weapons that were promised for each of those blocks.
For example, when the F-35B Block 2B became cleared for IOC, there were many stories about what it cannot do; that really is not the point. The plane will evolve its capabilities over time based on spiral development. The point is that it is a very capable combat jet at the block it has achieved already.
And the impact is immediate.
Stealth from the sea is brand new for the Marine Corps and Navy.
Question: How do you manage risk in the program?
Answer: Everything has to be evaluated, and our team does an excellent job of evaluating the risk at hand versus the need for the test. A high-risk test, which has a high value, is the flight that you should be flying more so than the flight that is next to nothing for risk but is questionable as to whether it even has to be flown from a value perspective.
The very first level of risk reduction, of what we do in a program like this, is to scope out what is really necessary to do and what is not necessary to do.
Your best safety net is creating a culture that is not risk averse but is positive, proactive, and dedicated to moving forward, yet at the same time does not compromise safety.
You need sound reasons for doing your testing; not just testing for testing’s sake.
Question: In your testing, how robust have you found the F-35 as baseline aircraft to be?
In test, the concern about cascading failure states is always very high when you start to operate the airplane. This concern is based on the general fact that once you integrate everything, a small problem all of a sudden can cascade into an enormous problem.
What I will say has been absolutely outstanding in the F-35 design has been the robustness of failure accommodations on the airplane. When systems fail, it is difficult for me to come up with examples of things that have not generally displayed tremendous — tremendously good — robustness. And, by robustness, I mean being able to isolate the problem while being able to continue operating the aircraft.
Editor’s Note: Below is a helpful overview with regard to the Pax River test facilities.
Editor’s Note: A press release from the Joint Program Office dated February 10, 2016 highlighted passing 50,000 flight hours for the F-35 fleet.
F-35 Lightning II aircraft operating at 12 different locations worldwide surpassed the 50,000 flight hour mark this month.
The first flight hour was achieved by an F-35B aircraft, BF-1, June 1, 2008. The 25,000 flight hour milestone occurred in December 2014, six years and six months later. As a sign of program growth and maturity, the second 25,000 flight hours were reached only one year and two months later.
“The F-35 program continues to grow and accelerate as we complete additional flight testing and increase deliveries to our U.S. and partner warfighters,” said Lt. Gen. Christopher Bogdan, F-35 Joint Program Executive Officer. “The next 50,000 hours will be achieved much quicker as we double the size of the F-35 fleet worldwide in the next three years alone.”
Flight hours are divided into two main categories: Operational flying hours, flown by 155 jets delivered to six different nations, and System Development and Demonstration (SDD) flight test hours, flown by 18 aircraft assigned to the Integrated Test Forces at Edwards AFB, and NAS Pax River. Of the 50,000 hours, operational jets flew approximately 37,950 hours while SDD aircraft flew 12,050 hours. More than one third of the program’s flight hours were flown in 2015 alone. Among the three variants, approximately 26,000 hours were flown by the F-35A, 18,000 hours by the F-35B and 6,000 by the F-35C.
F-35s are flying at eight operating locations: Edwards Air Force Base, California, Eglin AFB, Florida, Hill AFB, Utah, Luke AFB, Arizona, Marine Corps Air Station Beaufort, South Carolina, MCAS Yuma, Arizona, Naval Air Station Patuxent River, Maryland, and Nellis AFB, Nevada. Jets are also flown at two F-35 depot locations at MCAS Cherry Point, North Carolina, and the Ogden Air Logistics Complex at Hill AFB, Utah. Flight hours were also recorded at the two F-35 production facilities at Cameri, Italy, and Fort Worth, Texas.
To date, more than 250 F-35 pilots and 2,400 aircraft maintainers from six nations are trained and more than 110 jets are jointly under construction at both production facilities.
Three distinct variants of the F-35 will replace the F-16 Fighting Falcon and A/OA-10 Thunderbolt II for the U.S. Air Force, the F/A-18 Hornet for the U.S. Navy, the F/A-18 and AV-8B Harrier for the U.S. Marine Corps, and a variety of fighters for at least ten other countries. Following the U.S. Marine Corps’ July 2015 combat-ready Initial Operational Capability (IOC) declaration, the U.S. Air Force and U.S. Navy intend to attain service IOC this summer and in 2018, respectively.
[1] Maack Honored with the Department of the Navy’s 2014 Test and Evaluation (T&E) Lead Tester Award, Pax River Press release, 2014 0501.