Abstract
Modern weapon systems and their associated test equipment are becoming increasingly reliant on software developed specifically for those platforms. The repair, adaptive modifications or upgrades, and change events (including integration and testing) made to operational software resident in a weapon system, or in support equipment, can have major impacts on equipment readiness and sustainment of those weapon systems. This forum will begin with a look at legislative and DoD policy impacts on software maintenance, followed by discussions on how DoD equipment software maintenance is performed and some of the challenges experienced in implementing software maintenance operations in the DoD maintenance community.
Agenda
1300-1309: Welcome and Overview – Greg Kilchenstein (OSD-MR) Presentation
1309-1310: Administrative Notes – Debbie Lilu (NCMS)
1310-1335: Software Sustainment: Statute, Policy, and Accountability – John Sutton (OSD-MR) Presentation
1335-1405: DoD Software Maintenance Study – Chris Miller (SEI)
1405-1430: Transition to Agile and Beyond – Nate Becker (Army Combat Capabilities Development Command) Presentation
1430-1455: Software Data Rights Framework – Curtis Jefferson (AFMC Systems Engineering Division)
1455-1500: Wrap-Up Presentation
Minutes
Event: On 29 September 2020, the Joint Technology Exchange Group (JTEG), in coordination with the National Center for Manufacturing Sciences (NCMS), hosted a virtual forum on “Software Maintenance”.
Purpose: The purpose of this forum was to examine and share information on new processes used to help identify, demonstrate, and transition sustainment capabilities to improve DoD maintenance operations. The forum provided descriptions of processes employed by each of the military Services as well as a program utilized to facilitate industry/academia and government cooperation in capability demonstration and development efforts.
Welcome: Greg Kilchenstein (OSD-MR) welcomed everyone to the forum and thanked the presenters and all the listeners for their attendance. He also stated how important these processes are in driving sustainment innovation and creating synergy to develop, identify, and transition capabilities to support the DoD maintenance community.
Administrative: This was an open forum. The presentations, along with questions and answers, were conducted through Adobe Connect. Four of the six presentations were also available online at the JTEG website at https://jteg.ncms.org/ . A separate audio line was used. We had over 70 participants from across DOD, industry, and academia join in the forum.
Software Sustainment: Statute, Policy, and Accountability – John Sutton (OSD-MR)
provided information on statutes and DoD policies. He pointed out that DoD’s reliance on software continues to grow at a rapid rate in both capabilities and costs, and detailed a GAO recommendation that DoD needs to better capture and report software sustainment costs. Sustainment policy facilitates compliance with the statutory requirement to report organic sustainment costs and level of effort. Lastly, John described current acquisition policy as in transition with regards to any distinction between software development and sustainment.
DoD Software Maintenance Study – Chris Miller (SEI) provided an informative brief on the background of software sustainment, and an overview of the SEI Software Sustainment Study. He described the differences between software sustainment vs hardware sustainment, whereas software sustainment is more like “continuous engineering” vs repair. For example, any software “fix or enhancement” can create more bugs or a new configuration, whereas hardware repairs fix root causes, do not change the configuration, and are repeatable processes. Chris also provided a software sustainment ecosystem overview and discussed the software sustainment community of practice that focuses on DoD’s organic software sustainment capabilities, policies, and software engineering enterprise.
Transition to Agile and Beyond – Nate Becker (Army Combat Capabilities Development Command (CCDC)) explained that the Armament Software Engineering Center (SEC) at Picatinny Arsenal implements and oversees software development and sustainment programs, using advanced measurement and analysis approaches (among other modern methods deployed). He also stated that the influence of Agile software development is starting to be pervasive across DoD programs. By implementing an Agile lifecycle model, the CCDC SEC expects to standardize tracking, reporting and metrics for projects where the following conditions exist: requirements are not fully understood, changes in requirements are expected during development, the majority of development staff is self-organizing, and the customer/user is actively involved during the entire lifecycle.
Software Data Rights Framework – Curtis Jefferson (AFMC Systems Engineering Division) discussed actions the USAF are undertaking to improve acquisition of software data rights. He identified the goal as to develop a standard engineering analysis framework designed to identify, acquire, document, & retain appropriate software data rights. The framework is to include provisions for timely acquisition of government subject matter expertise congruent with utilization of acquired SW data rights, involve cross-organizational involvement (program office and organic agency), and the framework tenets be included as part of core competency. The current status is that the Enterprise IP/SW Data Rights Framework has been developed with the underlying process/approach presented to AFLCMC/EN_EZ, AFLCMC/LZ, AFMC/ENS, AFMC JA, AFSC/EN, etc., and the process is under consideration by AFLCMC for submission as a standard process. They are also exploring the concept of the framework serving the standard Intellectual Property Strategy for software.
Q&A – A Q&A occurred after each briefer finished their presentation. Questions and answers will be posted on the JTEG website with these minutes.
Closing Comments: Greg Kilchenstein thanked the presenters for their contributions and all the work being done to support software sustainment efforts across the DoD. He suggested continuing the information exchange beyond the forum and the importance of collaboration within the DoD maintenance community.
Action Items:
- All cleared briefing slides were posted to the JTEG website at https://jteg.ncms.org/ prior to the forum start.
- Obtain Distribution “A” level slides for the remaining briefs.
Next JTEG Meeting: The next scheduled JTEG virtual forum is 27 October 2020, 1:00 – 3:00 pm EST. The topic is “Emerging Non-Destructive Inspection Capabilities”.
POC this action is Ray Langlais, rlanglais@lmi.org, (571) 633-8019
Q&A
Software Sustainment: Statute, Policy, and Accountability: John Sutton (OSD-MR)
Q1. Is Software sustainment highlighted anywhere in our acquisition policy (5000.01/02)?
A1. The new series is broken down into a number of pathways. 5000.UG is all about software. However, sustainment is not highlighted, though they talk continuously about engineering.
Q2. Why is software accounting in Mx so important? Why does it matter?
A2. Due to the prominence of software. It allows us to keep the functions working.
Q3. How do we differentiate between software maintenance and software upgrade?
A3. It doesn’t make a distinction in the reporting.
Q4. Is there any policy on incorporating Software Maintenance into the Joint DSOR Process?
A4. Already there by the DSOR process.
Q5. As you pointed out, the interim s/w acquisition pathway policy does not discuss sustainment. There is also no DoD policy definition of “software maintenance.” Do you know if OSD is planning to address s/w sustainment or maintenance in the final pathway policy?
A5. I don’t have insight into that pathway.
Q6. John–Have you reviewed the recently published DoD Inst 5000.85 and how it discusses software sustainment? There are 34 mentions of software in the instruction.
A6. No, I don’t know.
Q7. The statutory definition of “depot-level maintenance and repair” expressly excludes “the procurement of major modifications or upgrades of weapon systems that are designed to improve program performance.” It seems the FMR that you cited to be aligned with the statutory definition. Would you agree?
A7. There are some ambiguities in the policy. For example, the USAF co-locates their software repair centers with depots.
Q8. In general, how do you “measure” software quality?
A8. There are a lot of legacy methods still ongoing, but new methods are being developed. Some groups have different polices for measuring defect density, etc.
DoD Software Maintenance Study: Chris Miller (SEI)
Q1. Did your study find anything “special” about software sustainment in the DoD compared to other high-reliability/high-availability industries?
A1. That’s a broad question. The answer is yes. For example, within SpaceX there is only one person who can make a change…Elon Musk. DoD on the other hand has several checks and balances that can slow or even stop the process, resulting in numerous time delays. There is also an acquisition process vs engineering issue.
Q2. Where do you see opportunities for industry to engage with SEI or DASD MR on the topic of software sustainment? Are there working groups or COPs that are open to industry?
A2. That’s more a John Sutton question. In the acquisition and sustainment (A&S) processes, the “S” are widely under-represented. Look at how doing development of software hand-off occurs. The “S” is missing. Any Acquisition forum could benefit from adding Sustainment.
Q3. On the topic of organic software skills — can you comment on the extent to which DoD has the requisite organic capability vs. reliance on support contractors?
A3. It varies, system platform by system platform. Moores Law…the rate of change of technology is so rapid, that support needs to be fluid and able to adapt. PPPs are becoming important to support DoD.
Q4. Can you talk a little about advances in middleware that enable Software as a Service providers to do business in the highly sensitive DOD cybersecure environment? (Tony Delgado)
A4. There is a lot of good. Many cloud approaches exist.
Transition to Agile and Beyond: Nate Becker (Army Combat Capabilities Development Command)
Q1. Can we revisit the earlier question…”What is the difference between software maintenance and software upgrade?”
A1. Sure. Think of it this way: Upgrade is a new capability. A sustainment release does not change form, fit, or function. They may be combined also.
Software Data Rights Framework: Curtis Jefferson (AFMC Systems Engineering Division)
Q1. Data rights can be determined below the CSCI level. Does the analysis factor in the DFARS concept of “segregability”?
A1. Excellent question. You have computer software elements. Make it clear those CSCI items, i.e. display items, weapon items, Don’t mix capability.
Q2. Q2. What rights, if any, does the government get when the software is delivered only as embedded on some hardware component? Does the government need to specify the software as a deliverable independent from the hardware component in which it is embedded, in order to have any rights?
A2. The DFARs specify that entitlement of government rights to software is based solely on governmental funding contribution. In terms of software, the target hardware is immaterial. For sustainment purposes, the software is typically delivered to the government separate from the hardware for which it’s intended to run on or be developed on. Software sustainment must be planned or programmed. So, if the government organic agency is to maintain a software system, the program office must contract for the delivery of a systems integration lab (SIL) with all applicable components – this represents the hardware delivery. Subsequently, as each block of software is developed by the contractor, software is delivered to the government on a per block ( or per increment if using agile-DevSecOps) bases.
Q3. What is the proper concern that the government should have regarding any open source software module included in deliverable software (both commercial and non-commercial), where the open source software license includes a “copyleft/viral” provision requiring that the resulting comprehensive software must also be dedicated to the open source community?
A3. I consulted the Air force Data Rights Guidebook, 2019 on this one. Here are some words from that document: “Legal review generates additional concerns with liability and distribution. Each license can have specific wording that limits the ability for the government to agree to the license… “rights of inspection” of either the source code it is included with or the facilities. For any system at or above the FOUO level, this clause is unacceptable.” I interpret this to mean the government must avoid contractual obligation ( or attempt thereof) to access to weapon system code outside of the government/DoD.
Q4. In your discussion about “complex algorithms,” you seemed to suggest that the government should have appropriate expertise to continue organic maintenance of the software. Do you believe that DFARS allows the government to reverse engineer or decompile non-commercial software? What about for commercial software?
A4. Assuming the government is entitled to at least government purpose rights, If the software sustainment strategy for a program includes sustainment of the target recognition capability which in most cases utilizes one or more data fusion algorithms, then; yes, the government should plan to obtain the appropriate level of subject matter expertise if it is to directly sustainment or change this capability. The DFARs include language in the definition of government purpose rights that indicates the government can use the delivered software for “government purposes”. I would interpret the activity of reverse engineering as a government purpose activity for sustainment of the software. For commercial software cases, the government must adhere to the OEM’s license clauses – per DFARS.