Agenda
1300-1305: Welcome, Intro & Purpose – Steve McKee (OSD-MR)
1305-1309: Administrative Notes – Ray Langlais (LMI)
1309-1319: REPTX Distant Support– AR/VR Setting the Stage – Janice Bryant (NAVSEA)
1319-1329: Augmented Reality Remote Maintenance Support (ARRMS) and Inspection – Michael Confessore / George Lehaf (NAS Lakehurst) / / Jason Bickford (NSWC PHE)
1329-1339: Augmented Reality Maintainer Operator Relay System (ARMORS) – Tom Mead (Army Applications Lab)
1340-1350: Maintenance Augmented Reality System (MARS) – Caleb Tomkowiak (AFLCMC-RSO)
1350-1400: Remote Inspection and Product Verification – Andy Meighan (DLA)
1400-1455: Panel Style Discussion / Questions & Answers – ALL
1455-1500: Wrap-Up – Steve McKee (OSD-MR)
Minutes
Event: On 25 April 2023, the Joint Technology Exchange Group (JTEG) hosted a virtual forum on “Augmented Reality, Virtual Reality, and Mixed Reality in Support of DoD Maintenance and Sustainment”.
Purpose: The purpose of this forum was to conduct a panel discussion of the challenges and solutions, research and development, and implementation of Augmented Reality, Virtual Reality, and Mixed Reality in support of DoD sustainment.
Welcome: Steve McKee, ODASD (Materiel Readiness), served as the moderator and welcomed everyone to the forum and provided a brief introduction of the presenters and emphasized the significance, challenges, and potential impact that augmented reality, virtual reality, and mixed reality innovations could have within the DoD sustainment community.
Administrative: There were 102 people in attendance on the open forum. The presentations, along with questions and answers, were conducted through Zoom.Gov which also offered a chat capability for comments and Q&A. Each presenter provided a brief ten-minute overview of their organizations’ applicable technologies and then the moderator conducted a series of questions and answers from the panel members. Discussion amongst the panel members was lively and audience participation was consistently present, primarily through the chat box. Several information links and contact information were posted on the chat which was saved (Attached).
REPTX Distant Support– AR/VR Setting the Stage – Janice Bryant (NAVSEA) provided an overview of the Repair Technology Engagement Exercise (REPTX) Distant Support capability transition event held March 21-23 which focused on field-ready mixed reality and telepresence technologies for troubleshooting and providing immediate remote expertise at the point of need. She also provided quick looks at underway replenishment challenges, REPTX solutions benefits, device agnostic solutions, AR/VR practicality, no established Authority to Operate (ATO), and noise and telepresence.
Augmented Reality Maintenance Support (ARMS) – Michael Confessore / George Lehaf (NAS Lakehurst) / / Jason Bickford (NSWC PHE) described ARMS as an application framework that connects a SME (Windows PC) to a maintainer (AR Headset) allowing for two-way communications over approved DoD networks. ARMS has established a cross-SYSCOM team where each participant can contribute based on their funding levels, and use cases required. After a couple demonstrations between Lakehurst and Patuxent River, the next step is integration with CANES and ATO accreditation for seamless connectivity and quality of service on surface combatants, followed by fielding in support of PEO IWS. The Aviation Maintenance Advancement Solutions (AMAS) program will submit a POM Issue Sheet for FY26 for AR technology transition in support of NAVAIR aviation maintenance support.
Augmented Reality Maintainer Operator Relay System (ARMORS) – Tom Mead (Army Applications Lab) explained the Army’s goal to develop a software application to allow soldiers to easily identify, document, and manage vehicle maintenance issues that includes an AR-based guidance system for operators and mechanics, and develop an AR/MR PMCS assistance tool that will provide relevant maintenance information in a more user-friendly format. ARMORS is a TRL-6, hardware agnostic, and hands-free solution that utilizes digital twin for training. The realized return on investment is defined in improved task accuracy, an increase in efficiency, and reduced soldier maintenance task time.
Maintenance Augmented Reality System (MARS) – Caleb Tomkowiak (AFLCMC-RSO) described the MARS concept as providing potentially unlimited maintainer cross-utilization, building proficiency via AR immersion, and delivering greater flexibility for austere operations via AR/VR ‘tele-maintenance. MARS has the capability to amplify tech data with annotated photos, expert-guided videos, and AR holographic overlays, and achieve greater task accuracy. MARS is hardware agnostic and allows subject matter expert (SME) knowledge to be presented side-by-side with tech data, while also recording the content. The RSO is working with DoD and industry partners to link MARS with USAF maintenance systems.
Remote Inspection and Product Verification – Andy Meighan (DLA) discussed the DLA remote inspection capability to support technical quality and acceptance by providing on-demand testing at the point of need. Using commercially available technology, the benefits include faster speed to contract execution, lower overall costs, and decreased testing cycle times. Andy also noted that users must consider mutual agreements made among cybersecurity and IT engineers to ensure risks to the interoperable networks are mitigated IAW established protocols.
Q&A – The moderator asked the panelists a series of questions on the implementation of augmented reality, virtual reality, and mixed reality innovations within their DoD sustainment organizations, and inquired about information sharing amongst the Military Services and DLA community. A recording of the Q&A session can be found at:
Topic: JTEG Virtual Technology Forum
Date: Apr 25, 2023 12:58 PM Eastern Time
Share recording with viewers:
https://lmigov.zoomgov.com/rec/share/yipsghDhorCYJ7XEou7UtnSgDF6oUosw9gwwI4Lq0BowFqI06bzdBhEO6uCm36ti.geSnfvJQe2p-RrYW Password: 8#Tl4WLgfU
Closing Comments: Steve McKee thanked the panelists and participants for their attendance and all the work being done to support the development and use of augmented reality, virtual reality, and mixed reality innovations in support of DoD sustainment operations.
Action Items:
- Obtain copies of remaining cleared presentations once they are approved to post to a public website, and post to the JTEG website at https://jteg.ncms.org/.
Next JTEG Meeting: The next scheduled JTEG virtual forum is 23 May 2023, 1:00 – 3:00 pm EST. The topic is “Emerging Technologies”.
POC this action is Ray Langlais, rlanglais@lmi.org, (571) 633-8019
Chat
From Dr. Garett Scott Patria to Everyone 12:59 PM
Greetings from the U.S. Army Ground Vehicle Systems Center (Detroit Arsenal)!
From Steve McKee to Everyone 12:59 PM
Welcome!!! 🙂
From Federico.Sciammarella to Me (Direct Message) 01:15 PM
What is NMCI
From Corey Countryman to Everyone 01:19 PM
Comment on the ATO process for AR/VR – we’re looking at how we can move the development process to a DevSecOps / CICD type pipeline, which in theory brings something called a “continuous ATO” with configuration change cycle time in days, not months.
From Mike Confessore to Everyone 01:20 PM
@coreycountryman we’re doing the same on the NAVAIR side, currently getting started on a DevSecOps space
From Corey Countryman to Everyone 01:22 PM
Outstanding, Kelly! Mike – are you working with the PlatformOne/BlackPearl folks?
From Mike Confessore to Everyone 01:31 PM
The PMA is still making a determination on which environment they’d like to use, but yes, we are definitely looking at PlatformOne/BlackPearl
From Janice Bryant, SEA 05T to Everyone 01:32 PM
Tracking speedier answers, but the cost both initially and to maintain is significant
From DanRusin to Everyone 01:35 PM
Question. where are the digital twins? We have heard about these ‘digital twins’ but who paid for them, or are they still a wish-list item? Are the Digital Twins actually a thing now?
From Corey Countryman to Everyone 01:37 PM
Question for the ARMORS folks – Army’s big AR initiative was IVARS … which apparently had some serious issues with the “VR sickness” mechanisms. Are there lessons learned in terms of display design, or cuing mechanisms from that available?
From Thomas Mead – Army Applications Laboratory to Everyone 01:39 PM
@corey we are utilizing the MR headsets so not so much on the closed in aspects of VR so not a lot of the “sickness” but like the Navy we found that adoption is different based on the age, rank and experience
From Dr. Garett Scott Patria to Everyone 01:40 PM
…not to be confused with MARS (Military Auxiliary Radio System)…for all you HAMS out there 🙂
From DanRusin to Everyone 01:40 PM
Who on this call is from Army CASCOM or Ordnance school? interested to contact after.
From Thomas Mead – Army Applications Laboratory to Everyone 01:43 PM
@dan Rusin the digital twins are housed on the device within the Software. Industry has the ability to develop the twin but DEVCOM AC has the EMMA Project who have developed Difital twins too
@dan I can link you up with CASCOM or Ordnance school. we just briefed the Commandant of the Ordnance school on the ARMORS project
From Corey Countryman to Everyone 01:43 PM
@Tom and @Dan – also note, ONR has an ongoing SBIR for automated content (dimensional digital twin) creation, topic N221-069
From Shawn Harrison, DAU to Everyone 01:45 PM
Would be interested in speakers perspectives on whether the 3D models are derived from the Authoritative Source of Truth for the systems (e.g., from a Product Lifecycle Management system) and what model format(s) are supported. Thanks. Great initiatives!
From Thomas Mead – Army Applications Laboratory to Everyone 01:45 PM
@corey just to confirm ONR = Office of Naval Research?
From Corey Countryman to Everyone 01:46 PM
Office of Naval research, aye. Key “container” formats for AR/VR are both OMG/Kronos glTF, and PIXAR’s USD formats.
From Dr. Garett Scott Patria to Everyone 01:50 PM
@shawnharrison: Great question (i.e. me too)
From Kelly Malone – Taqtile to Everyone 01:53 PM
Taqtile’s Manifest software used on both ARMORS and MARS automatically converts 3D model files into USDZ and GLTF formats for rendering on device. Accepted model formats include: .gltf, .fbx, .glb, .obj, .dae, .3mf, .3ds, .stl, .glb, .usdz*
The platform also supports remote rendering via cloud systems
static and animated models are supported.
From Jason Bickford to Everyone 01:54 PM
@Shawn – regarding whether or not we need to derive from an authoritative source – it totally depends on the application (and risk). If you’re working on a system with variants that have significantly different components or maintenance, and/or you need CM to the right vessel then authoritative source is best. If the maintenance action is fairly general and resolution / accuracy is not required, then a custom model is fine.
(in my opinion…) 🙂
From Shawn Harrison, DAU to Everyone 01:56 PM
@Jason, Kelly, Corey – thanks!
From Jared Currie, Ctr, RSO to Everyone 01:57 PM
Two questions:
- How have people made a business case/return on investment decision for AR?
- Who are people using to develop their AR modules?
From Bill Nickerson to Everyone 01:59 PM
@Jason – Have you looked at how PEO MLB and N-PLM tools will work as the Authoritative Source of Truth for the models and overall configuration management?
From Mike Confessore to Everyone 01:59 PM
In the case of NAVAIR, we’ve relied on our supporting PMA’s to crunch the numbers in terms of ROI. This has often involved annual estimates on travel costs for SME’s, delays for the fleet, etc. On the Navy side, we’re doing all of our AR development in-house, occasionally with CSS support
From Jason Bickford to Everyone 02:00 PM
@Nickerson – yes and I can’t wait for it to be the gold standard! 🙂 PLM also makes the models more accessible which can significantly reduce cost.
From Bill Nickerson to Everyone 02:02 PM
@Jason – good. We are working with them to try to get the onboard system integrated with the N-MRO solution but will need to configuration manage the models that are used for the analytics on the platform in addition to the tech and configuration management. Let’s talk about this soon.
From Shawn Harrison, DAU to Everyone 02:03 PM
@BillNickerson – would also like to connect with you on N-PLM… at your convenience please contact shawn.harrison@dau.edu. Thanks.
From Jason Bickford to Everyone 02:03 PM
@Nickerson yeah, it’ll be a huge asset. Then we just need to provide the justifications to our PEOs to fund acquisition of that tech data.
From Bill Nickerson to Everyone 02:09 PM
@ShawnHarrison happy to follow-up. Please send me an e-mail introduction to george.w.nickerson4.civ@us.navy.mil so we can connect. I’ll include @Jason in our conversation as well. If someone else wants to join, the more the merrier!
@Jason – roger on PEO’s not buying tech data. That has to change. As we make the case for how it is useful, it will be easier. As they see what it costs them to buy it after the fact when it’s no longer a competitive environment, it will be an easier case.
From Federico.Sciammarella to Me (Direct Message) 02:09 PM
We hear a roadblock to scaling is that content development for work instructions takes time and can be expensive. Would there be interest in demonstrating how AR content generation software can analyze standard operating procedures and link references to external databases to develop AR deployment-ready content to minimize human intervention in the development process?
From Deniz Ferrin to Everyone 02:14 PM
On the topic of ROI: The commercial industry has performed some studies on ROI and can translate to the work being performed in the DoD. A couple resources I found very helpful are located at: https://www.datafrond.com/wp-content/uploads/2019/10/The-Total-Economic-Impact-of-PTC-Vuforia_2019.pdf AND at https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE3luwM
From Corey Countryman to Everyone 02:16 PM
@Deniz – I’m really glad you mentioned those; you’ve done some really good digging for those.
From Steve Privatt PSNSIMF to Everyone 02:17 PM
Are MARS and ARMORS being developed in a cloud environment or with on-prem servers? If cloud, is there an opportunity for cross-service adoption while still in the development stage?
From Caleb Tomkowiak to Everyone 02:18 PM
MARS is being developed on the cloud. I’ll defer to Tom on ARMORS, but I’d like to connect up with the ARMORS team to see what collaboration opportunities may exist.
From Kelly Malone – Taqtile to Everyone 02:21 PM
Taqtile’s Manifest used by both MARS and ARMORS can run in the cloud, at the edge and also operates in offline mode (with caveats such as certain features like remote assistance would clearly be unavailable offline).
the back-end is containerized to maximize flexibility in deployment options and approaches.
From Thomas Mead – Army Applications Laboratory to Everyone 02:21 PM
are the other Services working primarily with Engineering type folks to develop or work directly with end users?
From George Lehaf to Everyone 02:23 PM
Navy is working with NATEC reps, fleet users, and the SMEs throughout our remote assistance developments.
From Janice Bryant, SEA 05T to Everyone 02:23 PM
Thanks all, have to leave for another commitment
From Thomas Mead – Army Applications Laboratory to Everyone 02:27 PM
All I forgot to put my Email on my Slides. Please feel free to reach out Thomas.m.mead.civ@army.mil
From Dr. Garett Scott Patria to Everyone 02:39 PM
post content on milSuite and @mention folks and #tag the snot out of it, so it digitally threads in with related content and feeds the AI/search engine
From DanRusin to Everyone 02:39 PM
have to drop off. good discussion. D. Rusin, US Army DEVCOM C5ISR
From Dr. Garett Scott Patria to Everyone 02:40 PM
unfortunately, MS Teams is stovepiped by tenants (no Cross Service) 🙁
Example: https://www.milsuite.mil/book/groups/military-xr
From Caleb Tomkowiak to Everyone 02:42 PM
Ah, definitely open to another system for collaboration (Teams just as an example). It’s nice to have a place to ask others working similar efforts on similar use cases between engagements like these
From Corey Countryman to Everyone 02:43 PM
Maintenance training and performance support use cases are well addressed. Future use case I’m hoping we can look deeper into are 1) Virtual Design (Keyport has dipped its toes into this) where folks can evaluate Human Factors aspects before cutting metal. 2) Mission architecture/engineering – war games in a VR format. And “synesthetics” – the ability to map into the AR/VR headsets sensory information that humans normally can’t perceive (e.g., hearing RF fields or seeing IR).
From Federico.Sciammarella to Everyone 02:52 PM
www.mxdusa.org
Federico.sciammarella@mxdusa.org
We hear a roadblock to scaling is that content development for work instructions takes time and can be expensive. Would there be interest in demonstrating how AR content generation software can analyze standard operating procedures and link references to external databases to develop AR deployment-ready content to minimize human intervention in the development process?
concept project
From Jared Currie, Ctr, RSO to Everyone 02:54 PM
that’s a good (interesting) concept. the content development side of AR content concerns me because it impacts the business case
From Deniz Ferrin to Everyone 02:55 PM
Have a great day all!