March 9, 2022 – By Intuidex
Intuidex, Inc. partnered with Serco North America (Serco NA) to develop a use case for predictive analytics in low resource environments leveraging Intuidex’s Higher-Order Low-Resource Learning™ (HO-LRL™) to improve performance over traditional predictive models.
This use case will be presented at the Sixth Annual Workshop on Naval Applications of Machine Learning (NAML), March 22-24, 2022.
Poster Schedule Session 1A, Wednesday March 23, 2022: Rapid Exploitation of Human Language in a Low-Resource Environment by Alex Rojas, Serco North America Inc. See the full schedule here.
Presentation Into Video – courtesy Serco North America
Data quality, consistency, and integrity remain a common challenge across artificial intelligence (AI) use cases, including in natural language processing (NLP). Most AI algorithms and machine learning methods rely on a large number of training data examples in order to provide higher accuracy predictions.
To better address predictive analytics in low resource environments, Intuidex and Serco NA tested the use of Intuidex’s proprietary HO-LRL™ technology to predict schedule overruns in the delivery of military assets utilizing condition and maintenance planning data. Currently, teams of personnel within the military are required to produce Availability Duration Scorecard (ADS) predictions.
Previous modeling of predictive analytics performed by Serco NA using a traditional machine learning algorithm produced a baseline solution which provided better results than the human expert-driven ADS approach (14% versus 12.45% mean absolute percent error) but there was room for improvement.
In this test, NLP features were incorporated into a Support Vector Machine (SVM) model enhanced to use HO-LRL™. The HO-LRL™ technique transforms the data to focus on important discriminators for target conditions.
Findings
Metric | HO-LRL™ | Original Implementation |
Fβ=0.25 / Accuracy | >99% | 86% |
Error / MAPE (Mean Absolute | <1% | 14% |
Training Time | <20 seconds (avg. depends on | 1:300 hrs. ratio |
Training Data | 6,500 records (0.2% of original | 3,158,219 records |
Time to Implement | 2 weeks | Few months when data available |
Conclusions
● Using HO-LRL™ yielded > 99% Fβ=0.25, a 13% increase from the original implementation!
● Training time was greatly reduced to <20 seconds (avg. depends on data set size and resources)!
● The amount of data used was only 0.2 % (6500 records) of the original data (3,158,219 records). The smallest amount of data modeled was only 180 samples, 30 with the overrun condition and 150 with non-overrun. HO-LRL™-enhanced SVM beat standard SVM by several points with >99% confidence!
● Take away: What is remarkable is that this represents a >99.99999% reduction in training data and subsequent training time but resulted in significantly increased accuracy!
HO-LRL™ is part of Intuidex’s Watchman Analytics Suite from which several products, including Watchman for Defense™ (W4D) are derived. W4D provides multi-INT fusion and alerting, including pattern of life and anomaly detection. HO-LRL™ is a data transformation for machine learning in low-resource settings and supports both generative and discriminative learning, including natural language processing (NLP) with deep learning networks and latent embeddings.
HO-LRL™ can be used in a wide array of use cases and applications and provides numerous benefits, such as a much lower threshold of training data, significantly less training time, and greatly improved predictive performance in low-resource settings including real-time streaming data scenarios.
Read the full presentation here.
