Instructions to use nxp/deep-autoencoder-imx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use nxp/deep-autoencoder-imx with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Anomaly Detection - Deep AutoEncoder
Overview
Anomalous sound detection (ASD) is the task of identifying whether the sound emitted from a target machine is normal or anomalous. This deep autoencoder model is part of the MLPerf Tiny benchmark suite [1]. It is trained in an unsupervised way, meaning that only normal sound samples are used as training data; at inference time the reconstruction error between the input feature vector and the autoencoder's reconstruction of it is used as an anomaly score. The model takes a 640-dimensional log-mel spectrogram feature vector as input and outputs a reconstruction of that same vector.
This repository contains the pre-quantized int8 TFLite model published by
MLCommons, downloaded as-is and
compiled for NXP i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors. It also contains the script used to
download and compile these assets (recipe.sh) and an inference example (example.py).
Key Features
- Fully-connected deep autoencoder for unsupervised anomalous sound detection.
- Operates on 640-dimensional log-mel spectrogram feature vectors extracted from audio (128 mel bands x 5 frames).
- ~267,928 parameters, ~0.528 MOPS.
- int8 quantized, int8 input / int8 output, for accelerated NPU inference on NXP i.MX platforms.
Model Description
This repository contains the MLPerf Tiny deep autoencoder model, quantized to int8 by MLCommons and compiled by NXP for i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors.
- Base Model: MLPerf Tiny anomaly detection
- Original Model Authors: MLCommons
- Original License: Apache-2.0
- Modified by: NXP
Modifications
This model is a derivative work with the following changes from the original:
- Quantization: None performed by NXP. The int8 TFLite model is published directly by MLCommons and downloaded
as-is by
recipe.sh; no local re-quantization or calibration dataset is used. - Compilation (i.MX 93): Compiled with Vela for the Ethos-U65 NPU,
tags
lf-6.18.20_2.0.0andlf-6.18.37_2.1.0. - Compilation (i.MX 95 / i.MX 952): Converted with the eIQ Neutron SDK, versions
3.1.3,3.2.1, and3.2.3. - i.MX 8M Plus: The hardware-independent int8 model is used directly (no Vela/Neutron compilation needed for the
VeriSilicon NPU delegate);
assets/imx8mplus/ad01_int8.tfliteis identical toassets/quantized_tflite_model/ad01_int8.tflite.
Original model available at: https://github.com/mlcommons/tiny/tree/master/benchmark/training/anomaly_detection.
Model Information
| Information | Value |
|---|---|
| Input shape | Feature vector (1, 640), int8 (log-mel spectrogram, 128 mel bands x 5 frames) |
| Input example | example_input.wav (sound file from the ToyADMOS dataset [2]) |
| Output shape | Reconstructed feature vector (1, 640), int8 quantized; dequantize with the output tensor's scale / zero point, as done in example.py |
| Output example | example.py prints the anomaly score (mean squared reconstruction error, a float32 value computed from the dequantized output) for example_input.wav |
| FLOPS | 0.528 MOPS |
| Number of parameters | 267,928 |
| File size (int8) | 271 KB (assets/quantized_tflite_model/ad01_int8.tflite) |
Supported Hardware and Software Configurations
All assets run with the eIQ LiteRT (TensorFlow Lite) runtime and the corresponding NPU delegate (VX for i.MX 8M Plus, Ethos-U for i.MX 93, Neutron for i.MX 95 / i.MX 952). The same information is captured in release_manifest.json.
| Asset | Runtime | Precision | HW target | BSP & SDK versions | Link to asset |
|---|---|---|---|---|---|
| i.MX 8M Plus Quantized | LiteRT | INT8 | i.MX 8M Plus | - | Download |
| i.MX 93 Precompiled | LiteRT | INT8 | i.MX 93 | BSP lf-6.18.20_2.0.0, VELA_TAG = lf-6.18.20_2.0.0 | Download |
| i.MX 93 Precompiled | LiteRT | INT8 | i.MX 93 | BSP lf-6.18.37_2.1.0, VELA_TAG = lf-6.18.37_2.1.0 | Download |
| i.MX 95 Precompiled | LiteRT | INT8 | i.MX 95 | BSP lf-6.18.20_2.0.0, Neutron SDK v3.1.3 | Download |
| i.MX 95 Precompiled | LiteRT | INT8 | i.MX 95 | BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.1 | Download |
| i.MX 95 Precompiled | LiteRT | INT8 | i.MX 95 | BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.3 | Download |
| i.MX 952 Precompiled | LiteRT | INT8 | i.MX 952 | BSP lf-6.18.20_2.0.0, Neutron SDK v3.1.3 | Download |
| i.MX 952 Precompiled | LiteRT | INT8 | i.MX 952 | BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.1 | Download |
| i.MX 952 Precompiled | LiteRT | INT8 | i.MX 952 | BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.3 | Download |
The hardware-independent int8 model is also provided for host (x86) inference as
assets/quantized_tflite_model/ad01_int8.tflite (identical to the i.MX 8M Plus asset above).
Limitations and Biases
This model inherits all limitations from the original MLPerf Tiny deep autoencoder model. Additional limitations:
- Task scope: This model only produces an anomaly score for a fixed 640-dimensional log-mel feature vector; it does not classify the type of anomaly, and it has no built-in decision threshold (the threshold must be chosen by the integrator based on the target machine and acceptable false-positive rate).
- Input format: The model expects a fixed-size (1, 640) feature vector derived from a specific log-mel
spectrogram feature extraction pipeline (128 mel bands x 5 frames, see
example.py); different feature extraction parameters will produce invalid results. - Dataset bias: The model is trained on the ToyADMOS dataset, which targets specific miniature-machine sound sources. Detection quality may not generalize to machines, acoustic environments, or noise conditions not represented in that dataset.
Training
The model was trained by MLCommons / the original MLPerf Tiny benchmark authors on the ToyADMOS dataset [2], in an unsupervised manner using only normal sound samples. NXP did not retrain or fine-tune the model; the int8 TFLite weights are downloaded as-is from MLCommons. Training scripts are available on the MLCommons Tiny GitHub page.
Download and Run
Running the example
Install the dependencies from requirements.txt first. An example of how to use the model is in example.py (run
on a PC):
# Run on PC:
python example.py -m assets/quantized_tflite_model/ad01_int8.tflite -i example_input.wav
example.py accepts the following arguments (all optional, shown with their defaults):
-m,--model(default:assets/quantized_tflite_model/ad01_int8.tflite): path to the TFLite model file.-i,--input(default:example_input.wav): path to the input.wavfile.-o,--output(default: none): optional path to write the anomaly score as text.
The example loads a .wav file, extracts log-mel spectrogram feature vectors, runs the autoencoder, and reports the
mean squared reconstruction error as the anomaly score.
Please refer to the i.MX Machine Learning User Guide for details on how to run the models on each i.MX platform.
Regenerating the assets with recipe.sh
The recipe installs the eIQ Neutron SDK from the NXP eIQ PyPI Distribution Portal. Configure pip once to use it:
python -m pip config --user set global.index https://eiq.nxp.com/repository
python -m pip config --user set global.index-url https://eiq.nxp.com/repository
python -m pip config --user set global.trusted-host eiq.nxp.com
python -m pip config --user set global.extra-index-url https://pypi.org/simple
python -m pip config --user set global.find-links https://eiq.nxp.com/repository/
To download the quantized int8 model and compile the platform-specific variants, run:
bash recipe.sh
The Vela tag and eIQ Neutron SDK version can be passed as optional positional arguments (defaulting to
lf-6.18.20_2.0.0 and 3.2.3 respectively):
bash recipe.sh [IMX_ETHOS_U_VELA_TAG] [EIQ_NEUTRON_SDK_VERSION]
# example:
bash recipe.sh lf-6.18.37_2.1.0 3.2.3
recipe.sh generates:
assets/quantized_tflite_model/ad01_int8.tflite- the hardware-independent int8 model, downloaded from MLCommons.assets/imx93/<IMX_ETHOS_U_VELA_TAG>/ad01_int8_vela.tflite- Vela-compiled for i.MX 93.assets/imx95/<EIQ_NEUTRON_SDK_VERSION>/ad01_int8_converted.tflite- Neutron-converted for i.MX 95.assets/imx952/<EIQ_NEUTRON_SDK_VERSION>/ad01_int8_converted.tflite- Neutron-converted for i.MX 952.
assets/imx8mplus/ad01_int8.tflite is the same file as assets/quantized_tflite_model/ad01_int8.tflite, provided
under the i.MX 8M Plus directory for convenience.
License
This model is released under the Apache License 2.0 (see LICENSE), the same license as the original MLCommons MLPerf Tiny deep autoencoder model.
Citation
If you use this model, please cite both this work and the original model:
- C. Banbury et al., "MLPerf Tiny Benchmark," arXiv preprint arXiv:2106.07597, 2021.
- Y. Koizumi et al., "ToyADMOS: A dataset of miniature-machine operating sounds for anomalous sound detection," 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), IEEE, 2019.
@article{banbury2021mlperf,
title={Mlperf tiny benchmark},
author={Banbury, Colby and Reddi, Vijay Janapa and Torelli, Peter and Holleman, Jeremy and Jeffries, Nat and Kiraly, Csaba and Montino, Pietro and Kanter, David and Ahmed, Sebastian and Pau, Danilo and others},
journal={arXiv preprint arXiv:2106.07597},
year={2021}
}
@inproceedings{koizumi2019toyadmos,
title={ToyADMOS: A dataset of miniature-machine operating sounds for anomalous sound detection},
author={Koizumi, Yuma and Saito, Shoichiro and Uematsu, Hisashi and Harada, Noboru and Imoto, Keisuke},
booktitle={2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)},
pages={313--317},
year={2019},
organization={IEEE}
}
- Downloads last month
- 81