Anomaly Detection - Deep AutoEncoder

Overview

Anomalous sound detection (ASD) is the task of identifying whether the sound emitted from a target machine is normal or anomalous. This deep autoencoder model is part of the MLPerf Tiny benchmark suite [1]. It is trained in an unsupervised way, meaning that only normal sound samples are used as training data; at inference time the reconstruction error between the input feature vector and the autoencoder's reconstruction of it is used as an anomaly score. The model takes a 640-dimensional log-mel spectrogram feature vector as input and outputs a reconstruction of that same vector.

This repository contains the pre-quantized int8 TFLite model published by MLCommons, downloaded as-is and compiled for NXP i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors. It also contains the script used to download and compile these assets (recipe.sh) and an inference example (example.py).

Key Features

  • Fully-connected deep autoencoder for unsupervised anomalous sound detection.
  • Operates on 640-dimensional log-mel spectrogram feature vectors extracted from audio (128 mel bands x 5 frames).
  • ~267,928 parameters, ~0.528 MOPS.
  • int8 quantized, int8 input / int8 output, for accelerated NPU inference on NXP i.MX platforms.

Model Description

This repository contains the MLPerf Tiny deep autoencoder model, quantized to int8 by MLCommons and compiled by NXP for i.MX 8M Plus, i.MX 93, i.MX 95, and i.MX 952 processors.

Modifications

This model is a derivative work with the following changes from the original:

  • Quantization: None performed by NXP. The int8 TFLite model is published directly by MLCommons and downloaded as-is by recipe.sh; no local re-quantization or calibration dataset is used.
  • Compilation (i.MX 93): Compiled with Vela for the Ethos-U65 NPU, tags lf-6.18.20_2.0.0 and lf-6.18.37_2.1.0.
  • Compilation (i.MX 95 / i.MX 952): Converted with the eIQ Neutron SDK, versions 3.1.3, 3.2.1, and 3.2.3.
  • i.MX 8M Plus: The hardware-independent int8 model is used directly (no Vela/Neutron compilation needed for the VeriSilicon NPU delegate); assets/imx8mplus/ad01_int8.tflite is identical to assets/quantized_tflite_model/ad01_int8.tflite.

Original model available at: https://github.com/mlcommons/tiny/tree/master/benchmark/training/anomaly_detection.

Model Information

Information Value
Input shape Feature vector (1, 640), int8 (log-mel spectrogram, 128 mel bands x 5 frames)
Input example example_input.wav (sound file from the ToyADMOS dataset [2])
Output shape Reconstructed feature vector (1, 640), int8 quantized; dequantize with the output tensor's scale / zero point, as done in example.py
Output example example.py prints the anomaly score (mean squared reconstruction error, a float32 value computed from the dequantized output) for example_input.wav
FLOPS 0.528 MOPS
Number of parameters 267,928
File size (int8) 271 KB (assets/quantized_tflite_model/ad01_int8.tflite)

Supported Hardware and Software Configurations

All assets run with the eIQ LiteRT (TensorFlow Lite) runtime and the corresponding NPU delegate (VX for i.MX 8M Plus, Ethos-U for i.MX 93, Neutron for i.MX 95 / i.MX 952). The same information is captured in release_manifest.json.

Asset Runtime Precision HW target BSP & SDK versions Link to asset
i.MX 8M Plus Quantized LiteRT INT8 i.MX 8M Plus - Download
i.MX 93 Precompiled LiteRT INT8 i.MX 93 BSP lf-6.18.20_2.0.0, VELA_TAG = lf-6.18.20_2.0.0 Download
i.MX 93 Precompiled LiteRT INT8 i.MX 93 BSP lf-6.18.37_2.1.0, VELA_TAG = lf-6.18.37_2.1.0 Download
i.MX 95 Precompiled LiteRT INT8 i.MX 95 BSP lf-6.18.20_2.0.0, Neutron SDK v3.1.3 Download
i.MX 95 Precompiled LiteRT INT8 i.MX 95 BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.1 Download
i.MX 95 Precompiled LiteRT INT8 i.MX 95 BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.3 Download
i.MX 952 Precompiled LiteRT INT8 i.MX 952 BSP lf-6.18.20_2.0.0, Neutron SDK v3.1.3 Download
i.MX 952 Precompiled LiteRT INT8 i.MX 952 BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.1 Download
i.MX 952 Precompiled LiteRT INT8 i.MX 952 BSP lf-6.18.37_2.1.0, Neutron SDK v3.2.3 Download

The hardware-independent int8 model is also provided for host (x86) inference as assets/quantized_tflite_model/ad01_int8.tflite (identical to the i.MX 8M Plus asset above).

Limitations and Biases

This model inherits all limitations from the original MLPerf Tiny deep autoencoder model. Additional limitations:

  • Task scope: This model only produces an anomaly score for a fixed 640-dimensional log-mel feature vector; it does not classify the type of anomaly, and it has no built-in decision threshold (the threshold must be chosen by the integrator based on the target machine and acceptable false-positive rate).
  • Input format: The model expects a fixed-size (1, 640) feature vector derived from a specific log-mel spectrogram feature extraction pipeline (128 mel bands x 5 frames, see example.py); different feature extraction parameters will produce invalid results.
  • Dataset bias: The model is trained on the ToyADMOS dataset, which targets specific miniature-machine sound sources. Detection quality may not generalize to machines, acoustic environments, or noise conditions not represented in that dataset.

Training

The model was trained by MLCommons / the original MLPerf Tiny benchmark authors on the ToyADMOS dataset [2], in an unsupervised manner using only normal sound samples. NXP did not retrain or fine-tune the model; the int8 TFLite weights are downloaded as-is from MLCommons. Training scripts are available on the MLCommons Tiny GitHub page.

Download and Run

Running the example

Install the dependencies from requirements.txt first. An example of how to use the model is in example.py (run on a PC):

# Run on PC:
python example.py -m assets/quantized_tflite_model/ad01_int8.tflite -i example_input.wav

example.py accepts the following arguments (all optional, shown with their defaults):

  • -m, --model (default: assets/quantized_tflite_model/ad01_int8.tflite): path to the TFLite model file.
  • -i, --input (default: example_input.wav): path to the input .wav file.
  • -o, --output (default: none): optional path to write the anomaly score as text.

The example loads a .wav file, extracts log-mel spectrogram feature vectors, runs the autoencoder, and reports the mean squared reconstruction error as the anomaly score.

Please refer to the i.MX Machine Learning User Guide for details on how to run the models on each i.MX platform.

Regenerating the assets with recipe.sh

The recipe installs the eIQ Neutron SDK from the NXP eIQ PyPI Distribution Portal. Configure pip once to use it:

python -m pip config --user set global.index https://eiq.nxp.com/repository
python -m pip config --user set global.index-url https://eiq.nxp.com/repository
python -m pip config --user set global.trusted-host eiq.nxp.com
python -m pip config --user set global.extra-index-url https://pypi.org/simple
python -m pip config --user set global.find-links https://eiq.nxp.com/repository/

To download the quantized int8 model and compile the platform-specific variants, run:

bash recipe.sh

The Vela tag and eIQ Neutron SDK version can be passed as optional positional arguments (defaulting to lf-6.18.20_2.0.0 and 3.2.3 respectively):

bash recipe.sh [IMX_ETHOS_U_VELA_TAG] [EIQ_NEUTRON_SDK_VERSION]

# example:
bash recipe.sh lf-6.18.37_2.1.0 3.2.3

recipe.sh generates:

  • assets/quantized_tflite_model/ad01_int8.tflite - the hardware-independent int8 model, downloaded from MLCommons.
  • assets/imx93/<IMX_ETHOS_U_VELA_TAG>/ad01_int8_vela.tflite - Vela-compiled for i.MX 93.
  • assets/imx95/<EIQ_NEUTRON_SDK_VERSION>/ad01_int8_converted.tflite - Neutron-converted for i.MX 95.
  • assets/imx952/<EIQ_NEUTRON_SDK_VERSION>/ad01_int8_converted.tflite - Neutron-converted for i.MX 952.

assets/imx8mplus/ad01_int8.tflite is the same file as assets/quantized_tflite_model/ad01_int8.tflite, provided under the i.MX 8M Plus directory for convenience.

License

This model is released under the Apache License 2.0 (see LICENSE), the same license as the original MLCommons MLPerf Tiny deep autoencoder model.

Citation

If you use this model, please cite both this work and the original model:

  • C. Banbury et al., "MLPerf Tiny Benchmark," arXiv preprint arXiv:2106.07597, 2021.
  • Y. Koizumi et al., "ToyADMOS: A dataset of miniature-machine operating sounds for anomalous sound detection," 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), IEEE, 2019.
@article{banbury2021mlperf,
  title={Mlperf tiny benchmark},
  author={Banbury, Colby and Reddi, Vijay Janapa and Torelli, Peter and Holleman, Jeremy and Jeffries, Nat and Kiraly, Csaba and Montino, Pietro and Kanter, David and Ahmed, Sebastian and Pau, Danilo and others},
  journal={arXiv preprint arXiv:2106.07597},
  year={2021}
}

@inproceedings{koizumi2019toyadmos,
  title={ToyADMOS: A dataset of miniature-machine operating sounds for anomalous sound detection},
  author={Koizumi, Yuma and Saito, Shoichiro and Uematsu, Hisashi and Harada, Noboru and Imoto, Keisuke},
  booktitle={2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)},
  pages={313--317},
  year={2019},
  organization={IEEE}
}
Downloads last month
81
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collections including nxp/deep-autoencoder-imx

Paper for nxp/deep-autoencoder-imx