litert-community/MiDaS-small

MiDaS small — LiteRT (fp16, NHWC, GPU-clean)

midas_small_256_fp16.tflite is MiDaS v2.1 small (MiDaS_small, the CNN MiDaS with an EfficientNet-Lite3 backbone — not the DPT/ViT variants) converted to LiteRT for on-device monocular depth estimation. Given one RGB image it predicts a per-pixel inverse-depth map (near = bright, far = dark).

It is the model used by the LiteRT compiled_model_api/depth_estimation Android sample.

Files

File	Precision	Size
midas_small_256_fp16.tflite	fp16 weights	~33 MB

Specs

Task	Monocular depth estimation
Source	torch.hub.load("intel-isl/MiDaS", "MiDaS_small")
Input	1 x 256 x 256 x 3 float32, RGB, ImageNet-normalized, NHWC (interleaved)
Output	1 x 256 x 256 float32, relative inverse depth

Pre-processing: resize to 256×256, normalize with ImageNet stats (mean = [0.485, 0.456, 0.406], std = [0.229, 0.224, 0.225] on [0,1] pixels), write as interleaved NHWC RGB float32.

Post-processing: min-max normalize the output and map through a color LUT (the sample uses inferno).

Why this conversion

The graph lowers entirely to GPU-clean builtins — no attention, no Flex/Custom ops, no GATHER_ND, no >4D reshapes:

CONV_2D x73, ADD x27, DEPTHWISE_CONV_2D x24, RELU x7, RESIZE_BILINEAR x5, RESHAPE x1

• Channel-last I/O (to_channel_last_io) so the model takes NHWC 1x256x256x3 directly, matching the interleaved RGB the app writes (no input transpose).
• fp16 via AI Edge Quantizer FLOAT_CASTING — half the size, runs natively on the GPU delegate. Dynamic-range int8 is intentionally avoided (it favors the CPU/XNNPACK path, not the GPU delegate).

Fidelity

• Converted fp32 vs. original PyTorch (real image): corr 1.0000, max|diff| ~1.6e-3.
• fp16 vs. fp32: corr 0.9999998 (≈0.27 % of the depth range).

On-device (Pixel 8a, verified)

The fp16 model compiles to 234 / 234 nodes on the LiteRT GPU delegate (LITERT_CL) — full GPU residency, no CPU fallback — at ~1–3 ms / inference (best 1.1 ms). RESIZE_BILINEAR align_corners=True is GPU-supported as-is; no model change needed.

Minimal usage

Android (Kotlin, CompiledModel GPU)

val model = CompiledModel.create(context.assets, "midas_small_256_fp16.tflite",
    CompiledModel.Options(Accelerator.GPU), null)
val inputs = model.createInputBuffers()
val outputs = model.createOutputBuffers()
inputs[0].writeFloat(nhwc)             // [1,256,256,3] ImageNet-normalized RGB, NHWC
model.run(inputs, outputs)
val depth = outputs[0].readFloat()     // [1,256,256] relative inverse depth

Python (desktop verification)

MEAN = np.array([0.485, 0.456, 0.406], np.float32)
STD  = np.array([0.229, 0.224, 0.225], np.float32)
import numpy as np
from PIL import Image
from ai_edge_litert.interpreter import Interpreter

img = Image.open("photo.jpg").convert("RGB").resize((256, 256))
x = ((np.asarray(img, np.float32) / 255 - MEAN) / STD)[None]      # [1,256,256,3] NHWC

it = Interpreter(model_path="midas_small_256_fp16.tflite"); it.allocate_tensors()
it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()
d = it.get_tensor(it.get_output_details()[0]["index"])[0]         # [256,256]
d = (d - d.min()) / (d.max() - d.min())                           # near = bright
Image.fromarray((d * 255).astype(np.uint8)).save("depth.png")

Training data & PII

This is a weights-exact format conversion of Intel ISL's MiDaS v2.1 small; no new training was performed. MiDaS was trained for monocular depth on a mix of ~10 public depth datasets (e.g. ReDWeb, DIML, MegaDepth, WSVD, 3D Movies). These contain photos of real scenes that may incidentally include people and other PII; none was deliberately collected and this conversion adds none. The model outputs a relative-depth map only and performs no identification. Apply your own content/PII filtering before deployment. See the original MiDaS repo for dataset details.

Performance

Measured on a Pixel 8a (Tensor G3, Android 16) with the standard TFLite benchmark_model tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.

Runtime	Backend	Graph on GPU	Latency
TFLite benchmark_model (TfLiteGpuDelegateV2)	GPU (OpenCL)	234 / 234	19.9 ms
TFLite benchmark_model	CPU (XNNPACK, 4 threads)	—	XNNPACK declined the graph

Any on-device figure recorded when this model shipped came from a different runtime. It was taken through LiteRT's own CompiledModel accelerator (logcat reports it as LITERT_CL), which is the path the Kotlin sample app and the LiteRT API use, and it appears elsewhere on this card. The rows above are the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. The two are not comparable, so read the rows above as a reproducible floor rather than as this model's speed on LiteRT.

XNNPACK declines these fp16 graphs — it reports failed to delegate DEPTHWISE_CONV_2D and then fails to allocate tensors — so there is no usable CPU number. Disabling XNNPACK falls back to reference kernels, which measured about 20× slower than the GPU on models of this size and would not represent CPU inference anyone would ship.

Snapdragon NPU (Hexagon)

The NPU is 3.65x faster than the GPU (1.64 ms against 5.97 ms) and loads 10.85x faster (117 ms against 1265 ms).

backend	compiled	inference (median / min)	load
NPU (Hexagon v81)	on-device JIT	1.64 ms / 1.60 ms	117 ms
GPU (Adreno)	—	5.97 ms / 5.83 ms	1265 ms

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.76, where 1.0 is the throttling threshold.

The NPU rows ran the published file unchanged. LiteRT compiled it for the Hexagon on the device at first load. That first compile took 2.2 s here. The load column above is the cached load every later run pays. Recipe and the runtime libraries it needs: NPU guide.

GPU wiring: GPU guide.

Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with the LiteRT benchmark_model tool from litert-cli-nightly 0.2.0.dev20260805: CPU inference (XNNPACK, 4 threads), 3 invocations per file of 10 warm-up plus 50 timed runs (the tool caps a phase at 150 s, so very slow graphs run fewer — the Runs column is the actual timed total). The latency is the median across invocations; the spread is the min–max over all timed runs. No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0).

File	Inference (median)	Spread (min–max)	Runs	Peak memory
midas_small_256_fp16.tflite	92.9 ms	92.6–95.9 ms	150	152 MB

License & attribution

• MiDaS weights: MIT (Intel ISL).
• EfficientNet-Lite3 backbone: Apache-2.0.

Original work: Ranftl et al., "Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer" (MiDaS), https://github.com/isl-org/MiDaS.

Reproducing the conversion

A self-contained converter (litert-torch + ai-edge-quantizer) lives in the sample under compiled_model_api/depth_estimation/conversion/:

pip install litert-torch ai-edge-quantizer torch timm matplotlib pillow
python convert_midas_litert.py out 256