wuklab-NUC16Intel NPU 5 (intel_vpu)OpenVINO 2026.0 int8

Where does Intel NPU compile time go?

Compiling a model for the Intel NPU under OpenVINO can take minutes. This shows exactly where that time goes — by reading the NPU compiler's own per-pass timing report and presenting its real nested MLIR TimingScope tree as an interactive, expandable hierarchy. No stage labels are imposed: every node below is a pipeline group or pass the compiler itself printed, with its own timing.

Measured on a fanless Intel Core Ultra 7 356H mini-PC (NPU 5) on the real int8 OpenVINO IR, by parsing the compiler's IE_NPU_LOG_FILTER=vpux-compiler report — so it profiles the exact compiler you run, no source build required.

Key findings

• NPU compilation is expensive and scales with model size. A cold compile runs 15–28 s for Qwen2.5-1.5B, 82–242 s for Qwen2.5-7B (int8). One NPUW_LLM compile is actually 7 sub-model compiles.
• Almost all of it is one-time codegen the blob cache removes. Warm reuse of a cached blob is ~0.7–1.9 s — up to 125× faster than cold.
• The compiler's own timing tree splits a compile into two groups: IE to VPUIP pipeline (the optimizer/scheduler, ~81%) and ELF pipeline (the backend/serializer, ~19%). Expand the tree below to walk the real hierarchy down to individual passes.
• One pass dominates the large-model compile: FeasibleAllocation — the memory allocator/scheduler — sums to 73 s across Qwen2.5-7B's sub-models (~31% of the whole compile). It sits inside IE to VPUIP pipeline.
• The bottleneck tracks size, not modality. The same tree for a vision transformer (ViT-large) has the identical shape, and its heaviest pass is also FeasibleAllocation — a ViT and an LLM of similar compile cost stress the same passes.
• No source build needed. This is the compiler's own report, printed at INFO level via IE_NPU_LOG_FILTER=vpux-compiler — it profiles the exact compiler you run.

1 · Is compilation actually costly?

Cold compile (no cache) versus the warm blob-reuse floor — the whole cost is one-time codegen, so reusing a cached blob is up to 125× faster.

model	window	cold s	warm-reuse s	cache ×	peak MB
qwen25_1_5b	128	14.8462	0.7192	20.6	775.5
qwen25_1_5b	256	16.7754	0.7176	23.4	781.7
qwen25_1_5b	512	17.9899	0.7135	25.2	775.5
qwen25_1_5b	1024	23.8757	0.719	33.2	770.6
qwen25_1_5b	2048	28.2005	0.7405	38.1	801.7
qwen25_7b	256	81.977	1.7571	46.7	1107.8
qwen25_7b	1024	226.2104	1.9059	118.7	880.1
qwen25_7b	2048	241.8275	1.9334	125.1	884.3

Compile time grows ~window^0.24 for the 1.5B but ~window^0.55 for the 7B — bigger models pay a steeper penalty for a longer static prompt window.

2 · The compiler's own timing tree

Below is the actual MLIR TimingScope tree for a cold Qwen2.5-7B compile — the same report the compiler prints, rendered as an expandable hierarchy. A single NPUW_LLM compile is several sub-model compiles (prefill / decode / KV variants); each opens to Total → Compile network → {IE to VPUIP pipeline, ELF pipeline} and then down to individual passes. Click any row to expand it; the bar shows its share of that sub-model's compile.

Heaviest passes (summed across all sub-models)

The per-pass time cost, at a glance — the same passes you can drill to in the tree below.

3 · Same tree, a vision model — size, not modality

The identical report for a vision transformer (ViT-large) has the same shape and the same heaviest pass — so the bottleneck is set by model scale, not whether it is vision or language.

Method & reproducibility

The installed compiler prints this per-pass tree at LOG_INFO when OV_NPU_LOG_FILTER=vpux-compiler is set before import openvino; the harness captures the native stderr (fd-level) and reconstructs the tree from the report's indentation — the nodes and numbers are the compiler's, unmodified (passes below 0.05 ms are elided for size). Cold = NPU_BYPASS_UMD_CACHING=YES; warm = an OpenVINO CACHE_DIR. Env /data/envs/openvino-3.12 (OpenVINO 2026.0.0), NPU driver-compiler 1.33. The tree structure comes from the compiler's pass pipeline (src/vpux_compiler/src/NPU50XX/dialect_pipeline_strategy.cpp in openvinotoolkit/npu_compiler). Code: experiments/npu_compile/ in the edge-ai repo (stages.py::parse_timing_tree · make_page.py). Run folders are write-once.