modular__skills/import-model/references/pitfalls.md

Universal pitfalls (index)

Find your symptom below, then load the one category file it points to.

Config and registration → pitfalls-config.md

• arch.py::name must match architectures[0] exactly
• default_encoding should match the Hub checkpoint dtype
• Memory estimator reads HF torch_dtype, not the adapter's post-cast dtype
• gelu_new ≠ gelu ≠ gelu_tanh
• Import and config API traps

Weight adapter → pitfalls-weights.md

• Tied embeddings need shared tensor objects
• stacked_qkv=False keeps HF's unfused Q/K/V key names
• lm_head.weight is the canonical state-dict key — never rename to output.weight
• Don't trust strict=False weight loads
• numpy.from_dlpack does not support bfloat16
• Embedding row-count may exceed vocab_size

Graph build → pitfalls-graph.md

• Scaffold is not a port
• ops.constant requires device=
• Partial-rotary padding is interleaved
• ops.sum keeps the reduced dim
• Stack-vs-block residual in recurrent / shared-weight architectures
• Subgraph cache assumes uniform layer shape

Serve and verify → pitfalls-serving.md

• Test decode at 16+ tokens, not just 1
• trust_remote_code=True with .to("cuda") can produce NaN
• Text comparison is best with greedy
• ARCHITECTURES = [arch] export is mandatory for --custom-architectures
• Fake HF oracle (broken reference before you compare MAX)