Can you tell, before answering,
whether a small model is enough?
Sending every prompt to the largest model wastes money and energy. A gate that predicts, from the prompt alone, when a small model is enough would cut both cost and CO2. Many tools claim to do it. Few publish how well it actually works.
I measured it on 458 prompts, each answered by a small and a large model and graded by LLM judges. Structural features (length, shape, keywords) scored 0.50: pure chance. A gate built on embeddings reached 0.59, a weak but real signal. Then the ceiling appeared: the judges disagreed with each other on 33% of prompts, so the labels themselves were noisy. The conclusion reshaped the project: gating alone is not the lever, routing is. A negative result, kept in plain sight because it saved months.