Define the result before choosing the model
A document assistant, a support triage workflow, and a product research tool do different jobs. Before comparing models, write down what a useful result must contain. A document summary might need every deadline and its supporting passage. A triage result might need the correct team and a clear reason for escalation. Ask the process owner to define the cases that should pass, fail, or reach a person.
Routing has become a configurable capability
Microsoft Foundry’s model router selects an eligible model for each request. Its documented modes are Balanced, Cost, and Quality, and a model subset lets a team restrict the routing pool. Context requirements still matter: Microsoft identifies the smallest underlying model as the effective context limit and recommends choosing a suitable subset for larger inputs. Availability also depends on access and deployment configuration. Those controls create options to test; they do not establish savings for your particular workflow. Microsoft model router documentation.
Put a quality gate around the choice
Microsoft’s evaluation documentation describes measures including groundedness, tool-call accuracy, and task completion, alongside custom evaluators. Our suggested pilot is to compare a fixed model and a routed configuration on the same representative, permissioned test set. Include incomplete documents, ambiguous requests, and examples that should be escalated. Have process owners review important decisions. Set acceptance thresholds before running the comparison, and investigate failures individually rather than letting an average score hide them. Microsoft evaluation documentation.
Measure the complete workflow
Use observed billing and execution records to compare the cost per accepted result. Include retries, retrieval, evaluation, fallback calls, and the staff time needed to correct outputs. Track response time and escalation frequency separately. A lower token bill can be offset by extra review or repeated calls. Report the conditions of the trial, the sample size, and the uncertainty; avoid projecting a small pilot’s savings across the whole business.
Start with one task and keep the choice reviewable
Begin with a narrow workflow and a clear owner. Record the routing mode, permitted models, prompts, and evaluation criteria so another team member can explain the decision. Microsoft notes that its active router version can gain underlying models without a new version identifier; record the relevant configuration and repeat the comparison when the model pool changes. Keep a tested route back to the previous configuration. Expand only when the evidence supports useful results at an acceptable operating cost. Microsoft model router documentation.