Short findings from model selection, retrieval behavior, tool calling, evaluation, and failure-case testing.
Why this note exists.
R&D is useful when it reduces delivery uncertainty. Experiments should help the team decide what is reliable enough to use and what must stay behind a human review boundary.
Observations
- Model choice matters less than context design in many workflow cases.
- Tool calling needs tests around failure and ambiguous intent.
- Evaluation cases should reflect the buyer workflow, not generic benchmarks.
What we learned
- Keep R&D tied to a delivery question.
- Write down failure cases and operational assumptions.
- Convert useful findings into reusable service patterns.
Useful artifact
- Workflow map or decision note.
- Test case, approval log, or delivery handoff.
- Reusable pattern for a future service path.