The ROUGE metric is poorly suited for this challenge because it focuses on surface-level n-gram overlap rather than capturing the correctness, clinical safety, or reasoning quality of the responses.

• For competitors, ROUGE encourages optimizing for word or phrase similarity rather than genuinely replicating the structured clinical reasoning of a trained professional. That pushes participants to “game” phrasing rather than solve the real challenge of safe, explainable, and context-aware medical decision-making.
• For the host and end users, ROUGE-based submissions risk being unusable in practice, since a clinician-like summary with correct-sounding words might still miss vital reasoning steps or even contradict best practice. In a safety-critical domain like healthcare, this could cause serious harm if the metric cannot distinguish between a clinically sound response and a superficially similar but incorrect one.

A better evaluation metric — for example, one based on expert manual scoring or a more robust semantic similarity measure aligned with clinical best practice — would both encourage more meaningful solutions and deliver higher real-world value to PATH and its partners.