Build a Reusable Prompt Evaluation Checklist
Goal
Create a reusable prompt scorecard so versions can be compared under the same conditions.
Prerequisites
Prepare one real task, three representative inputs, and a table for recording results.
Steps
- Define the output purpose, audience, required content, and prohibited content.
- Fix three inputs covering a normal case, an edge case, and an underspecified case.
- Score accuracy, completeness, format compliance, and actionability from 1 to 5.
- Run prompt A with one tool and fixed settings, then run prompt B after changing only one element.
- Save raw outputs, scores, failure reasons, and the single change planned for the next iteration.
Expected result
You have a traceable A/B record that shows which evaluation dimension changed.
Troubleshooting
- Results vary too much: run each version three times.
- Scores feel subjective: add one passing and one failing example per criterion.
- Several variables changed: revert and change one item at a time.