During retrospectives, agile teams would benefit from following the Lean Startup concept of testing their improvement hypotheses using scientific methods. My experience during agile retrospectives was that for any 'improvements' agreed upon by the team, there was no deliberate validation that the change actually makes the hoped for improvement. The trick is to select quantifiable measures that can be directly linked to the 'improvement' i.e., cause and effect are firmly linked. In essence the Scrum team will conduct an experiment to determine if for some action/activity/change the team will get a measurable improvement.
It's important that the team only do one improvement at a time so that any effects can be attributed to the improvement. For example, if you're introducing pair-programing to reduce acceptance tests failures but you've also introduced a new test harness to help increase test coverage, you might find it difficult or impossible to determine which of these two affected acceptance test results and by how much.
The retrospective process steps would be:
1. Clearly state the exact problem the improvement will solve
2. Clearly state what the results will be with the improvement in place
3. Develop the falsifiable* hypothesis for the improvement (this is the experiment summary)
4. Identify the metrics and measures to collect
5. Identify how the metric data are collected
6. Identify how the collected data is presented
7. Identify how the team will know the hypothesis is proven true beyond a reasonable doubt
8. Identify how the team will know the hypothesis is proven false beyond a reasonable doubt
The information gathered above is the output of the retrospective. The team should place the falsifiable hypothesis for improvement [retrospective experiment] in a prominent location so that everyone in the organization can see what the experiment is, the problem the team hopes to solve and the hoped for results of the improvement.
*A falsifiable hypothesis is a statement that can be proven wrong through observation or experiment. The structure of a falsifiable hypothesis is, "The team will [make this improvement] by [doing/not doing this action or activity]."