In a Nutshell 3 runs, Amazon Mechanical Turk, External HITs One HIT for each set of 5 documents = 435 HITs (2175 judgments) $0.20 per HIT = $0.04 per document Run 3 Stepwise…