Gold questions are a standard mechanism to detect insincere workers on crowdsourcing platforms. They usually rely on the assumption that workers should obtain perfect accuracy on the task. In this work, we are interested in crowdsourcing difficult multi-class visual recognition tasks, for which this assumption is not met, and we propose a novel method for generating gold questions in this context.

