In the tasks of machine learning and data analysis it is often imperative to collect a massive amount of manual markup. Given a small number of performers, this work could take months.
We know how to make it fast and cheap! The talk encouraged by the experience of the Yandex.Toloka crowdsourcing platform creation and use shares clues on the quality control, motivation of performers and various models of aggregating separate judgements.
Download presentation