Is human evaluation through surveys common in ML research?[R][D]
We are also comparing its feedback with outputs from Qwen/OpenAI APIs on the same images.
Proof1 community thread
Key points
- My postdoc and PhD mentors suggested creating a Google Form where participants compare the feedback and select which one is more accurate or useful.
- Is this type of human evaluation common in machine learning research?
- Is a sample size of around 60 participants reasonable?
- If reviewer feedback later requires additional evaluation, is it common to conduct another survey?
Sources (1)
- [1]Is human evaluation through surveys common in ML research?[R][D]r/MachineLearning (top, daily) · Oct 13, 03:16 PM
We are also comparing its feedback with outputs from Qwen/OpenAI APIs on the same images.
My postdoc and PhD mentors suggested creating a Google Form where participants compare the feedback and select which one is more accurate or useful.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 13, 20264x 6000 Pro for the company
- Oct 13, 2026black-forest-labs/flux-3-image-klein-comfy
- Oct 13, 2026How far are we from a local model being able to optimize it's own runtime stack?
- Oct 12, 2026[AINews] Meta Muse Wonder & Wonder Full: MSL’s Claude 5.5-tier Frontier Model
- Oct 12, 2026Alibaba Qwen Introduces QwenGyre: Elastic Reinforcement Learning RL for XLong-Horizon Agents
- Oct 12, 2026AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers
