17/01/2023
***WUDC FEEDBACK OVERVIEW***
Hey everyone,
We (the Adjudication Core) decided to present to you some data related to feedback collected during Madrid WUDC 2023.
Massive thanks to our Tab Director Sébastien Dunne Fulmer for their help to extract all the data for us!
https://docs.google.com/spreadsheets/d/1ZP8gZpzW59nbVXDkbIobM_n39K6yp4b87DAfvxfqi7g/edit?fbclid=IwAR2kTg3-tr4H1e5S0fYalBu-X08yx3cUspgDY4mSA3fthtCpwQe6-pxLx5g =2034793890
Data is anonymised, but below you can find some further explanation to the Feedback Comments part prepared by the Adj Core:
1. Some trends pointed out in feedbacks which negatively influenced numerical scores for chairs from the panelist perspective:
a) Time management:
• time of deliberation, (if for example run significantly over time - 30 min)
• length of the OA (rarely, but mentioned if OA were longer than 20 min)
• how much time chair let each panelist to talk (if everyone had a chance to properly speak during discussion)
• how much time was spent on comparisons (80% of deliberation time spent on 1 comparison and rushing over others)
b) Discussion:
• how chair handled panel disagreements, was chair pushing for their call or were leading discussion
• was there pressure to reach unanimous decision
• how chair justified and summarized certain splits
• how discussion was reflected in the OA given
2. Some trends pointed out in feedbacks which negatively influenced numerical scores for chairs from the teams and panelists perspective:
OA content:
• pairwise comparisons (lack of pairwise comparison such as OG vs OO, CG vs OG, CG vs OO, CO vs CG, CO vs OO, CO vs OG usually reflected in lowering the score for the chair)
• interventionism (if teams felt that judge was completing some arguments for teams, using personal weighing metrics without further explanation)
• crediting material to other teams (calling closing teams derivative without explicit breakdown of the arguments and which parts were similar)
• being comparative with regards to burdens, analysis and rebuttals (many comments on higher burdens being placed on other teams and judges not doing a comparison of why one argument is more persuasive than the other)
• spending similar times on splits and clashes - teams deserve similar attention when being discussed in the OA (or at least justification why there is less time spent on particular parts of the debate)
• using buzzwords without further explanation
3. Some trends pointed out in feedbacks which negatively influenced numerical scores for panelists from the chairs perspective:
a) Issues with properly tracking the debate:
• panelists missing crucial parts of the debate; not including that in their reasoning;
b) Not responding directly to the questions being asked:
• example: bringing closing team to top half comparison
• bringing back issues already discussed
• focusing on less relevant parts of material, not on main clashes which panel want to discuss atm
c) Justification for flipping the call
• if you changing your initial call without clear reasoning or justification it might seem that you only doing that to please the chair
d) Lack of explanation for initial call
• judges should always have some justification for their initial ranking, don’t need to be extensive
e) Being rude and interrupting panel discussion
• self-explanatory - chair duty is to lead the discussion and panelists should participate in the discussion with respect to everyone present in the panel
f) Yawning, not taking notes and looking bored or distracted during speeches, using phone during speeches
• teams do not fill feedbacks on panelists (unless chair rolled), but people were noticing that kind of behavior and it came back to us pretty quickly
Please note this is based on all pieces of feedbacks and generalised - does not mean your chair/panelist/team followed this.
Couple of examples of positive trends and highly appreciated feedbacks (reflected in scores) can be found below (anonymised):
• A) comprehensive, across everything in the debate B) good and fair explanations of why things weigh against each other in the way that they do. Doesn’t say anything in the OA that isn’t based in the debate that actually happened, but also has a good level of criticality in assessing claims. Also v good comprehension, can really understand the debate. Both in isolation in the teams cases, but also in understanding how they interact.
• Extremely good OA. Very clear, very detailed, nonetheless efficient. Great tracking of individual mechanisms - did not buy rebuttal that only attacked point headings. Insightful critique of arguments, very useful feedback.
• OA was really sharp. All of the material in the debate was touched in OA. All comparisons were addressed and very clear. Mentioned logical gaps that made cases weak all in comparison to other teams, which really helped understand the call and the things we could have done better.
• Did an excellent job in managing and engaging with all panelists equally and letting them express their opinions, whilst effectively explaining their reasoning at the point their were differences.. The OA was excellent, breaking down the debate in a great detail and analytical depth and offering insightful observations as to why certain splits fell the way they did
• Good and clear OA in which the most important headlines were discussed. Is able to nuancely discuss subclashes, who wins them and how they interact. During the discussion gave clear instructions and ample time for wings to engage and react.
• Managed the panel well without domineering or cutting people off. Clear concise weighing and explanation of match ups. Willing to listen and adjust accordingly given our discussion and reasoning. Good engagement of issues. Was helpful as well when we asked questions.
ADDITIONAL TAKEAWAYS:
1. Comments in feedback matters for judge improvement and justification for scores and ability to find out main improvement points - please provide commentary while filling feedback
2. Comments such as “Excellent”, “Very good” given to every judge who gave you first does not bring much value - presumably every feedback was different, please gave some comments so it can be compared
3. Feedback is not perfect measure, but best which we have for now - teams and judges who are not filling feedbacks through the entire comp should not be break eligible
4. Reputation and cv of the judge should not stop you from providing honest feedback - bitching to your friends afterwards brings 0 change.
5. Being good chair does not equal being good panelist and vice versa - there are different skills required and judges should try to do well in both positions as rotations are something normal and in many cases is not a punishment being demoted from chairing
Main question which we received after WUDC: How I can improve as a judge? Below some recommendations:
Resources (examples, there are more on youtube if you search):
Korea WUDC training program: https://www.youtube.com//playlists
IMPROVING AT JUDGING BY PANEL DISCUSSION | TRAINING ACADEMY EUDC MADRID: https://www.youtube.com/watch?v=aNazoFLX3LY&t=3533s
Judging - Advanced Training Debate Workshop: Week 10: https://www.youtube.com/watch?v=g8En6RPpu2w
Judging at EUDC: Oral Adjudications, with Yair Har-Oz: https://www.youtube.com/watch?v=Txmy9Jzg2qc&t=708s
How to be a good / useful wing judge (BP format)
https://docs.google.com/document/d/17ayx1E7ycLXRxZeeOnrePy7bKgS8w9NXEYpO7gKzn5s/edit?fbclid=IwAR3gx0QrV_w5RtKshqhDBSDDlWkl8yWGrvNj-WawdhIWwAoqv2EvWCBAGvQ
Judging prep comps and spars
• always ask for feedback, some specific comments - people with who you judged
• go for spars, ask if you can panel more experience judges
• watch round online, try to give OA - exchange with someone and check each other OAs - is this clear? were all comparisons made, time yourself
Tips and tricks for better OAs:
• 6 comparisons - ensure that you covering them explicitly in your OA
• buzz words - ensure that you explain what you mean by “not persuasive”, what analysis exactly should be added and where
• think what you would like to hear: “how i could win this round/perform better? what to do?”
• use the words which teams used - you might need to work on your tracking a little bit, but it’s extremely useful, in many cases when we paraphrase the teams cases they feel we changed something, we misunderstood their intention - that way people will feel heard and also appreciate that you pay attention to their words
Tips and tricks for better panel management:
• always start timer and check how much time you have left after each comparison
• ask people “which comparisons if any you think are close” - it can help you better plan your deliberations, which part you might have easy discussion
• ask specific questions, for example: “what do you think was the strong rebuttal presented by CG in response to CO extension”
• refer to teams words and explicit weighings in order to try sort out panel disagreements
If you have any questions feel free contact us via email [email protected]
We hope you find that useful,
Madrid WUDC 2023 Adj Core
PS. all remaining individual feedbacks will be sent out by Friday this week!
Feedback Distributions All Sources score,count 0,11 1,23 2,90 3,185 4,260 5,360 6,600 7,798 8,910 9,578 10,339 From Judges score,count 0,2 1,14 2,67 3,129 4,182 5,272 6,427 7,475 8,526 9,327 10,144 From Teams score,count 0,9 1,9 2,23 3,56 4,78 5,88 6,173 7,323 8,384 9,251 10,195 Feedback Sources...
