GP Individual Quiz Appeals Panel?

This is a repository for ephemeral and/or superseded information posts and discussion topics which are nevertheless thought worthy of retention.
User avatar
Chris
Site Admin
Site Admin
Posts: 792
Joined: Thu Sep 15, 2011 1:02 pm
Location: Burton on Trent

GP Individual Quiz Appeals Panel?

Postby Chris » Sun Dec 02, 2012 5:40 pm

Gareth Kingston has suggested on Facebook that maybe at GPs a few people over lunch could get to the bottom of disputed answers?

In response I said...

"I absolutely support the idea of getting the scores right. But we'd need a quick and simple way of then identifying all the scores that needed amending. It could be OK when we have just one or two questions under scrutiny.

I'm prepared to give this a trial at Coventry next month. Perhaps, Gareth, you'd be willing to chair a panel of, say, three volunteers tasked with acting as an appeal panel? We maybe also need to agree that the time for lodging an appeal is 10 minutes after the conclusion of the marking. We also need to find a way of politely turning away frivolous appeals, lest the panel get swamped with pointless challenges? ..."

P.S. I think we might also need to restrict the process to straightforward issues of fact - rather than complaints that the wording of a question was ambiguous (in such cases everyone labours under the same handicap).


This thread has been opened up to permit discussions of this idea. All are welcome to pitch in.
Chris Jones
Director, Quizzing Ltd., BQA and iQa

ntfc2
Master Quizzer
Master Quizzer
Posts: 15
Joined: Mon Nov 14, 2011 9:59 pm

Re: GP Individual Quiz Appeals Panel?

Postby ntfc2 » Sun Dec 02, 2012 6:27 pm

Having thought about, may I suggest the following:

1, At the end of the marking of each genre the QM asks if anyone would like to challenge an answer. Anyone wishing to challenge must state the question number and their alternative answer.

2, The challenge can only only be on matters of fact rather than opinion.

3, Anyone who is affected by a challenge (either had the answer given, or agrees with the challenge) puts an asterisk on the hand-in sheet, and puts details on the back. The score handed in is the score according to the answers given by the QM.

4, The Appeals panel uses available technology to assess the merits of each appeal. Where a clear factual error can be established appeals are upheld. Where there are alternative answers that are equally acceptable, the appeal may be upheld. Where there is no clear case for an answer being equal to or better than the answer originally supplied, then the appeal is declined.

5, The panel submits a report to the QM, and scores are amended where required

User avatar
Chris
Site Admin
Site Admin
Posts: 792
Joined: Thu Sep 15, 2011 1:02 pm
Location: Burton on Trent

Re: GP Individual Quiz Appeals Panel?

Postby Chris » Sun Dec 02, 2012 7:18 pm

Yes, I like that.
Chris Jones
Director, Quizzing Ltd., BQA and iQa

DrIanBayley
Grand Master Quizzer
Grand Master Quizzer
Posts: 4
Joined: Tue Dec 20, 2011 12:21 pm

Re: GP Individual Quiz Appeals Panel?

Postby DrIanBayley » Sun Dec 02, 2012 8:05 pm

I agree with this idea but I think it needs a bit of work.

(I've just realised that this post has the same spoiler as on John's Facebook timeline so I'm editting it. I'm changing the answer. Let's say the official answer was Venezuela but I and many others wrote Colombia.)

Take the contraversial fish question. I wrote Colombia so of course I'd write
PW 19. Colombia
on my question slip in the hope of getting an extra point. If the panel found the same evidence that John did then I would have this rejected and the scores would remain the same.

However, everybody who had written Venezuela would also be required to write
PW 19. Venezuela
on their answer sheet so that they could be fined one point if it turned out that the panel found in favour of Colombia. So what do you do about those people who don't remember to indicate their answer for PW19? Fine them a point anyway in case they are strategically forgetting on purpose?

Or shall we say that the panel only has the authority to add an alternative answer rather than change an answer?

Also, we need some clear rules about what we are and are not going to allow as sources.

ntfc2
Master Quizzer
Master Quizzer
Posts: 15
Joined: Mon Nov 14, 2011 9:59 pm

Re: GP Individual Quiz Appeals Panel?

Postby ntfc2 » Mon Dec 03, 2012 12:48 am

Well the person making the note of the contentious question would be the marker not the quizzer, and they would be overtly prompted to do so.So I don't think it's the problem you imagine.

And you are right, that anyone who put either the answer being challenged or the alternative answer would have to have a note made of it so scores could be adjusted up or down. But that's only right. It's not a fine. How can it be if the answer they put is wrong? You aren't denying them something they are entitled to.

I think re: sources there would have to be a degree of trust that the panel are able to distinguish between sources in terms of credibility. For example, a blog or homemade web page may or may not be right. Wikipedia is generally ok and on the fish question backs up the challenge. The Smithsonian website backs it up too, and would probably be deemed a credible source by most. And then there is the museum that displays the fish in question. They are the most credible of all. The panel would have to agree that the sources are compelling enough to make a confident judgement.

User avatar
Chris
Site Admin
Site Admin
Posts: 792
Joined: Thu Sep 15, 2011 1:02 pm
Location: Burton on Trent

Re: GP Individual Quiz Appeals Panel?

Postby Chris » Mon Dec 03, 2012 2:10 am

I'm grateful to Ian for raising the point about those quizzers with whom the appeal does not originate.

It ought to be simple enough for us here in Quizzing Towers to ensure that the reverse of the score/tear off portion of the fly sheet contains a form for ALL markers to complete in the event of a dispute. Perhaps an entabulation allowing up to three entries for each listed genre.

Using the example of the fish question, when the appeal is confirmed at the end of the marking of the PW genre, EVERYONE would immediately go to the reverse of the fly sheet and enter, under PW, 19 (i.e. question 19) and "country name". It would look something like this...

Image

Given that everyone will have to make a note of the relevant answer for every appeal, and that this will be mildy onerous, this should serve to discourage frivolous appeals (without hopefully scaring more 'timid' folk off genuine ones).
Chris Jones
Director, Quizzing Ltd., BQA and iQa

AndrewFrazer
Candidate Master Quizzer
Candidate Master Quizzer
Posts: 15
Joined: Mon Nov 14, 2011 8:32 am

Re: GP Individual Quiz Appeals Panel?

Postby AndrewFrazer » Mon Dec 03, 2012 11:04 am

This all seems sensible.

There will of course be questions where additional valid answers emerge, rather than the original answer being wrong. And in some cases this may depend on interpretation of the question. Therefore, while I generally agree that disputes should be about fact rather than opinion/ambiguity, I think that ambiguity that reasonably leads to a different answer (as opposed to ambiguity which just makes a question with one answer harder to understand) should be in play. Further, I think that the overall philosophy should be relatively generous in allowing alternative answers, while only disallowing the original answer where there is clearcut error.

The best example I have from Saturday is Culture TB7 (which I didn't raise then because it was only a TB and this system was not in place). This was the one about the country known as Chosun. This is basically a/the Korean word for Korea(n). It does appear from Wiki that the word is more commonly used in the north than the south to describe Korean script (and possibly in the country's official name, but that's not explicit). But I don't think the word as such can be exclusive to the north. Furthermore, the state which rules the whole peninsula from the 14th to late 19th century is commonly known as the Chosun dynasty/empire, which is why I put just plain Korea as my answer. I naturally think this should have been allowed.

Andrew

ntfc2
Master Quizzer
Master Quizzer
Posts: 15
Joined: Mon Nov 14, 2011 9:59 pm

Re: GP Individual Quiz Appeals Panel?

Postby ntfc2 » Mon Dec 03, 2012 11:58 am

I think alternate answers could be looked at. For example, if someone put "Greasy Grass" instead of "Little Big Horn" it would not take long to establish that "Greasy Grass" is an acceptable alternative.

I also think chosun is acceptable to look at, as long as you can find explicit reference rather than drawing inferences from other facts.

What isn't realistic is to look into "well if you read the question this way it leads to a slightly different answer" questions.

The reality is that the panel will need to be able to make judgements quickly and there ought to be a guillotine, so that if an issue cannot be resolved by a certain time the appeal fails. Not because it is wrong, but because it cannot be established in a practical timescale. This can't drag on all afternoon throughout the team quiz.

I would propose that 3 results are available from an appeal:
Upheld
Rejected
Not proven

The latter recognising that there are limits to what can be achieved by the panel in the time available, or that there is contradictory evidence but nothing to sway a view one way or the other.

ntfc2
Master Quizzer
Master Quizzer
Posts: 15
Joined: Mon Nov 14, 2011 9:59 pm

Re: GP Individual Quiz Appeals Panel?

Postby ntfc2 » Mon Dec 03, 2012 11:59 am

And Chris's table is how I envisaged it too.

User avatar
Chris
Site Admin
Site Admin
Posts: 792
Joined: Thu Sep 15, 2011 1:02 pm
Location: Burton on Trent

Re: GP Individual Quiz Appeals Panel?

Postby Chris » Mon Dec 03, 2012 12:34 pm

Just in reply to Andrew; I'm not sure the Chosun question is a good example. The question asks...

"Which country is known as..." and not "Which country was known as..."

Accordingly, a quizzer is alerted to the fact their answer must be a the name of a country as it is currently known/currently exists.

My concern about 'ambiguity issues' (if only from a practical standpoint) is that I am often tackled by quizzers over lunch who have a gripe about a question's wording when, as often as not, the problem is that THEY misread it. If we have too broad a scope of appeal the panel may get swamped and we could spend the entire afternoon deconstructing the quiz.

Maybe we can give this a limited trial to begin with - i.e. on issues of fact, including discussion on acceptable, alternative answers (something I've always been happy to consider, and allow) - and if that goes well, and if the panel have scope and time to broaden things, then, but only then think about increasing the scope. But remember they will need time to get their lunch etc.. We really ought to be trying to limit their commitment to about 15 minutes work, if possible.

I also think anyone who raises a weak appeal, that is subsequently rejected, ought to consider buying Gareth et al a drink for wasting their time and foreshortening their lunch-break :)
Chris Jones
Director, Quizzing Ltd., BQA and iQa


Return to “Archive (past events)”

Who is online

Users browsing this forum: No registered users and 8 guests