Experiments with the 1-5 Scoring system/ Results Posted

Finally, as the main boards settle down, I am pleased to present the results of 2 trial runs of a alternate scoring methodology. All past and current judges, and all who understand the scoring system, please feel free to give your insightful comments.
Just remember to be respectful, before deciding to blast the hard work that went in designing, recruiting and testing this methodology. At least, 8 folks with prior judging sytem contributed to the 2 runs of mock judging using the existing weekly nomination databases.

OBJECT: To evaluate and experiment with an alternate scoring system as posted in an earlier thread: http://www.poems-and-quotes.com/discussion/topic.html?topic_id=133336

With the aim of:
1) Having all 5 Judges score on all poems, to have a much fairer consensus VS. as few as 1 or 2 judges picking a winner, in the current system.
2) Ensuring that all judges read and evaluate all poems with an open mind, diluting any subconscious favors or pursuasions.
3) Reducing ties - Unfortunately it did not. But there is a solution (will comment below)

NOTES: 1) Comments need only be written on the top 2,3 or 4 poems by each judge (the one's on which he/she has used his/her 4's and 5's).
2) Low scoring winners such as 3,3,3,3,3, pointed in the earlier thread will automatically be eliminated, as such poems will not do not have any comments.
3) HM's will remain unchanged. Any poem with a 4 or 5 that has a comment by judges will still be an HM.

METHOD: The following method was used for the mock judging:
All the poems on the nomination should be voted on, not just the 3 best one's. The scoring should be on a 1-5 scale as follows:

1= subpar
2= average
3= Above average
4= Superior/Very Good
5= Outstanding/Excellent

Please also note:
1) Use 5 and 4 only twice, but feel free to use 3, 2, 1 unlimited times, to score all the nominated poems. The limitation on use of 5's and 4's will disallow the judges picking a 5 for all their winners
2) This way an individual judge will not be pressured to pick just 3 of the top poems.
3) Score no more than 3 on your poem/s, and let the other 4 determine the merit of your poem (this was used for the mock game only). In reality, to be fair the 6th judge should score as a sub on a seated judge's poem.
4) As all the other remaining 4 judges will also be forced to vote on your top rated poems (the two 5's and the two 4's), ties may or may not occur at the same rate, but if they do, they reflect that the tying poems were equally good by a consensus opinion VS. single or dual opinion. Favoritism and other issues are therefore greatly diluted when a consensus opinion is used.

RESULTS:

CRITICISMS:
1) Ties are not eliminated. But a soln exists (see below).

CONCLUSIONS: 1) The winners are chosen from a pooled vote casted by all 5 judges as shown, not from just 1 or 2 judges opinion.
2) Ties are not eliminated. The ties in this mock judging run simply go on to show that there are many equally good poems some weeks that should be given equal weighting and should not be eliminated or overlooked easily, without due consideration by a tie breaking sixth judge. The sixth judge should be used to break the ties, similar to the recently adopted tie breaker method in the current system.

These are the results from the Second of the 2 Mock runs, using 5 experienced judges in each run (Thank you Judges for your time and energy to conduct this experiment):

WEEKLY CONTEST 1-16-12 MOCK JUDGES SCORED FROM 1-5 (But used only 2 of 5's and 2 of 4's and unlimited 1,2,3's)

What a feeling 3 3 4 3 4 = 17
by Yaki ( F P C D )
The Number Six 4 3 3 3 3 = 16
by Lioness ( F P C D )
The Diary Of Tears ( Seven ) 1 3 3 4 2 = 13
by ADEL ( F P D )
Rushed Destiny 2 3 2 3 3 = 13
by PinkyPrincess ( F P C D )
Little Miss Happiness 2 3 2 2 1 = 10
by Average thoughts ( F P C D )
Inundated 3 3 3 3 2 = 14
by Jenni ( F P C D )
Inbetween Emptiness 3 3 2 3 2 = 13
by AngelDust ( F P C D )
Hands 5 5 5 3 3 = 21
by Kiko ( F C D )
When I'm Gone 3 3 2 3 2 = 13
by gIrL ( F P C D )
You are my girl for life 1 3 1 2 1 = 8
by thomasburdine
U Love I 1 3 3 3 2 = 12
by Everlasting ( F P C D )
Past, Present, Future 2 3 2 3 3 = 13
by Tyler Beckham
Stringing Hearts With Lyrics 2 3 2 3 2 = 12
by Imperfectly Writing On The Walls ( F P C D )
Emptiness of Fear (Villanelle) 3 3 2 3 3 = 14
by Lonely Rider ( F P C D )
Exit Sign 4 3 3 2 2 = 14
by mandy ( F C )
Elephants 5 3 3 2 2 = 15
by Lebanese Phoenix ( F P C D )
Shadow Of A Marionette 3 3 3 4 3 = 16
by Tara Kay ( F P C D )
Impermeable Layer 5 3 2 5 2 = 17
by Ronnie
Free Bird Parade 3 4 3 3 3 = 16
by Maple Tree ( C )
Vulnerability Of A Dolly 3 3 3 3 3 = 15
by Tara Kay ( F P C D )
Time of absence 3 3 3 3 2 = 14
by Chelsey ( F P C D )
Passenger Seat 3 3 3 3 3 = 15
by The Queen ( F P C D )
Luscious Craving 2 3 2 3 2 = 12
by Innocent Fairy ( F P C D )
In circles 2 3 4 3 5 = 17
by The Princess ( F C D )
Bide deep 2 3 3 3 3 = 14
by Schemilix ( F P C D )
I am a Monkey 2 3 1 1 1 = 8
by Bobo
We Can 2 3 1 3 2 = 12
by gIrL ( F P C D )
Sewn Up Excuses 3 3 2 3 3 = 14
by LTFR Club
Because 3 3 5 3 3 = 17
by Melissa ( F P C D )
Solitude 3 3 3 3 3 = 15
by Yaki ( F P C D )
She doesn't know but I wept too 3 3 2 3 2 = 13
by Karla ( F P C D )
Content 3 3 3 3 3 = 15
by Yaki ( F P C D )
The Muse Of A Beautiful Woman 2 3 2 3 3 = 13
by Tara Kay ( F P C D )
Bleeding Crimson Rain 3 3 3 5 3 = 17
by Hellon ( F C D )
What it means to miss someone 1 3 2 2 2 = 10
by Julie
64 Olive Trees 3 3 3 3 3 = 15
by Lebanese Phoenix ( F P C D )
Facebook Flashback 2 3 1 3 2 = 11
by Lostlove1 ( F P C D )
Pleasure In Fear 3 3 2 2 1 = 11
by Jon914 ( P )
Last to Know 2 3 2 3 2 = 12
by PinkyPrincess ( F P C D )
A Happy Place 1 3 1 3 2 = 10
by Ronnie
My Memory Library 1 3 2 3 5 = 14
by Born As a Sin ( F C )
Don't Be Sad When I Die.. 1 3 1 3 2 = 10
by lOsT sOuL ( F C )
A Human Touch 2 5 1 3 3 = 14
by Lostlove1 ( F P C D )
When Rainbows Clash 3 3 3 3 4 = 16
by Lonely Rider ( F P C D )
Sincerely the Earth 2 4 2 3 3 = 14
by TJ BECKER Arizona Eagle ( F C D )

MOCK JUDGING WINNERS:

1st Place: Hands By kiko Score - 21

2nd Place: What a Feeling By Yaki Score - 17
2nd Place: Impermeable Layer By Ronnie Score - 17
2nd Place: In Circles By The Princess Score - 17
2nd Place: Because By Melissa Score - 17
2nd Place: Bleeding Crimson Rain By Hellon Score - 17

3rd Place: The Number Six By Lioness Score - 16
3rd Place: Shadow of a Marionette By Tara Kay Score- 16
When Rainbows Clash By Lonely Rider Score - 16

WEEKLY SITE CONTEST WINNERS 1-16-12:

Number six By Lioness 7+4 = 11
Hands By kiko 10+10 =20
What a Feeling By Yaki = 10

3rd Place ties: The Queen 10, Ronnie 10

These are the results from the First of the 2 Mock runs, using 5 experienced judges in each run (Thank you Judges for your time and energy to conduct this experiment):

NOMINATIONS FROM WEEK OF 1-9-12
MOCK JUDGES SCORED FROM 1-5 (But used only 2 of 5's and 2 of 4's and unlimited 1,2,3's)

Exbury Garden 3 5 2 2 5 = 17
by Lioness ( F P C D )
Metaphorical Case Love 2 3 3 2 3 = 13
by Lebanese Phoenix ( F P C D )
Enlighten 3 3 2 3 3 = 14
by Yaki Love ( F P C D )
New Mexico 3 3 3 4 3 = 16
by Colm ( F C D )
Hands 3 4 4 5 3 = 19
by Kiko ( F C D )
Origami 3 3 5 3 5 = 19
by Karla ( F P C )
Cardiac Confession 5 3 3 4 4 = 19
by Monster ( F P C D )
An Eclipsed Sunshine 4 3 2 3 3 = 15
by Ms Eclipsed Sunshine ( F P C D )
I am a Snowflake 3 3 3 3 3 = 12
by Maple Tree ( C )
Acid Ghost 2 3 2 3 2 = 12
by Maple Tree ( C )
Vanishing Moonbeams 3 3 3 3 2 = 14
by TJ BECKER Arizona Eagle ( F C D )
Sincerely the Earth 3 5 2 3 3 = 16
by TJ BECKER Arizona Eagle ( F C D )
Train ride to? 2 3 2 3 3 = 13
by Jenni ( F P C D )
Crude Assests 3 3 2 3 3 = 14
by Tara Kay ( F P C D )
Bleeding Crimson Rain 3 4 4 5 4 = 20
by Hellon ( F C D )
Sweet Taste Of Satisfaction 2 3 3 3 3 = 14
by Tara Kay ( F P C D )
Have You Forgotten? 2 3 2 3 2 = 12
by Britt ( F P C D )
Remapping Morals 2 3 2 1 2 = 10
by Vi veri veniversum vivus vici ( F P C D )
Crimson needs 1 3 2 1 1 = 8
by Lithium1027
Content 2 3 3 3 2 = 13
by Yaki Love ( F P C D )
A trance of serendipity? 2 3 2 3 2 = 12
by Everlasting ( F P C D )
The Diary Of Tears ( Seven ) 2 3 3 1 2 = 11
by ADEL ( F P D )
Elephants 2 3 3 2 3 = 13
by Lebanese Phoenix ( F P C D )
Self Annihilation 3 2 3 3 1 = 12
by Beautiful Chaos ( F P C D )
A Human Touch 5 3 3 3 3 = 16
by Lostlove1 ( F P C D )
Little Miss Happiness 1 3 2 1 1 = 8
by Average thoughts ( F P C D )
Exit Sign 2 3 2 3 1 = 11
by mandy ( F C )
I became 3 3 4 3 3 = 16
by Yaki Love ( F P C D )
Free Bird Parade 3 3 5 3 2 = 16
by Maple Tree ( C )
Apart- 3 3 3 3 3 = 15
by Poet on the Piano ( F P C D )
What it means to miss someone 3 3 2 2 2 = 11
by Julie
The Lost Creak Of Summer (Collab) 4 3 2 3 3 = 15
by Lebanese Phoenix ( F P C D )
Did you become yourself? 2 3 2 3 2 = 12
by Karla ( F P C )
Rushed Destiny 3 3 2 2 3 = 13
by PinkyPrincess ( F P C D )
Unspoken Apologies 3 3 2 2 3 = 13
by gIrL ( F P C D )
Solitude 2 3 2 3 3 = 13
by Yaki Love ( F P C D )
Your image 3 3 2 3 3 = 14
by Liliana ( F P C D )
Inbetween Emptiness 2 3 2 2 2 = 11
by AngelDust ( F P C D )
Nature's Cry 2 3 2 2 2 = 11
by Matthew
Bide deep 3 3 3 3 2 = 14
by Schemilix ( F P C D )
Rocky Roads 2 3 3 3 2 = 13
by Dead Soul ( F )
Mystery To Be Defined 1 3 2 1 2 = 9
by xTruthxByxAccidentx ( F P C )
Emptiness of Fear (Villanelle) 3 3 3 3 3 = 15
by Lonely Rider ( F P C D )
Love You Less 1 3 2 1 3 = 10
by gIrL ( F P C D )
Luscious Craving 2 3 2 1 2 = 10
by Innocent Fairy ( F P C D )
Impermeable Layer 2 3 2 2 2 = 11
by Ronnie
I am a Monkey 1 3 1 1 2 = 8
by Bobo
Facebook Flashback 2 3 2 3 3 = 13
by Lostlove1 ( F P C D )
When I'm Gone 2 3 3 1 3 = 12
by gIrL ( F P C D )

MOCK JUDGING WINNERS:

1st Place: Bleeding Crimson Rain By Hellon 20 points

2nd Place: Hands By kiko 19 Points
2nd Place: Origami By Karla 19 Points
2nd Place: Cardiac confession By Monster 19 points

3rd Place: Exbury Garden By Lioness 17 Points

ACTUAL WEEKLY CONTEST WINNERS 1-9-12:
Acid Ghost By Maple Tree: 10+10=20 points

An eclipsed sunshine
By Ms Eclipsed Sunshine (10+7) = 17
Enlighten
By Yaki Love (10+7)=17

ACKNOWLEDGEMENTS AND NOTES:

1) Many many thanks to my Judges, who gave their valuable time to test this system.
2) All who volunteered were experienced judges. Not one complained about the methodology, but I have not yet polled them as to what they thought of this method compared to the existing 10, 7, 4 scoring system.
3) Most of them wish to remain anonymous, for reasons understandable, so please do not criticize the judges, only the method. I do prefer that they remain anonymous, as by identifying themselves they may be subject to personal attacks. I will take all personal attacks on their behalf.
4) Please do not be repetetive with your comments and criticisms. I personally will prefer if you wish to add to your original comment, do so by EDITiNG your original post, so as to allow everyone an equal chance to be heard. I would love to see a much wider participation in this forum, for it to have a constructive value.
5) The above test run results will differ than the actual contest for that date/week, as judges have different tastes. But, please be assured, that results here are to test an alternate methodology and not to usurp the results from our current panel of esteemed judges. Likewise, contest participants (i.e. writers) please do not be offended if you see lower scores here, though you may be a winner or earned a HM in the actual contest. It would be unfortunate, if you use the results from here on your poems to criticize the proposed methodology. Likewise, for all those poets whose poems were nominated, if you scored lower than the winners or really low, please do not be disappointed, and discouraged. The purpose here is to get a feedback from the judges of a mock contest for our own writing betterment. Do not forget, I am one of you who writes for pleasure and not to show my prowess or compete with the real poet-wanna-bees.

I cannot say, how effective it is! I need to examine the results a bit more, to be honest and compare it...so that I be saying the truth.

As for the method itself, I was one of those judges and i thought it was fair as a "method" not sure as a complete result, because I already read all the poems through the weak, it did not take me much time to decide what vote shall I cast for an X poem.

I think this is doable...functional somehow!

as long as judges don't have to write comments for all the poems lol.

The sustem is not on display here? not the current one or your one..

What you have done is make a mockery of the weekly contest, and pit one set of judges against another. And make one lot of weekly winners feel undeserving, and the ones that won with your judges cheated.

Who the hell would want to be a judge on this site??? Sure as hell not me, this whole weekly contest is becoming a joke

Oh one more thing, which set of judges are right?

Oh one more thing, which set of judges are right?

^
O.o wait..is it about the judges! or the method itself ?

I think it's about the method, not the judges...
bit got lost now, I guess.

Really? My poem A HUman Touch=1 Subpar? Whatever-well now I'm all for the unveiling of MOCK judges...

"Who the hell would want to be a judge on this site??? Sure as hell not me, this whole weekly contest is becoming a joke"
^I agree Nicko~!

I'll go first. I was a mock Judge in the first round-
I thought it was a great idea actually but now I am not so sure- Yes it forces the judge to use his votes carefully And no one knows or sees what the others are voting for- but theres still the ties and the Judges can still favor their friends or whatever...

I think A Lonely Soul's aim was to compare the two judging methods and not two batch of judges.

And I find it really interesting.... what do we lose if we compare?

Way too many ties, which is what we've all been trying to avoid. The ties are even wider spread here rather than just having two poems possible and a sixth judge choosing. Too complex.

I'm kind of with Nicko, if this were the method, I wouldn't accept to judge, and I don't think a lot would. It seems like it would be a lot of work. I take my time when judging, and write out the titles I narrow it down to. With all the extra votes, if would fry my brain.

And someone gave that monkey poem a three? Really??

I will say, though, the judging on my poem wasn't right IMO... Shouldn't have had any 3s.. it was definitely not a good poem.

I was judge #3 first round in case anyone wonders...

Now hows about that awesome Monkey poem judge...stand up and take a bow!

I do see the advantages of this system, although it would really, really be a lot of work for the ones judging. The one thing that should be added is the judges not knowing who wrote which poem, because this still allows room for personal feelings coming into play..and favoritism.

If we were to have one person accepting all poems* that we, as poets WANT to be judged fairly and we feel are eligable for a win, and all poems would in turn be viewed by anonymous judges then the system could work. Now you sometimes have poems that are very personal ( family member dying etc.) and it gets zero attention..or downvoted by certain persons who feel it is their duty to do so with each poem in the contest...could be very, very painful to the poet, who never asked for his poem to be nominated;)

* Off course those poems should only be posted to the individual accounts after the contest is over.

This thread hurts my eyes tho..

but atleast lonely soul is trying to come up with Ideas, kudos.

I'd like to hear from the mock judges on what they thought especially judge #3 second round lol

That's another issue running into it - without comments people want to know why they got a 1, 2, or 3... if you have unlimited amounts of 1-3, you're not going to want to vote/comment on 40 something poems. I'd assume you'd want to comment on the 4's and 5's only.. while that makes sense, it's the ones who get 1-3's wanting justification for the vote. More work for judges = less participation in judging. :(

Give em a 1-knock them out of the race with no comment or reason why...I see that happening too.

I was one of the judges. I don't know which number, so don't ask me! : ) But here were my criticisms.

1. It took me 1 hour and 50 minutes to complete judging, and I had even read some of the poems beforehand, throughout the week.

2. I felt like all the poems in the 3 -5 categories had to be read twice, because I couldn't score them without comparing them to all the others. After my first read-through, I had about 4 poems ranked a 4, and several 5's as well. I had to go through and re-rank them within the context of each other. So, most of the 4 poems got pushed down to three, and the extra fives were fours (or even threes). To me, this offers the same difficulty that judges already face, in picking three poems out of 45 and ranking them 10, 7, and 4. I don't see it as being enough of a change from that.

3. (And this is the big one) I really don't like the idea of giving poems low scores. With the current system, if you think a poem is low quality, you just ignore it. No feelings hurt. With this way, people are going to wonder why they got a 1 or 2, and probably be offended by it. I think there's no need to point out exactly what I think of a "lower" poem. I'd rather just pass it over for now, and give the poem room and time to grow without discouraging them. If one of those poets enters a month-long contest or something, that's a bit different. They're asking to be judged. But nobody asks to be nominated and judged for the weekly contest, and I don't think they should be discouraged and critiqued for it.

Connie, I don't see your poem having a 1? It says 5, 3, 3, 3, 3. It should equal 17, though, not 16.

I sometimes feel that there are too many poems to be judged fairly, only having 3 votes available but the way experimented here means that many people would be given low scores and have no feedback on why, and it seems a little more confusing for the judge host,

Plus look at all the ties, it's just ridiculous, the current way isn't perfect but it seems to work relitavely well.

It does seem like it has more ties, in the second and third place but it was worth the attempt.

I was one of the judges too and it took me long to rate the poems because there were so many good poems. I don't know about the system now but I do think we should have 5 winners instead of only 3 per week.

Sibs:

A Human Touch 2 5 1 3 3 = 14
by Lostlove1 ( F P C D )

and

Facebook Flashback 2 3 1 3 2 = 11
by Lostlove1 ( F P C D )

Same Judge, Hmmm....2nd round Judge #3

Sibs and Karla can you please claim your Judge spot #
its easy to figure out- just look back and figure out where you used your 4's and 5's...
unless of course you'd rather keep it secret

Well done for putting the effort into it, but I think I agree with most of the points above that indicate that it doesn't really work out.

The main merit of this system is that all nominees get an idea of how well they scored. This would have been more accurate and efficient if judges were not forced to rank the poems by comparing them to each other (which takes the most time), but by giving them on-the-spot scores (which takes a few seconds). This means lifting the restrictions on the number of 4s and 5s each judge has- something that wouldn't influence the contest too much anyway, given that ties are inevitable. The main problem would be the winners may not be guaranteed judges' comments.

Personally, I would give a poem here a 1 much more easily than a 5. If the writer had nominated his/her own poem for the contest (as is being discussed in the other thread), I would have no qualms about being so unforgiving.

The on-the-spot scoring method (in liu of ranking poems) would be a great way for judges to handle the influx of poems. As each poem is scored, it could be hidden from their list. This way, the judges wouldn't have to wait till a certain day to compare all the final poems. The poems would stand in their own right.

Fine..

I would personally like to ask that downrating judge to explain why they rated my human touch poem a 1-either here or in pm....
Not that they will ever fess up lol

but hopefully they burnt their bridge on ever rating or judging anyones poem on this site-

You know that poem was better than a 1

COme out Come out wherever you are...

*edited for profanity*

Hold your horses lady...

It's not that a judge is afraid of your woman anger; they didn't like your poem. Why this nagging, then? And why would it differ.. If you think you have props, then you shouldn't be mad lol

Sorry Sibs.
Anyways you all take your contest and do with it what you like. It will never be fair no matter what you do.

LP what is a props

Open your eyes man...they dont like me! Thats ok. I have asked that they tell lonely soul why they rated as they did not the facebook one it was just a fast little jingle...My Human Touch only

Boys I think we found our downrater!

I mean if you think you're a great writer, then you shouldn't mind.

I was a judge in the 1st round also.

And hey lostlove, one of your poems in the !st round has got perfect scores! Though in the 2nd round, the same poem scored low...

That means judging is a taste. They either like it or not

I agree judging is a taste...But God knows corrupt people put in high places can indeed change things so I think it has to start there. So whatever...

I'm over it almost :)

Your right I do have props :) Have a great day LP

I must say, I'm not a fan of all of the ties in here. I didn't think it would turn out that way...but then again I suppose if there is more freedom for the judges to give away points, there must be more room for ties. It's a good idea but I think it's a bit too complex for something that is more so just for fun than anything.

Lostlove: maybe this judge just didn't like your style? It happens. I wouldn't take it to heart.

Well seems like that judge would pm directly and tell me why he lol gave my poem such low ratings.
And I give them the same challenge Colm gave me... To write a poem from the point of view of an animal domesticated by humans-

I mean geez the same scoring as the monkey poem. wow

I got a 1 as well on U love I but I don't feel it should make me feel bad about it, It just say that someone didn't like my style. But that's ok, not everyone will like it.

Thank you for doing this! That must have been a heck of a lot of work. At the end of the day a suggestion was put to the test and it didn't work. Ok great now we can move on from that. At least it was tried. That to me is great. No one said it'd be perfect or that it'd be better. I'm just happy that a solution was offered and tried. Instead of bitching about this that and the other people actually took the time to try and find a solution. That to me is a step in the right direction.

I do however feel that things are getting complicated for this particular topic.

I'd like to really suggest giving senior members only 1 vote. This really cuts down on the poems for the judges and makes the senior members have a bit more thinking and comparing on their hands before they hit vote.

In any case my only issue ever was with people getting ties then some people getting the HM when they in reality tied with the 3rd winning poem. Just the entire computer generated 'win' was too impersonal for me. But if it's complicating things I think we should go back to how we had it.

Lost love: I didn't read your poem. But I have to be honest, a little modesty goes a long way.

"I mean geez the same scoring as the monkey poem. wow"

Wayta bring yourself down to earth there. Lol.

So what's being done about this? Are we sticking with the 6th mystery tie-breaking judge? I sure as shoot hope so.

Wayta bring yourself down to earth there.

lol Jordan, that's so true. Sorry, but them girls shouldn't belittle pieces!

The changes instituted earlier remain in effect.

Thank you for attempting this experiment, ALS, since it goes a long way to reflect the simple genius with which Janis set up the current system.