Board index DeFRaG Competitions DFWC 2012

Evaluation System

Re: General discussion

Postby Arcaon » Mar 3rd, '11, 8:00 pm

<hk> wrote:
he should've pushed it a bit, instead of being happy with virtually the same score

You are probably talking about the NPAS.
The 2nd player doesn't know he is second. That doesn't change in any of those systems. He always has to give his best because he has to get maximum time difference on rank 2 because he can take that difference into the next round.
You are saying it as if the players go like. "Oh so he has 14:800, I can do 14:600 but I don't care. It's just 200 less." This argument would be valid if players could see each others times before the deadline. That is not the case.


Yes. But if I take it to the extreme, just to clarify my point:

If rank 1 got 100 points, and rank 2 got 50, every good player would probably try to explore every route and grind it to perfection, just to be sure. If the points were in some linear fashion derived from time, one would be more inclined to think in the lines of "meh, it's barely possible to improve more than +200; I'll be fine with this time", submit the demo, and become third with a good 90-95 points (if we pretend max is 100).

This would be more obvious and apparent in the last round(s), when someone's leading pretty nicely. "Rank 2 would have to beat me with 1-1.5 seconds to win; It's impossible to beat this time with one second, so this will be good enough."

---

But okay, about fairness.

A system based on individual times would probably be one of the most fair ones, providing that the rank 1 points are fixed. E.g. 100 points for rank 1, and everything else relative to that.

If it was some system where the times were simply added together in some manner, factors like individual strengths (someone only being good at strafe maps for example), map exploits (i.e. dfwc2010-4), and map lengths would weigh too much.

Only the skill/point ratio has been taken into consideration above.
Arcaon
 
Posts: 325
Joined: Oct 16th, '09, 1:36 am

Re: General discussion

Postby $$-albion » Mar 3rd, '11, 9:28 pm

Warning! Massive mind blowing post inbound! tl;dr at the end of the post.

At this rate there will never be a ranking system :s

Some of my thoughts about the points ranking system and "add times" ranking system:
  • Less discrepancies when using the points system (either 2-point values or 1-point values)
  • Discrepancies in adding times together can get too big when total amount of maps is high (eg. 10 maps could totally fuck it all up)
  • DFWC is a gathering place for hardcore strafewhores and offline heroes grinding strafe maps and most of all the weapon/combo maps. Differences of 1 or more seconds in time are already very likely to occur on the first map. This adding up to the previous statement.
  • As arca said, route and skills are not the only factors in making a good time, sometimes you need a little luck, therefor it's hard to chose a ranking system which rewards the most skilled player.
  • Some ppl(people) will be satisfied with their time though they could shave off another +0.2xx which could cost them greatly on maps <10s, but won't have as much effect on maps >30s. On the <10s maps it could be a loss of maybe 5 or even more ranks, which would turn out bad in the points system, but less in the added time system when longer maps are yet to be played. For the >30s maps it's a bit on par, +0.2xx will probably mean less than 5 rank difference and stays almost the same for the added points system.

K, that'll be it for now. Maybe I'll add some more later, dunno :p

Also here's my 2 cents on trying to minimize the discrepancies in both ranking systems.
It's based on a 10 map total DFWC, qualification round is optional and a form of elimination is "necessary".

So, 10 maps that consist of 3 times 3 maps that are in the same difficulty category (easy, intermediate, hard) or a mixture of side by side difficulties (easy-intermediate, intermediate-hard, hard-MASTER SERGEANT OFFICER EXTREME TO THE MAX skill) and 1 final round map which will make up either the top 5 or top 10, this would be the choice of the DFWC admins and of course the amount of ppl(people) that will actaully play.

Now onto the basic concept of it.
  • QUALIFICATION ROUND: as I said before, it's totally optional and up to the DFWC admins. Nonetheless, it can be a "great" way to deal with if a huge mass of players decide to take part in the DFWC, if this would be the case. Also, because it's a qualification round, the meaning of it is purely to be sure if one is "allowed" to compete, so no absolute ranking should be applied here. Still, it can give a preview of the level of the competing players you are up against. Now onto the actual rounds.
  • ROUND 1: play through maps 1, 2 and 3, apply one of both major ranking systems and this will be the ranking for round 1 (more info later on).
  • ROUND 2: eliminate a certain percentage of the participants and reset the scores. Play thourgh maps 4, 5 and 6, apply rankings, same as for round 1.
  • ROUND 3: well you get the point now.
  • FINAL ROUND: ok, so after playing 3 rounds, there will be a certain amount of ppl(people) left, I'd say about 10, who will battle for the title of MASTER OF THE UNIVERSE. Wait no, that's my title. They'll battle for the title of dfwc winrar, yuy for me *forever alone*.Obviously no ranking system is necessary in the final round, although this will require a map where all your leetness (route, skills & luck) will be necessary to make an epic time. Best time = winrar.
  • ROUND... Ah, almost got you olollolololollolololo: no but seriously, all you raging homo's read the next few lines.

Ok, no let's clear up some things. For starters, "WTF YOUS DOING WITH MY TIEMS? Y U NO KEEP THEM?", this is probably what everyone's saying after they read the word "reset the rankings". In my opinion this is a rather good system, in which it will require the dedication of the player all through the DFWC, instead of slacking the first few rounds (like I did last year, after which I gave up cause I couldn't continue playing due to the terrible timing for me). It also creates the opportunity to correct yourself when you go to the next round after getting 2 good times and 1 lesser time, which could've set you back a few points or a few seconds depending on the system used. Ok, on the next issue: what to do with the ppl(people) that owned at a certain round? Yeah, this is where I scratched my goatee upon thinking of some sort of "reward". This is, in my eyes, one of the few things that lacks in this kind of system. I won't discuss this matter now, as I only have the 2 most obvious rewards in mind, either giving them some bonus points or bonus time, again depending on the ranking system.

Ok, I think this pretty much covers my 2 cents. If you think I forgot to clear up something, please do tell me, any questions, agreements, disagreements, the same matter. Also, you can rant about it as much as you want, but please, don't be a fool and quote this whole shit. Quote only the pieces you wanna rage about. Last but not least, don't rant when you didn't read through it all. I've put up quite some time into this, while instead I should've been studying, if you don't want to read it, don't rant about it. Also, kudos if you read through the whole thing <33 :D

tl;dr me explaining how I invented the universe.
User avatar
$$-albion
 
Posts: 291
Joined: May 6th, '09, 9:15 pm
Location: Bruges, Belgium

Re: General discussion

Postby TittenIgnition » Mar 3rd, '11, 10:22 pm

$$-albion wrote:Ok, I think this pretty much covers my 2 cents.

albion, thats less like 2 cents and more like $5.
wtf i have a signature?
User avatar
TittenIgnition
 
Posts: 455
Joined: Jun 4th, '10, 1:24 am
Location: QuakeNet

Re: General discussion

Postby $$-albion » Mar 3rd, '11, 10:28 pm

KittenIgnition wrote:
$$-albion wrote:Ok, I think this pretty much covers my 2 cents.

albion, thats less like 2 cents and more like $5.

For $5 I'd write an essay ;D :p Btw what did you think of my masterpiece? :p or did you just read the tl;dr :p
User avatar
$$-albion
 
Posts: 291
Joined: May 6th, '09, 9:15 pm
Location: Bruges, Belgium

Re: General discussion

Postby MidiMightyMoe » Mar 3rd, '11, 10:44 pm

To me the only question is to know if it makes sense to quantify people skill, espacially when different kind of skills are involved (strafe, weapons etc.), and if that's really what we are looking for.
MidiMightyMoe
 
Posts: 68
Joined: Jan 3rd, '10, 2:05 pm

Re: General discussion

Postby Leaf » Mar 3rd, '11, 11:19 pm

flu wrote:
Leaf wrote:I'm pants at defrag, so it make little difference what ranking system they use for me as a competitor. As a spectator, I'd prefer to see in-game skillz rewarded, rather than in-depth map analysis that leads to winning routes like those in dfwc2010-1 and dfwc2010-4.

If someone gained 2 secs over someone else purely because of good strafing, I'm fine with rewarding the size of the gap with points. I'd probably be fine with a clever, creative route. But unfortunately people can also gain 2 seconds by studying the map in detail with a level editor, or worse by a mapper intentionally creating a hidden shortcut for friends to exploit.



Actually, I agree with you. So, What is your suggestion?


I haven't considered what rules I think would work best for a defrag competition. I like the negative points idea. Something like 1st gets 0 points. 2nd -2, 3rd -4, 4th -8, 5th -16, and so on. It's simple enough, and makes every round important.

Perhaps they should use the same rules as 2008, as it makes it easier for everyone.

$$-albion wrote:For $5 I'd write an essay ;D :p


Here you go: http://www.fiverr.com/ :)
Leaf
 
Posts: 20
Joined: Apr 6th, '10, 10:03 pm

Re: Evaluation System

Postby <hk> » Mar 3rd, '11, 11:36 pm

abilon, I stoped reading at "round3". You didn't read either this thread nor the info thread. Everything I read so far was just bogus. We either dismissed things you state up there, like the long map <-> short map problem (fixed through normalisation) or we already say that there will be 6 maps. Not more. Pleas read the info thread and this thread from the beginning to the end. Or else we will be talking in circles here.
User avatar
<hk>
Menstruating 24/7
 
Posts: 1040
Joined: May 2nd, '09, 12:06 am
Location: Ingolstadt Germany

Re: Evaluation System

Postby $$-albion » Mar 4th, '11, 12:21 am

<hk> wrote:albion, I stoped reading at "round3". You didn't read either this thread nor the info thread. Everything I read so far was just bogus. We either dismissed things you state up there, like the long map <-> short map problem (fixed through normalisation) or we already say that there will be 6 maps. Not more. Pleas read the info thread and this thread from the beginning to the end. Or else we will be talking in circles here.


My suggestion just stated the way wherein it would be most likely to have as less discrepancies as possible in the ranking system, when using the basic points system or added time system, when based on a 10 map DFWC, therefor relevant to the topic. Nowhere it said this DFWC needed 10 maps

Yes, I read the info a long time ago, didn't bother to look again, cause nothing changed, but I forgot about the map limit of 6 maps. Though from the info stated as now, everything seems to be taken care. Although it says "still deciding on most things", you posted practically anything time/map/rank related already. In my eyes that's either decided or most of those things are assumptions which can still vary greatly. You should clarify which things are final and which are still up for dicussion (obviously you made a thread for date and ranking system), but it still lacks the point where it all went wrong last year. Which binaries will be legit? Will we get another adjusted binary like last year, which wasn't all that great and some ppl(people) never even noticed the announcement of the binary, or will we be able to use certain existing binaries (baseq3, dfengine, ioq3, ...) or only one specific binary? Personally I care more about that info, cause that'll decide if I'll play or don't even have to bother to follow the DFWC anymore.
User avatar
$$-albion
 
Posts: 291
Joined: May 6th, '09, 9:15 pm
Location: Bruges, Belgium

Re: Evaluation System

Postby AL!EN » Mar 4th, '11, 12:35 am

albion, wow, that is unlikely for you but hey: good job !
"strafe is the most important 'weapon' in defrag" - <hk>
User avatar
AL!EN
 
Posts: 282
Joined: May 2nd, '09, 8:26 am
Location: Poland

Re: General discussion

Postby AL!EN » Mar 4th, '11, 12:38 am

<hk> wrote:Negative Point Accumulation System (NPAS):
The best score on a map is counted as 0. The diffecence in milliseconds to rank1 is defined as the point value. Points add up over maps. The player with the least points, wins.

Example:
Code: Select all
23:000, rank1, 0 Points
23:080, rank2, 80 Points
...


This would be my choice.

23:000, rank1, 0 Points
23:080, rank2, 80 Points
24:264, rank3, ??? 1264 Points ?
"strafe is the most important 'weapon' in defrag" - <hk>
User avatar
AL!EN
 
Posts: 282
Joined: May 2nd, '09, 8:26 am
Location: Poland

PreviousNext

cron