Some thoughts on ranking systems

For topics that last throughout the whole season
Post Reply
User avatar
Shmion84
Moon princess
Moon princess
Posts: 3383
Joined: Mon Apr 02, 2012 6:33 pm
Badges:
ImageImage
Melon Pan: 95
2017 Female Favorite: Konjiki No Yami
2017 Male Favorite: Kyon
Wish: ISML still exists next year
Location: Central Europe

Some thoughts on ranking systems

Post by Shmion84 » Tue Dec 31, 2013 12:07 am

For a long time I wanted to compare the overall strengths of the characters. Overall Votes Ranking has the disadvantage of a very different user participation over the years.

Another ranking is the Success Ranking. Every year the characters gain points for their success in the tournament. The winner (of the heavenly tiara) gains 1 point. The runner-up gains 1/2 = 0.5 points (two characters in the finals). The defeated semifinalists gain 1/4 = 0.25 points (four characters in the semi-finals). And so on. The defeated characters in the first round of the divisional finals gain 1/32 = 0.03125 points (32 characters in the first round). Characters who did not qualify for the Elimination Period gain 1/64 = 0.015625 points in 2008 and 2009, 1/50 = 0.02 points in 2010 and 2011 and 1/72 = 0.01388.. points in 2012 and 2013.

The following image shows the ranking of all 177 characters. Top 10: Misaka Mikoto, Tachibana Kanade, Gokō Ruri (Kuroneko), Shana, Katsura Hinagiku, Akiyama Mio, Fate Testarossa, Nagato Yuki, Eucliwood Hellscythe and Aisaka Taiga.
imageShow
Image
Average points per year are shown in the right column. Top 10 average points: Tachibana Kanade, Gokō Ruri (Kuroneko), Misaka Mikoto, Akiyama Mio, Fate Testarossa, Eucliwood Hellscythe, Shana, Katsura Hinagiku, Nagato Yuki and Nakano Azusa. Best Nova characters are Kuroyukihime and Shiina Mashiro (rank 12 each).

Yet another ranking is the Overall Wins Ranking (see: viewtopic.php?f=30&t=1593" onclick="window.open(this.href);return false;). Its disadvantage is the different number of league matches per year. 63 in 2008 and 2009, 49 in 2010 and 2011 and 35 in 2012 and 2013.

A variation is the Overall Win Percentage Ranking (see: viewtopic.php?f=30&t=677&start=40#p200035" onclick="window.open(this.href);return false; for my first thoughts on this ranking). 1 win in 2008 and 2009 is worth 1/63 = 0.015873..% , 1 win in 2010 and 2011 is worth 1/49 = 0.020408..% and 1 win in 2012 and 2013 is worth 1/35 = 0.028571..%.

The following image shows the ranking of all 177 characters. The win percentages are added together in the blue column.Top 10: Shana, Nagato Yuki, Suzumiya Haruhi, Katsura Hinagiku, Aisaka Taiga, Saber, Misaka Mikoto, Sakagami Tomoyo, Fujibayashi Kyō and Akiyama Mio
imageShow
Image
Average win percentages are shown in two columns on the right:
i) in the yellow column average win percentage per year - sum of the win percentages divided by the number of years. Top 10: Shiina Mashiro, Tachibana Kanade, Takanashi Rikka, Gokō Ruri (Kuroneko), Shana, Kuroyukihime, Yūki Asuna, Aisaka Taiga, Eucliwood Hellscythe and Akiyama Mio
ii) in the green column average win percentage overall - sum of wins divided by the sum of matches. Top 10: Shiina Mashiro, Tachibana Kanade, Takanashi Rikka, Gokō Ruri (Kuroneko), Shana, Kuroyukihime, Yūki Asuna, Aisaka Taiga, Akiyama Mio and Eucliwood Hellscythe

For some characters, there are large differences between the two averages. Some characters were successful in the past (63 matches per year) and unsuccessful in the present (35 matches per year). In this case the overall average is higher than the per year average - for example Fujibayashi Kyō, Holo and Ichinose Kotomi (more than 5 percentage points difference each). Few characters were unsuccessful in the past and successful in the present. In this case the per year average is higher than the overall average - for example Misaka Mikoto (4.30 percentage points difference)

All ranking systems have advantages and disadvantages. I would like to start a discussion about rankings, alltimetables and relative tables.
Image
Gio
Ninja
Ninja
Posts: 4
Joined: Tue Oct 08, 2013 6:23 am
Melon Pan: 50

Re: Some thoughts on ranking systems

Post by Gio » Tue Dec 31, 2013 6:30 am

Looks like a good approximation about strenght but there must be a problem for younger characters, because as it is aditive, older characters will have higher values, and will take years (Litterally) to be in top ten, i.e. Shina Mashiro had a good performance in her first year and she is at place 32 ... some places down from Asahina Mikuru who has been in the competition for 5 years...

With some correction in years of participation would be a great measurement, it's still an interesting tool

Greetings
Gio
Ninja
Ninja
Posts: 4
Joined: Tue Oct 08, 2013 6:23 am
Melon Pan: 50

Re: Some thoughts on ranking systems

Post by Gio » Tue Dec 31, 2013 6:43 am

PD: sorry i didn't see the second hidden table

About overall wins ranking, there may be an adjustment in opponent strength and maybe also for certain stages in the competition

Greetings
User avatar
10ZHAbin
Spirit hunter
Spirit hunter
Posts: 2161
Joined: Tue Sep 04, 2012 9:13 am
Badges:
ImageImage
Worships: Leina
Melon Pan: 50
Location: Otaku Community

Re: Some thoughts on ranking systems

Post by 10ZHAbin » Tue Dec 31, 2013 8:20 am

This is just a personal thought, but I think it's impossible to actually rank every characters who has participated in ISML regular season fairly. The rules are changing through years with the number of matches decreasing, the number of voters has increased and decreased, the Nova and Stella division separating characters from going head to head in 1 vs 1, and those characters who couldn't exist in anime early enough are automatically at disadvantage. We can rank the characters based on their first year of participation through Total Match Wins, Total VF and Sum of Winning Match % like the following, but not all of them overall.
Total Match Won/Winning %Show
Image
Total VFShow
Image
Gio
Ninja
Ninja
Posts: 4
Joined: Tue Oct 08, 2013 6:23 am
Melon Pan: 50

Re: Some thoughts on ranking systems

Post by Gio » Tue Dec 31, 2013 9:32 am

I know it's hard, but i think it can be possible to rank them fairly if there is introduced a scored system, win over another player give them points that will depend of the strength of the opponent and differences in scores. Older players wont just win points trought the time, they can lose some too... reducing the bias caused by ammount of matches that is increased by years.

The problem with % match wins, is that isn't correlated with final positions and can not be compared with other seasons

Greetings
User avatar
Shmion84
Moon princess
Moon princess
Posts: 3383
Joined: Mon Apr 02, 2012 6:33 pm
Badges:
ImageImage
Melon Pan: 95
2017 Female Favorite: Konjiki No Yami
2017 Male Favorite: Kyon
Wish: ISML still exists next year
Location: Central Europe

Re: Some thoughts on ranking systems

Post by Shmion84 » Tue Dec 31, 2013 8:08 pm

I did not think about a ranking based on the first year of participation. That sounds interesting.

On the average win percentage per year (yellow column in my second image)
This is something different than average win percentage (total matches won divided by total matches played). A good example is the duo Saber - Sakagami Tomoyo. Both participated in every ISML Season.

Saber's record:
2008 - 43 of 63 matches won (68.25 %)
2009 - 45 of 63 matches won (71.43 %)
2010 - 41 of 49 matches won (83.67 %)
2011 - 32 of 49 matches won (65.31 %)
2012 - 30 of 35 matches won (85.71 %)
2013 - 22 of 35 matches won (62.86 %)
----------------------------------------------
Total - 213 of 294 matches won (72.45 %)
Average win percentage per year (437.23 %/6 years = 72.87%)

Sakagami Tomoyo's record:
2008 - 60 of 63 matches won (95.24 %)
2009 - 58 of 63 matches won (92.06 %)
2010 - 41 of 49 matches won (83.67 %)
2011 - 30 of 49 matches won (61.22 %)
2012 - 19 of 35 matches won (54.29 %)
2013 - 10 of 35 matches won (28.57 %)
-----------------------------------------------
Total - 218 of 294 matches won (74.15 %)
Average win percentage per year (415.06 %/6 years = 69.18%)

Both participated in 294 matches. Saber's average win percentage per year is higher (72.87% vs 69.18%), although Sakagami Tomoyo won more matches (218 vs 213).

The average win percentage per year ranking has - of course - also disadvantages. Two disadvantages have already been mentioned:
- It does not include the postseason
- Stella and Nova Divisions are separated. Shiina Mashiro won't get 100.00% score in a combined league.
Image
Gio
Ninja
Ninja
Posts: 4
Joined: Tue Oct 08, 2013 6:23 am
Melon Pan: 50

Re: Some thoughts on ranking systems

Post by Gio » Tue Dec 31, 2013 10:04 pm

I was thinking more about a general ranking that can be modified over the time, something like:

(Dx/Px)*k*(X)

where

Dx = Difference in votes
Px = Totals ammount of votes (player+opponent)
k = a constant value
X = Score opponent/Score of player

These 3 parts can fix some problems like

Dx: Isn't the same a win of 800 over 799 than a 1500 over 100, a small difference in points will give less points
k: because it's not the same a win in preeliminary rounds (With players with low strength) than a Elimination round
it can be adjusted as you wish, i was thinking about a k=10 for preliminary, K=15 for regular season and k=20 for Elimination round
X: because it's not the same i.e. that Kanade win over Ruri than a weak player, weaker players will give less points than stronger players

Also, there can be a bonus for first year of participation(idk, maybe 50% extra points, not decided yet), then new players with good performance can reach higher places easily...
i'll attach some images using data from 2009
SpoilerShow
Image
it's a group example but can work in 1vs1 matches too
SpoilerShow
Image
The good thing about scored ranking is that it can be used over the time, just modifying score variations over seasons, and can give a more accurate strength value

Greetings
User avatar
Shmion84
Moon princess
Moon princess
Posts: 3383
Joined: Mon Apr 02, 2012 6:33 pm
Badges:
ImageImage
Melon Pan: 95
2017 Female Favorite: Konjiki No Yami
2017 Male Favorite: Kyon
Wish: ISML still exists next year
Location: Central Europe

Re: Some thoughts on ranking systems

Post by Shmion84 » Tue Jan 07, 2014 6:44 pm

Gio wrote:I was thinking more about a general ranking that can be modified over the time, something like:

(Dx/Px)*k*(X)

where

Dx = Difference in votes
Px = Totals ammount of votes (player+opponent)
k = a constant value
X = Score opponent/Score of player

These 3 parts can fix some problems like

Dx: Isn't the same a win of 800 over 799 than a 1500 over 100, a small difference in points will give less points
k: because it's not the same a win in preeliminary rounds (With players with low strength) than a Elimination round
it can be adjusted as you wish, i was thinking about a k=10 for preliminary, K=15 for regular season and k=20 for Elimination round
X: because it's not the same i.e. that Kanade win over Ruri than a weak player, weaker players will give less points than stronger players
The factor X is the crucial point. Let A and B two characters, with score of A > score of B. If A wins the match, A will gain fewer points than B will loss. If B wins the match, B will gain more points than A will loss.
Since most of the favorites win their matches the average score decreases. A weak character will lose points with every defeat. So this system is deflationary and divergent.

I tested this system for the 2013 regular season without necklace-matches. K = 15 and initial score = 2000.
After Topaz 7 the ranking was as follows:
SpoilerShow
#1 Tachibana Kanade 2183
#2 Gokō Ruri (Kuroneko) 2167
#3 Takanashi Rikka 2148
...
#70 Iwasawa Masami 1900
#71 Izumi Konata 1887
#72 Hirasawa Ui 1868
average 1997
After 1st iteration:
SpoilerShow
#1 Tachibana Kanade 2349
#2 Gokō Ruri (Kuroneko) 2319
#3 Takanashi Rikka 2284
...
#70 Iwasawa Masami 1791
#71 Izumi Konata 1765
#72 Hirasawa Ui 1723
average 1989
After 2nd iteration:
SpoilerShow
#1 Tachibana Kanade 2501
#2 Gokō Ruri (Kuroneko) 2458
#3 Takanashi Rikka 2410
...
#70 Iwasawa Masami 1671
#71 Izumi Konata 1630
#72 Hirasawa Ui 1561
average 1976
So I would change the factor X.
Image
User avatar
tehyc
[Death by Embracing]
[Death by Embracing]
Posts: 2523
Joined: Wed Jan 01, 2014 1:10 pm
Badges:
ImageImageImageImageImage
Melon Pan: 36
2017 Female Favorite: Kuroyukihime
2017 Male Favorite: Sakamaki Izayoi
Wish: Stay alive

Re: Some thoughts on ranking systems

Post by tehyc » Thu Jan 16, 2014 6:47 am

My personal thought, if really want to know the overall strengths of characters, maybe can just use simple League Ranking for regular season, and use Success Ranking suggest by Shmion84 for post season.

By the way, can anyone give me the link to access to the full result of all year? I will work out with what i mean and see will it work well.
ImageImage
User avatar
maglor
~Fukou da~
~Fukou da~
Posts: 8642
Joined: Thu Feb 26, 2009 8:57 pm
Badges:
ImageImage
Worships: Abriel Nei Debrusc Borl Paryun Lafiel
Melon Pan: 75
2019 Female Favorite: Sakurajima Mai
2019 Male Favorite: Archer
2018 Female Favorite: Chtholly Nota Seniorious
2018 Male Favorite: Yang Wenli
2017 Female Favorite: Tomori Nao
2017 Male Favorite: Yang Wenli
Wish: More people being open to alternatives and compromises.

Re: Some thoughts on ranking systems

Post by maglor » Sat Mar 22, 2014 10:25 pm

I have an interesting alternative to SDO that will drive all would be manipulators crazy.

Instead of SDO, we can use SVO, which is short for Strength of Votes Opposed. ( Yes, I know I could have said SOV, but I avoided that for a reason )

Here is the methodology.

(1) On a given match day, calculate average votes received for all the contestants that had the match. That would mean calculate the average VF for all 72 contestants for that day. Using 2013's regular season, this number would be near 4000 to 4200. Remember that this number will be different per each match day. For example, 2013 Topaz 1 would have much higher average VF number. 2013 Emerald period would have lower number for average VF

(2) Divide everyone's VF for the day by that average VF number you calculated above. This means everyone should have number near 1 for that match day. We will call this VP = Vote points. If we record and sum the VP for all the matches a character was involved in, which can be called CVP = cumulative Vote Points, it will give us much better understanding of level of support a character gets compared to our current CVF, which can fluctuate greatly by outside influence.

(3) The winner of the match earns SVO = opponent's VP . This means beating an weak opponent won't let you earn much SVO. The loser of the match earns SVO = (the LOSER'S VP ) * ( The Loser's VP / the winner's vp ) = (the LOSER'S VP ) ^2 / ( The winner's VP ). So assume a match where the winner got 1.2 VP while the loser got 0.6 VP . The winner earns SVP = 0.6 for that match while the loser earns SVO = 0.3 for that match. On a tough match between Tier 1 contestants, The winner may get 1.3VP while the Loser gets 1.2 VP. In this case the winner earns SVO = 1.2 while the Loser earns SVO = 1.2^2/1.3 = 1.44/1.3 = 1.1077 . The message is obvious : More votes your opponent has, more SVO you can earn, if you win. + Even if you lose, if you can keep it close while having lots of votes yourself, you will earn good amount of SVO . Another byproduct of the whole thing is that matches between contestants with similar strength will contribute much more to CSVO = cumulative SVO compared to matches between contestants of differing strength.

(4) The necklace will be determined by following process. (a) 7 characters will be chose, first by number of wins, and then by CVP . Of course , each division gets minimum of 3 representatives. (b) I expect average difference between CSVO to be near 0.1/match * 7( = number of matches per necklace period ) ~ 1 ( rounded up to cover for uncertainties . The average difference between 1st and 2nd place finisher in necklace group match is near 10% = 0.1 . To make CSVO and the necklace match to have equal weights, we can calculate necklace score as 10*(votes received by the contestant in the necklace group match / ALL the votes casted for the necklace group match ) + CSVO .

Here are some interesting aspects of this. most of the characters entering the necklace group match will have CSVO score near 6 ( less than 7 since they will likely keep their opponent's VF number below average VF in any match ) . Winning the necklace group match will likely be similar to winning and thus earning SVO from one more match compared to the rest of the pack, which seems about right. It will be hard to accumulate large CSVO if most of your opponents are weak. Those that beat several near-tier-1-level characters will be well rewarded in CSVO

If someone wants to manipulate this CSVO, that person has to do something extremely counter-intuitive, thus very dangerous for their cause. In order to boost their favorite's SVO, they have to vote against their favorite, which will raise the risk of their favorite losing the match. In order to decrease their rival's SVO, they have to vote for the rival which will increase the chance that their rival will win. They won't likely change VP nor SVO much in matches where one contestant is likely to win huge.

So thoughts and criticism? I welcome all of you to try to calculate VP, CVP, SVO, CSVO, and necklace scores based on past matches. I suspect they will be easier to calculate compared to SDO and SAO.

- ps - SOCVP = Sum of opponents' CVP , SDOVP = Sum of defeated opponents' CVP , SOCSVO = sum of opponents' CSVO and SDOCSVO = sum of defeated opponents' CSVO should give us very good insight into quality of opponents and wins of a contestant.
Image
User avatar
Shmion84
Moon princess
Moon princess
Posts: 3383
Joined: Mon Apr 02, 2012 6:33 pm
Badges:
ImageImage
Melon Pan: 95
2017 Female Favorite: Konjiki No Yami
2017 Male Favorite: Kyon
Wish: ISML still exists next year
Location: Central Europe

Re: Some thoughts on ranking systems

Post by Shmion84 » Sun Mar 23, 2014 12:31 am

It is an interesting idea, particularly the SVO being independent from older matches.

I made some calculations for 2013 Season:
CSVO for 2013 Regular SeasonShow
Image
So the average SVO per match and character is about 0.70 to 0.71. Note: Tier I characters have low CSVO (Tachibana Kanade ranked #26 in Stella Division and Shiina Mashiro ranked #25 in Nova Division). Tier II characters have high CSVO.
CSVO for 2013 Aquamarine PeriodShow
Image
Necklace contenders in bold text. Tachibana Kanade would have somehow managed to win the Necklace under the proposed rules.
CSVO for 2013 Amethyst PeriodShow
Image
And the winner is...
Shana
Thanks to her high CSVO.

So I would change the factor 10 of your Necklace formula

Code: Select all

10*(votes received by the contestant in the necklace group match / ALL the votes casted for the necklace group match ) + CSVO 
Aside from these differences I noticed following - please correct me if I am wrong:

Match between A and B. Votes of A > 2 * Votes of B.
An additional vote for B would increase the SVO of A more than the SVO of B.

Example:
A's VP = 1.4
B's VP = 0.6
Therefore
old SVO of A = B's VP = 0.6
old SVO of B = (B's VP)^2/(A's VP) = 0.36/1.4 = 0.2571...

Additional voters increase B's VP to 0.61
Therefore
new SVO of A = B's VP = 0.61
new SVO of B = (B's VP)^2/(A's VP) = 0.3721/1.4 = 0.2658...

Increase of SVO of A = 0.61 - 0.6 = 0.01
Increase of SVO of B = 0.2658 - 0.2571 = 0.0097

Therefore Increase of SVO of A > Increase of SVO of B!
Image
User avatar
Midnight-Jasper
Intelligent device
Intelligent device
Posts: 1357
Joined: Sat Dec 31, 2011 9:47 pm
Badges:
Image
Melon Pan: 50
Wish: viva all, baby~
Location: Aloha Oe

Re: Some thoughts on ranking systems

Post by Midnight-Jasper » Sun Mar 23, 2014 1:34 am

Looking at what Shmion84 has provided I feel that this system doesn't reward winning enough compared to losing a match closely.
Especially those necklace matches where those who went 6-1 had a pronounced lead... (although I don't really understand why that is >< Maybe just a quirk of those two periods, something to do with the being T2?)
Seeing Kanade dwelling near the bottom of that Amethyst period feels quite unfair.
Image
User avatar
10ZHAbin
Spirit hunter
Spirit hunter
Posts: 2161
Joined: Tue Sep 04, 2012 9:13 am
Badges:
ImageImage
Worships: Leina
Melon Pan: 50
Location: Otaku Community

Re: Some thoughts on ranking systems

Post by 10ZHAbin » Sun Mar 23, 2014 1:53 am

Simply using the SVO feels like a heavy punishment for stronger characters outblowing weaker characters. While this factor does exist in SDO, at least it's not too significant with 49 matches in total per period to determine the final outcome of SDO and only 7 of those matches, the character herself would take part. Perhaps using the SDOVP will be a better idea?
User avatar
Midnight-Jasper
Intelligent device
Intelligent device
Posts: 1357
Joined: Sat Dec 31, 2011 9:47 pm
Badges:
Image
Melon Pan: 50
Wish: viva all, baby~
Location: Aloha Oe

Re: Some thoughts on ranking systems

Post by Midnight-Jasper » Sun Mar 23, 2014 2:50 am

10ZHAbin wrote:Simply using the SVO feels like a heavy punishment for stronger characters outblowing weaker characters. While this factor does exist in SDO, at least it's not too significant with 49 matches in total per period to determine the final outcome of SDO and only 7 of those matches, the character herself would take part. Perhaps using the SDOVP will be a better idea?
Well that would lend itself to manipulation like SDO, maybe more so because the character doesn't need to win to get a significant boost.
Image
User avatar
maglor
~Fukou da~
~Fukou da~
Posts: 8642
Joined: Thu Feb 26, 2009 8:57 pm
Badges:
ImageImage
Worships: Abriel Nei Debrusc Borl Paryun Lafiel
Melon Pan: 75
2019 Female Favorite: Sakurajima Mai
2019 Male Favorite: Archer
2018 Female Favorite: Chtholly Nota Seniorious
2018 Male Favorite: Yang Wenli
2017 Female Favorite: Tomori Nao
2017 Male Favorite: Yang Wenli
Wish: More people being open to alternatives and compromises.

Re: Some thoughts on ranking systems

Post by maglor » Sun Mar 23, 2014 3:45 am

Thank you for your analysis, Shmion84 . I do think the formula needs to be modified.

(1) VP idea stands. It is character's vote/average votes for that particular match day. Please remember that it is averaged within each match day to account for the fact that one day can have average of 3000 while the next round may have average of 5000

(2) The winner gets NP=Necklace point = Own VP + Opponent's VP = ( Total number of vote in the match ) / Average votes per character . The loser gets NP = Own VP . What this does is that for the winner, what is important is the amount of the interest that match attracted. For the loser, all it matters is how many votes she got. The final NP = sum of NP from 7 rounds + 10* (votes received by a character ) / ( total votes in the necklace group match ) .

(3) This truly makes the optimal strategy simply vote for the character you like. The BIG problem is, this will drastically increase the level of influence a consistent multivoter will have even if he controls only about 1% of the total votes .

(4) The formula also is simpler since there is no non-linear terms other than W-L.

(5) This will also mean there will be DIS-incentive to voting for matches not involving your character. A faction will tell its supporters to deliberately vote only in minimum number of matches, even if this means not voting for some of the characters a member likes.


----- Thinking further, (3) and (5) simply kills this NP formula

Another alternative is to use SVDO = sum of CVP of all opponents defeated. The calculation of this is more complicated. In many ways, SVDO is a better measurement of combined strength of defeated opponents, since it decreases penalty in NP for a defeated character losing a close match. Again, the problem is, this will be much easier to manipulate compared to current SDO or the SVO scheme above for a consistent multivoter who controls just 1% of the total votes.

------------PS 2

This CSVO scheme's biggest goal was to totally discourage multivoter/faction's attempt to manipulate Necklace score. It seems to raise other problems, sadly.
Image
User avatar
10ZHAbin
Spirit hunter
Spirit hunter
Posts: 2161
Joined: Tue Sep 04, 2012 9:13 am
Badges:
ImageImage
Worships: Leina
Melon Pan: 50
Location: Otaku Community

Re: Some thoughts on ranking systems

Post by 10ZHAbin » Sun Mar 23, 2014 4:06 am

Midnight-Jasper wrote:
10ZHAbin wrote:Simply using the SVO feels like a heavy punishment for stronger characters outblowing weaker characters. While this factor does exist in SDO, at least it's not too significant with 49 matches in total per period to determine the final outcome of SDO and only 7 of those matches, the character herself would take part. Perhaps using the SDOVP will be a better idea?
Well that would lend itself to manipulation like SDO, maybe more so because the character doesn't need to win to get a significant boost.
The first point, true. But character don't actually get much significant boost. I chose Topaz 2013 because that's where SDO manipulation was at its best. And here are roughly the character's SDOVP:
Shiina Mashiro: 50.555
Aisaka Taiga: 48.141
Eucliwood Hellscythe: 47.854
Misaka Mikoto: 47.320
Kousaka Kirino: 41.784
Hasegawa Kobato: 38.785
Kuroyukihime: 35.815

While Mashiro still managed to get on the good side of necklace match through manipulation, the difference in advantage between her and Mikoto greatly decreased from 47.527 to 3.235. Hime still hold onto the lowest, which is reasonable considering her Topaz match up was very weak overall and she lost to Asuna which made her 6-1 at disadvantage. SDOVP can't stop manipulation towards higher SDOVP, but there is not much effect when it comes to anti-SDOVP of a certain character, and the effect in boost is a lot less significant.

All of that aside, whatever the staff might choose to go with, I am suggesting putting 3 possible variables into account for necklace. The first two are obvious, the strength of opponent defeated (whatever ways it will be measured) and the necklace score. The third one is the strength of the character herself, which is CVP. No matter how the strength of opponent will be measured, the one fact that won't be changed is that in one head on match, increasing CVP means decreasing the strength shown by the opponent. In the current method there is not much of a point for this since most of the characters in Necklace Match will have pts of 21.000, however with CVP, those characters that has shown to outblow their opponents harder will receive a boost to replace the lost through destroying their oppopents.
User avatar
maglor
~Fukou da~
~Fukou da~
Posts: 8642
Joined: Thu Feb 26, 2009 8:57 pm
Badges:
ImageImage
Worships: Abriel Nei Debrusc Borl Paryun Lafiel
Melon Pan: 75
2019 Female Favorite: Sakurajima Mai
2019 Male Favorite: Archer
2018 Female Favorite: Chtholly Nota Seniorious
2018 Male Favorite: Yang Wenli
2017 Female Favorite: Tomori Nao
2017 Male Favorite: Yang Wenli
Wish: More people being open to alternatives and compromises.

Re: Some thoughts on ranking systems

Post by maglor » Sun Mar 23, 2014 4:25 am

10ZHAbin wrote:
Midnight-Jasper wrote:
10ZHAbin wrote:Simply using the SVO feels like a heavy punishment for stronger characters outblowing weaker characters. While this factor does exist in SDO, at least it's not too significant with 49 matches in total per period to determine the final outcome of SDO and only 7 of those matches, the character herself would take part. Perhaps using the SDOVP will be a better idea?
Well that would lend itself to manipulation like SDO, maybe more so because the character doesn't need to win to get a significant boost.
The first point, true. But character don't actually get much significant boost. I chose Topaz 2013 because that's where SDO manipulation was at its best. And here are roughly the character's SDOVP:
Shiina Mashiro: 50.555
Aisaka Taiga: 48.141
Eucliwood Hellscythe: 47.854
Misaka Mikoto: 47.320
Kousaka Kirino: 41.784
Hasegawa Kobato: 38.785
Kuroyukihime: 35.815

While Mashiro still managed to get on the good side of necklace match through manipulation, the difference in advantage between her and Mikoto greatly decreased from 47.527 to 3.235. Hime still hold onto the lowest, which is reasonable considering her Topaz match up was very weak overall and she lost to Asuna which made her 6-1 at disadvantage. SDOVP can't stop manipulation towards higher SDOVP, but there is not much effect when it comes to anti-SDOVP of a certain character, and the effect in boost is a lot less significant.

All of that aside, whatever the staff might choose to go with, I am suggesting putting 3 possible variables into account for necklace. The first two are obvious, the strength of opponent defeated (whatever ways it will be measured) and the necklace score. The third one is the strength of the character herself, which is CVP. No matter how the strength of opponent will be measured, the one fact that won't be changed is that in one head on match, increasing CVP means decreasing the strength shown by the opponent. In the current method there is not much of a point for this since most of the characters in Necklace Match will have pts of 21.000, however with CVP, those characters that has shown to outblow their opponents harder will receive a boost to replace the lost through destroying their oppopents.
Assuming this will be a typical number during necklace period, I am noting that average difference between adjacent ranks in SDOVP ( = SVDO )is (50.555 - 35.8150 / 6 = 2.457. I also see that average difference between adjacent ranks in necklace group match is 0.0350 . If we multiply necklace group match VF% ( note I am using 0 to 1 scale here, not 0 to 100 ) by 70, then the necklace group match should have similar weight to SDOVP . Using the multiplier of 70, in order for Kuroyukihime to win the necklace, she has to beat Mashiro by (50.555- 35.815 ) / 70 = 0.2106 or more. Hmm, 21% is unrealistic. Perhaps we should do the simple SDOVP + 100* necklace group VF%. That means Kuroyukihime needs to beat Mashiro by 15% which is doable.
Image
User avatar
10ZHAbin
Spirit hunter
Spirit hunter
Posts: 2161
Joined: Tue Sep 04, 2012 9:13 am
Badges:
ImageImage
Worships: Leina
Melon Pan: 50
Location: Otaku Community

Re: Some thoughts on ranking systems

Post by 10ZHAbin » Sun Mar 23, 2014 4:53 am

This is just my opinion, but to me, if a character only has defeated weak characters within a certain period, I don't think that certain character deserve to be the winner of that period even if she is the strongest out of all 72. Which is probably why there is a good need for a right seeding in tiers so there won't be a big gap between one charatcer's opponent strength to another.
User avatar
maglor
~Fukou da~
~Fukou da~
Posts: 8642
Joined: Thu Feb 26, 2009 8:57 pm
Badges:
ImageImage
Worships: Abriel Nei Debrusc Borl Paryun Lafiel
Melon Pan: 75
2019 Female Favorite: Sakurajima Mai
2019 Male Favorite: Archer
2018 Female Favorite: Chtholly Nota Seniorious
2018 Male Favorite: Yang Wenli
2017 Female Favorite: Tomori Nao
2017 Male Favorite: Yang Wenli
Wish: More people being open to alternatives and compromises.

Re: Some thoughts on ranking systems

Post by maglor » Sun Mar 23, 2014 2:42 pm

OK, folks. I need some help. Here are the stats needed to advance or squash this.

For 2011, 2012, and 2013 Aquamarine, please tell me

(1) CVP
(2) SVDO = SDOVP = Sum of Vote Points of the defeated Opponents
(3) Necklace Score = SVDO + 50 * NGP ( = Necklace Group performance = ( votes received by the contestant in the 7 character necklace group match ) / ( total number of votes cast for the necklace group match) . Since using 100 as multiplier has chance to do too much, I thought 50 would be more appropriate.

What we need to see is
(1) SVDO for 7 win characters be higher than those for 6 win characters in almost all the cases in these 3 aquamarine periods.
(2) Final Necklace Score ranking be different from SVDO ranking in at least one place in at least two out of three years.
(3) Final Necklace Score ranking using SVDO be different from Necklace Score Ranking we used for the year in question in at least one place in at least two out of three years.

Please note that what I mean by ranking is the ranking of all 7 who entered necklace group match. It would be interesting to see if any of the 7 would have been different if necklace group selection was (1) number of wins (2) CVP .

The reasons are

(1) Any scoring system that doesn't adequately reward wins won't likely be accepted by the public.
(2) Necklace Group match should be capable of making some difference. We need to see whether it is too dominant a factor, or too small a factor
(3) If the final result isn't that different from current results, there isn't much of reason to change anything.
Image
User avatar
Shmion84
Moon princess
Moon princess
Posts: 3383
Joined: Mon Apr 02, 2012 6:33 pm
Badges:
ImageImage
Melon Pan: 95
2017 Female Favorite: Konjiki No Yami
2017 Male Favorite: Kyon
Wish: ISML still exists next year
Location: Central Europe

Re: Some thoughts on ranking systems

Post by Shmion84 » Sun Mar 23, 2014 6:15 pm

maglor wrote:OK, folks. I need some help. Here are the stats needed to advance or squash this.

For 2011, 2012, and 2013 Aquamarine, please tell me

(1) CVP
(2) SVDO = SDOVP = Sum of Vote Points of the defeated Opponents
(3) Necklace Score = SVDO + 50 * NGP ( = Necklace Group performance = ( votes received by the contestant in the 7 character necklace group match ) / ( total number of votes cast for the necklace group match) . Since using 100 as multiplier has chance to do too much, I thought 50 would be more appropriate.
The maximum was marked in bold text.

Image
maglor wrote: What we need to see is
(1) SVDO for 7 win characters be higher than those for 6 win characters in almost all the cases in these 3 aquamarine periods.
Yes. SVDO for 7 win characters is pretty constant (between 45 and 50 points).
maglor wrote:(2) Final Necklace Score ranking be different from SVDO ranking in at least one place in at least two out of three years.
Yes. For example: only in 2011 the character with the highest SVDO got the highest Final Necklace Score.
maglor wrote:(3) Final Necklace Score ranking using SVDO be different from Necklace Score Ranking we used for the year in question in at least one place in at least two out of three years.
Yes. For example: In 2011 Akiyama Mio gained 3rd rank under old rules and 5th rank under proposed rules. In 2012 Shana gained 5th rank under old rules and 4th rank under proposed rules. In 2013 Kuroyukihime gained 3rd rank under old rules and 6th rank under proposed rules.
maglor wrote:It would be interesting to see if any of the 7 would have been different if necklace group selection was (1) number of wins (2) CVP .
In 2011 Nakano Azusa (CVP 8.87895438) and Gokō Ruri (Kuroneko) (CVP 8.705217814) would have replaced Nakamura Yuri and Eucliwood Hellscythe.
In 2012 no changes (exact 7 characters with 7 wins).
In 2013 Yūki Asuna (CVP 9.154928839) would have replaced Kōsaka Kirino.

You might double check these calculations.
Image
Post Reply