Necklace formula discussion

For topics that last throughout the whole season
Locked
User avatar
Shmion84
Moon princess
Moon princess
Posts: 3383
Joined: Mon Apr 02, 2012 6:33 pm
Badges:
ImageImage
Melon Pan: 95
2017 Female Favorite: Konjiki No Yami
2017 Male Favorite: Kyon
Wish: ISML still exists next year
Location: Central Europe

Necklace formula discussion

Post by Shmion84 » Sat Oct 17, 2015 9:35 am

After a Regular Season with the new PSAO formula it's time to discuss the pros and contras of this formula and discuss some alternatives. The alternatives may include small changes (for example Necklace Vote Percentage + PSAO/4) or completely new formulas.

Please keep in mind: This topic won't decide the necklace formula for 2016. However, this topic shall collect ideas and arguments for the final decision by the ISML Staff.
Image
User avatar
avery-kun
Moon princess
Moon princess
Posts: 3128
Joined: Thu Oct 04, 2012 4:25 am
Badges:
ImageImageImage
Worships: Berserker!
Melon Pan: 135
2017 Female Favorite: Illyasviel Von Einzbern
2017 Male Favorite: Gilgamesh
Wish: Illya 2017!

Re: Necklace formula discussion

Post by avery-kun » Mon Oct 19, 2015 4:40 pm

Well. I'll just throw out wacky ideas and see what I can come up with. "Brainstorming"

N="some number"

1. (VP of All Opponents) / N + VF%
2. (VP of Defeated Opponents) / N + VF%
3. VP / N + VF%
4. VF%
5. Changing voting to Instant Runoff (or something else)
6. SDO / N + VF%
7. PSAO / N + VF%
8. SAO + VF%
9. VD / (Avg Division VD) + VF%
10. (Avg VF%) / N + VF%
11. (Avg VF% of Opponents) / N + VF%
12. (Avg VF% of Defeated Opponents) / N + VF%
13. VF% * N + SDO (or PSAO or SAO or etc...)
14. [(Avg VF% of Opponents) * (Avg VF% of Opponents' Opponents) * (Avg VF%)] / N + VF%
15. [(Avg VP of Opponents) * (Avg VP of Opponents' Opponents) * (Avg VP)] / N + VF%

PSAO

Pros:
1. 6-1 Contestants have a shot at winning.
2. PSAO attempts to provide an gauge of how well a contestant performed in a period relative to her opponents. So a necklace contestant with stronger opponents should have an advantage.

Cons:
1. 6-1 Contestants have a shot at winning.
2. PSAO/SDO may require staff to do considerable work in order to "balance" the schedule. Bias at this point can lead to "corruption" of schedule and results.
3. Each successive period makes manipulation of PSAO easier.
4. Contestants' strength is neither constant nor perfectly predictable after prelims. Lower tier contestants that are stronger than expected in regularseason provides a natural advantage to whichever 'captain' they just happened to be assigned in a period. This gives some 'captains' an advantage that they have no control over.
5. PSAO/SDO awards 'winners' a flat score, regardless of how 'impressive' their victory was. However, loses are treated based on how 'impressive' their performance was. Thus all victories are treated as equal when that is not the case. (i.e. Shana (rank 16) defeating Mashiro(rank 1) is more impressive than Kotori (rank 4) beating Illya (rank 5), but PSAO won't reflect this).
Image
User avatar
maglor
~Fukou da~
~Fukou da~
Posts: 8694
Joined: Thu Feb 26, 2009 8:57 pm
Badges:
ImageImage
Worships: Abriel Nei Debrusc Borl Paryun Lafiel
Melon Pan: 75
2019 Female Favorite: Sakurajima Mai
2019 Male Favorite: Archer
2018 Female Favorite: Chtholly Nota Seniorious
2018 Male Favorite: Yang Wenli
2017 Female Favorite: Tomori Nao
2017 Male Favorite: Yang Wenli
Wish: More people being open to alternatives and compromises.

Re: Necklace formula discussion

Post by maglor » Tue Oct 20, 2015 3:45 am

avery-kun wrote: 5. PSAO/SDO awards 'winners' a flat score, regardless of how 'impressive' their victory was. However, loses are treated based on how 'impressive' their performance was. Thus all victories are treated as equal when that is not the case. (i.e. Shana (rank 16) defeating Mashiro(rank 1) is more impressive than Kotori (rank 4) beating Illya (rank 5), but PSAO won't reflect this).
This isn't true. Just meeting a strong opponent improves SAO portion of PSAO , but if you don't win, the P part of PSAO will hurt very much. It also is purely opponent based score, thus it isn't suppose to differentiate narrow difference in the rank of the opponent ( rank 1 vs rank 5) as in the given example. If the contestant's starting rank is also considered, you can have unstable case where rank 30 character may be two characters in top 10 and achieve almost insurmountable score merely with these two victories. You must look at PSAO as a total period performance, not a single round performance. This was by design as we wanted to have something that won't depend on only one or two round, as it is much easier and tempting to manipulate just one or two matches instead of entire period worth of matches for multiple contestants in the division.
Image
User avatar
Shmion84
Moon princess
Moon princess
Posts: 3383
Joined: Mon Apr 02, 2012 6:33 pm
Badges:
ImageImage
Melon Pan: 95
2017 Female Favorite: Konjiki No Yami
2017 Male Favorite: Kyon
Wish: ISML still exists next year
Location: Central Europe

Re: Necklace formula discussion

Post by Shmion84 » Tue Oct 20, 2015 7:43 pm

avery-kun wrote:Well. I'll just throw out wacky ideas and see what I can come up with. "Brainstorming"

N="some number"
15 optionsShow
1. (VP of All Opponents) / N + VF%
2. (VP of Defeated Opponents) / N + VF%
3. VP / N + VF%
4. VF%
5. Changing voting to Instant Runoff (or something else)
6. SDO / N + VF%
7. PSAO / N + VF%
8. SAO + VF%
9. VD / (Avg Division VD) + VF%
10. (Avg VF%) / N + VF%
11. (Avg VF% of Opponents) / N + VF%
12. (Avg VF% of Defeated Opponents) / N + VF%
13. VF% * N + SDO (or PSAO or SAO or etc...)
14. [(Avg VF% of Opponents) * (Avg VF% of Opponents' Opponents) * (Avg VF%)] / N + VF%
15. [(Avg VP of Opponents) * (Avg VP of Opponents' Opponents) * (Avg VP)] / N + VF%
Thanks for your input. So VF% is the Necklace Vote Percentage.
I think we can classify these ideas in 4 groups:
(A) Necklace Score is only decided by the Necklace Match (4 and 5) - of course you need some tiebreakers to find the 7 competitors, but that's not the question at the moment
(B) Necklace Score is decided by the Necklace Match and the competitor's matches (10, also 3 and 9)
(C) Necklace Score is decided by the Necklace Match, the competitor's matches and the competitor's opponents' matches (1, 2, 6, 7, 8, 11, 12, 13) - 2, 6 and 12 only consider only defeated opponents' matches
(D) Necklace Score is decided by the Necklace Match, the competitor's matches, the competitor's opponents' matches and the competitor's opponents' opponents' matches (14, 15) - You can also add Rating Percentage Index (viewtopic.php?f=30&t=4506&start=60#p248170" onclick="window.open(this.href);return false; and following posts) to this group.

Regarding potential manipulation (tactical voting or one time mass voting in the Necklace Match) I assume the difficulty increases from group A to D. However, the number of affected matches (and competitors!) increases from group A to D. Let's try to solve this Gordian Knot.
Image
User avatar
maglor
~Fukou da~
~Fukou da~
Posts: 8694
Joined: Thu Feb 26, 2009 8:57 pm
Badges:
ImageImage
Worships: Abriel Nei Debrusc Borl Paryun Lafiel
Melon Pan: 75
2019 Female Favorite: Sakurajima Mai
2019 Male Favorite: Archer
2018 Female Favorite: Chtholly Nota Seniorious
2018 Male Favorite: Yang Wenli
2017 Female Favorite: Tomori Nao
2017 Male Favorite: Yang Wenli
Wish: More people being open to alternatives and compromises.

Re: Necklace formula discussion

Post by maglor » Tue Oct 20, 2015 10:01 pm

Shmion84 wrote:
Regarding potential manipulation (tactical voting or one time mass voting in the Necklace Match) I assume the difficulty increases from group A to D. However, the number of affected matches (and competitors!) increases from group A to D. Let's try to solve this Gordian Knot.
System alone can't solve this Gordian knot, as another important factor is rate of increase in difficulty. If we have great variety of voters, leaving none of the factions controlling more than 10% of the votes, increasing the difficulties is the better solution. But, if we lack variety, having 20+% of the voters following certain one or two simple priorities, then one or two smart leader of the faction can easily surmount the increase in difficulty, thus it may become necessary to reduce the number of matches they would care about. Also important is the characteristics of the faction as some factions like Kanade faction tends to minimize meddling into matches not of large consequences for their favorite character, while another faction like Mikoto or Asuna faction tends to meddle in all matches they can for whatever small gain available for their favorites. Another characteristic is the gracefulness of accepting defeat as more graceful factions, for the sake of long term benefit, will try to not anger other factions by meddling, while more short sighted faction will do everything they can, right now, without any regards to how others might feel. All these needs to be considered when we try to solve this Gordian knot.

Until 2014, ISML was blessed with several factions taking more graceful approach to the whole thing instead of short-sighted approach. This no longer is the case. We need to wonder how things will be like next year, in order to make good decision about the formula.
Image
User avatar
avery-kun
Moon princess
Moon princess
Posts: 3128
Joined: Thu Oct 04, 2012 4:25 am
Badges:
ImageImageImage
Worships: Berserker!
Melon Pan: 135
2017 Female Favorite: Illyasviel Von Einzbern
2017 Male Favorite: Gilgamesh
Wish: Illya 2017!

Re: Necklace formula discussion

Post by avery-kun » Wed Oct 21, 2015 1:50 am

maglor wrote: Until 2014, ISML was blessed with several factions taking more graceful approach to the whole thing instead of short-sighted approach. This no longer is the case. We need to wonder how things will be like next year, in order to make good decision about the formula.
What was the conclusion of the PSAO analysis? Was there any heavy handed tactics to help/hurt certain contestants?
Image
User avatar
Sphire
Space pirate
Space pirate
Posts: 387
Joined: Fri Mar 09, 2012 6:09 am
Melon Pan: 50
Wish: Love & Peace

Re: Necklace formula discussion

Post by Sphire » Mon Oct 26, 2015 4:56 pm

I'd prefer if characters going 7-0 actually meant something over manipulation of PSAO. As it is, there's very little downside to PSAO manipulation, because even if your character gets sniped at the end, well, the overall points gained is still higher.

You could limit the necklace match to 5 characters, limiting 6-1 characters, but I believe there was some logistical point to having 7 characters, so unlikely. Also, less participants probably means bigger percentages for top characters.

You could penalise 6-1 characters for losing, but they already lose PSAO points for not going 7-0.

You could reward characters that go 7-0 with bonus PSAO points. Say an extra 9 PSAO points, which in turn would provide them with an extra 3% buffer in necklace match result (against 6-1 characters only). Not too big (if I calculated right).

The necklace match formula is probably fine as it is, so is the usage of PSAO (not that I get the nitty gritty of the calculation, just the general idea). I just don't think there's enough reward for a character doing well in a given period over underling manipulation and necklace match voting. I'm not too sure about double votes for the necklace match either, especially when you get jarring results where the vast majority of votes go to 2/3 characters over the rest. But it's not too bad.

~2cents
Bloodborne AwesomeShow
User avatar
maglor
~Fukou da~
~Fukou da~
Posts: 8694
Joined: Thu Feb 26, 2009 8:57 pm
Badges:
ImageImage
Worships: Abriel Nei Debrusc Borl Paryun Lafiel
Melon Pan: 75
2019 Female Favorite: Sakurajima Mai
2019 Male Favorite: Archer
2018 Female Favorite: Chtholly Nota Seniorious
2018 Male Favorite: Yang Wenli
2017 Female Favorite: Tomori Nao
2017 Male Favorite: Yang Wenli
Wish: More people being open to alternatives and compromises.

Re: Necklace formula discussion

Post by maglor » Tue Oct 27, 2015 12:24 am

avery-kun wrote:
maglor wrote: Until 2014, ISML was blessed with several factions taking more graceful approach to the whole thing instead of short-sighted approach. This no longer is the case. We need to wonder how things will be like next year, in order to make good decision about the formula.
What was the conclusion of the PSAO analysis? Was there any heavy handed tactics to help/hurt certain contestants?
There definitely was multivoting attempt to manipulate PSAO that the analysis helped me find. There may have been some factional attempts to manipulate PSAO , but often it backfired on them as a character that received some boost for two or three rounds ended up losing a winnable 6th or 7th round match. Still, Topaz period and perhaps Emerald period's did see some boosts and anti-voting that may had some effect.

One problem in Topaz period analysis was that there was more serious attempt to boost some characters' SS number in order for them to have high seeding for the playoffs. This attempt affected large number of characters and thus made it harder to discern whether PSAO boost was for the necklace or just an accident. I think in end, there were heavy handed tactics carried out by some factions, most of the attempts via multivoting failed, and legit voting attempts ended up having some effect, but its effect size is likely much smaller than many people suspected, mainly because there were so many other effects in play.
Image
User avatar
avery-kun
Moon princess
Moon princess
Posts: 3128
Joined: Thu Oct 04, 2012 4:25 am
Badges:
ImageImageImage
Worships: Berserker!
Melon Pan: 135
2017 Female Favorite: Illyasviel Von Einzbern
2017 Male Favorite: Gilgamesh
Wish: Illya 2017!

Re: Necklace formula discussion

Post by avery-kun » Tue Oct 27, 2015 2:38 am

maglor wrote:
avery-kun wrote:
maglor wrote: Until 2014, ISML was blessed with several factions taking more graceful approach to the whole thing instead of short-sighted approach. This no longer is the case. We need to wonder how things will be like next year, in order to make good decision about the formula.
What was the conclusion of the PSAO analysis? Was there any heavy handed tactics to help/hurt certain contestants?
There definitely was multivoting attempt to manipulate PSAO that the analysis helped me find. There may have been some factional attempts to manipulate PSAO , but often it backfired on them as a character that received some boost for two or three rounds ended up losing a winnable 6th or 7th round match. Still, Topaz period and perhaps Emerald period's did see some boosts and anti-voting that may had some effect.

One problem in Topaz period analysis was that there was more serious attempt to boost some characters' SS number in order for them to have high seeding for the playoffs. This attempt affected large number of characters and thus made it harder to discern whether PSAO boost was for the necklace or just an accident. I think in end, there were heavy handed tactics carried out by some factions, most of the attempts via multivoting failed, and legit voting attempts ended up having some effect, but its effect size is likely much smaller than many people suspected, mainly because there were so many other effects in play.
God. I still can't wrap my head around the amount of effort and thought goes into vote checking. Nor the amount of effort that some people/groups put into trying to manipulate the results across the season.

So how does one render these efforts mute for the necklace?
Is that even the goal?

If math/technology/magic/etc was not an issue, how would one be able to keep the competition 'honest'?
Image
User avatar
exec
~Senior Member~
~Senior Member~
Posts: 1731
Joined: Mon May 11, 2009 5:15 pm
Badges:
Melon Pan: 15
2018 Female Favorite: Holo
2018 Male Favorite: Astolfo
2017 Female Favorite: Holo
2017 Male Favorite: Felix Argyle
Contact:

Re: Necklace formula discussion

Post by exec » Tue Oct 27, 2015 4:37 am

That would be an authentication probably, like on government elections, at the voting site they ask you your passport and make you sign the paper which say that you actually been to the voting sight. You could use other auth sites like google/facebook/yahoo/etc to do it for you, because in our case we don't care about the ID as much as we are about single vote per user, so as soon as someone votes you record their vote but forget their autherisation, but you still remember someone with that ID already voted so they can't vote again even from another IP.
Though this method has a problem and that is that people don't trust their personal info to someone even if you tell them that autherisation is made with the original site that they are registered at and you delete the info as soon as it's not needed anymore.
But at the same time this one has more secure ways of telling if someone is multivoting or not and preventing it.
On other hand people like to stay anonymous. Even if you say that you are not going to use any private info anywhere.
Image
User avatar
matchbaby
Necromancer
Necromancer
Posts: 887
Joined: Sat Jun 28, 2014 9:51 am
Badges:
ImageImage
Worships: Isla
Melon Pan: 65
Wish: Isla!!! All hail for Isla!!!
Shiro and Sora!! As least win something!!
Same as Illya~!
Location: Hong Kong, China

Re: Necklace formula discussion

Post by matchbaby » Tue Oct 27, 2015 10:01 am

I also give my idea

1. (Highest SS throughout the period * PSAO) / 100 + NGF% * k, k=[2,4]
[limit the PSAO manipulation]

2. For the 7 highest SDO chacters, they become the NM contesters, they will receive k=6 4 2 0 -2 -4 -6 base point according to the combine ranking: Sort characters' ranking in period as the period standing table (1.pts-> 2.SDO) , NM score = k + NGF%
[7-0 characters may lose the seat against 6-1 if they have a lower SDO, but SDO will not be one part of NM score so manipulation can be limited, also notice that SDO is used instead of PSAO]

3. NM score = Highest SS in period / 100 * NGF%
[However, SS is affected mainly by the schedule, not fair to character]

4. New PSAO
nPSAO = pts^2 / 441 * (2 strongest opponent's pts, without upsets, a 7-0 character's opp pts is 18.X and 16.X; a 6-1 character's opp pts is 21 and 16.X )
NM score = nPSAO /k + NGF%, k=[0.5,2]
[manipulate 2 underling is enough, multivoters]

5. NM score = NGF% /21 - total VA% in period / pts
[Multivoters! Just battle for each other! ]
Definition: VA% = VA in period / sum of 7 characters in period.

---- Actually I love 5 cuz anti-votes will be crazy and I love to see this hahaha. ----
shadowhunter
~Shadow Crafter~
~Shadow Crafter~
Posts: 929
Joined: Wed Oct 21, 2015 4:41 pm
Badges:
Melon Pan: 63

Re: Necklace formula discussion

Post by shadowhunter » Tue Oct 27, 2015 6:39 pm

2 ideas from me:
1. As it mentioned before, formula could be a Necklace Vote Percentage + PSAO/5. Maybe it will compensate some manipulation of PSAO.
2. This is bit lame idea, but anyway. At first, 7 NM participants get score from 1 to 7 according to their PSAO. Then, after voting, they get from 1 to 7 points according to the number of votes. But, there are many cons with that ranking.
User avatar
maglor
~Fukou da~
~Fukou da~
Posts: 8694
Joined: Thu Feb 26, 2009 8:57 pm
Badges:
ImageImage
Worships: Abriel Nei Debrusc Borl Paryun Lafiel
Melon Pan: 75
2019 Female Favorite: Sakurajima Mai
2019 Male Favorite: Archer
2018 Female Favorite: Chtholly Nota Seniorious
2018 Male Favorite: Yang Wenli
2017 Female Favorite: Tomori Nao
2017 Male Favorite: Yang Wenli
Wish: More people being open to alternatives and compromises.

Re: Necklace formula discussion

Post by maglor » Tue Oct 27, 2015 7:33 pm

Thank you all for ideas so far. Many do deserve more thoughts. I would also like to encourage everyone to take a look at other posters' ideas and give some comment. In addition to the methods itself, I would appreciate it if I can get a feel for how some people might react to any one of these ideas.
Image
User avatar
avery-kun
Moon princess
Moon princess
Posts: 3128
Joined: Thu Oct 04, 2012 4:25 am
Badges:
ImageImageImage
Worships: Berserker!
Melon Pan: 135
2017 Female Favorite: Illyasviel Von Einzbern
2017 Male Favorite: Gilgamesh
Wish: Illya 2017!

Re: Necklace formula discussion

Post by avery-kun » Tue Oct 27, 2015 8:43 pm

matchbaby wrote:I also give my idea

1. (Highest SS throughout the period * PSAO) / 100 + NGF% * k, k=[2,4]
[limit the PSAO manipulation]
1. Sounds like it would intensify situations where a weaker character upsets the front runner (i.e. when Chiyo beat Yukino in Topaz 7 and shot her SS through the roof), their SS*PSAO will shoot through the roof. I like it.
Image
User avatar
matchbaby
Necromancer
Necromancer
Posts: 887
Joined: Sat Jun 28, 2014 9:51 am
Badges:
ImageImage
Worships: Isla
Melon Pan: 65
Wish: Isla!!! All hail for Isla!!!
Shiro and Sora!! As least win something!!
Same as Illya~!
Location: Hong Kong, China

Re: Necklace formula discussion

Post by matchbaby » Wed Oct 28, 2015 2:49 am

avery-kun wrote:
matchbaby wrote:I also give my idea

1. (Highest SS throughout the period * PSAO) / 100 + NGF% * k, k=[2,4]
[limit the PSAO manipulation]
1. Sounds like it would intensify situations where a weaker character upsets the front runner (i.e. when Chiyo beat Yukino in Topaz 7 and shot her SS through the roof), their SS*PSAO will shoot through the roof. I like it.
Period SS Not all time SS
Period SS is 1-7%
Locked