Difficutly / Time Voting System

[Rebourne07]

As a trophy guide writer and the creator of the Platinum Difficulty Challenge 2022 I've become more and more aware over the past year of the need we have on this website for a way to vote on platinum difficulty ratings and estimated time to completion. 

 

Almost every time a guide is submitted there are comments saying no this is too high or no this is too low. I personally agree that the guide writer should be considered the subject matter expert in this so, in my opinion, their vote should be weighted more heavily than others. However, right now they are the absolute authority with no one having any real say in the matter other than them. The only feedback a user can give is a comment on the guide which is either ignored or if they are all taken seriously creates a yo yo of adjusting the rating based on feedback which only comes from a vocal minority.

 

My goal here is to brainstorm and create a forum that @Sly Ripper can read to give him ideas of how to best implement a way for players to vote.

 

I'll start off with a few suggestions:

 

• Auto-created threads for each game with polls where you can vote for difficulty ratings and time estimates and then guide writers can use those to help them determine a guide rating and players can use them to compare with the guide numbers and get a better feel of it being somewhere between the two.
• A voting system on the guide itself where we can vote with options like: way too low, too low, just right, too high, way too high. Then it can be up to the guide writer if he will change his rating based on that feedback, but even if its not changed players have access to the poll to see the general concensus.
• A link/button on each game on your profile that you've earned the platinum/100% for that you can click and vote difficulty and time and you could only vote if you have the platinum/100%. Averages for all these would then show on the game info section on the game's trophies page.
• Etc.

 

Please help me brainstorm more ideas, elaborate on my suggestions above or otherwise let Sly know you want this feature. I'm not advocating that guide difficulty and time estimates be replaced, rather they be enhanced by supplemental community voting. Thank you for reading and contributing.

[DaivRules]

I think a lot has been said in this thread on the utility of voting for game-related stuff:
 

https://forum.psnprofiles.com/topic/69299-can-we-make-it-so-platinum-achievers-can-vote-on-the-difficulty-of-a-game-and-also-show-that-in-the-guides/

 

Stance on the question asked in the title of the thread: No. Just because they're numbers, doesn't make them science that provide any value. The readers skill level may in no way match a guide writers, nor the average of a bunch of subjective voters, voting based on their own irrational way of assigning numbers to something interpreted as "difficulty" completely differently than everyone else.

 

[DrBloodmoney]

Not to be a nay-sayer - the idea comes from a good place - but I doubt such a system would work, for a couple of reasons:

 

 

 

Firstly...

... people don't actually vote their real opinions. They vote in extremes, to get the consensus to reflect their own opinions. 

 

What I mean by that is, if someone thinks a game is, say, a 4/10, but the current consensus vote is sitting at 6/10, a lot of them won't vote 4/10.

They will vote, say, 2/10, hoping to swing the consensus down closer to what their own opinion is, rather than just offering their own opinion to add to the mix.

 

You could make the voting blind to try and counter this... 

...but that is only liable to exacerbate the problem.

 

People will just assume what the consensus is, based on general chatter, (or the belief that they are so much more skilled than everyone else, because, "Hey, I'm awesome and everyone else sucks" ?)... and go even more extreme.

Half the votes will end up being in the 1/10 - 3/10 range, or the 8/10 -10/10 range, hoping to have the maximum impact, as they will be either assuming the consensus will be "too low," or assuming it'll be "too high".... and they know better.

 

 

Secondly...

... people tend to remember games in rose-tinted ways.

After finishing a game, people don't really remember how much they struggled, they just remember the sweet feeling of success they got at the end. Like Childbirth - or a really good shit -  the euphoric feeling of having finished softens the pain they went through in the process. 

 

You see this all the time over on .org.

They use a voting system for "time to platinum", and "difficulty".

 

If you look at comments by people made before people get platinums, vs. after they get it, they always seem to suddenly go from thinking something is harder, to easier.

 

Also, if you look at what people vote in the "Time to Platinum" ranges, then look at their actual playtimes on Exophase, you will see how wildly people misremember. (Not saying that's deliberate - it is just the nature of memory - we blot out the suffering / boredom / grind, in order to cope!)

 

 

 

 

Personally, I think a better idea is just to require guides to have a "Easier Than X / Harder than X" example from the same genre.

 

Saying, for example Elden Ring is a 6/10, or an 8/10, or a 4/10 will get some contingent riled up, no matter which one you chose.

(as we are all now, acutely aware!) ?

 

That's because different people find different genres harder, or easier, depending on taste, skill, etc.

 

However, saying it is "Easier than Sekiro / Harder than Dark Souls II" is much less liable to be controversial, as it is relative, not absolute. 

 

Whether all those games are considered by the individual reader to be easy as pie, or hard as fuck is irrelevant in that scenario- they still have a basis upon which to gauge the difficulty relative to their own tastes, and a rough idea of what they are in for.

 

Edited May 17, 2022 by DrBloodmoney

[Dreggit]

This would not be implemented. Time and difficulty are subjective to everyone and we would not take away tools from authors.

[dmland12]

I've only written one guide on here, but I'd be way more likely to put in the effort to write more if I didn't have to deal with the rating nonsense at all.  It's such a small thing in the scope of writing an entire guide, and yet, it's like the only thing people focus on and complain about (and do so no matter what values you put there).  I'd just throw it up to voting and let that be it.

 

The ratings are already kind of useless anyway since each guide writer has a different scale.  So, not sure how they would be worse if they were just voted on.

[Milktastrophe]

My 5/10 is different from your 5/10. Instead of a numerical value, there should be a list saying game A is harder than game B, easier than game C, similar difficulty to game D, etc.

[Xylobe]

As mentioned, difficulty is subjective enough that an average of a bunch of arbitrary numbers isn't any more tangibly meaningful than a single arbitrary number.

 

Completion time has at least some level of objectivity to it, but in my experience at least, most guide writers are receptive to feedback if you feel that a listed time is inaccurate or misleading.

[MonaSaxPayne]

this is one of those ideas that is so fundamental, its mind blowing why there's so much resistance to it

 

if the ratings are so subjective as to be worthless (seems to be the general consensus from reading these, and past, replies to the topic) then why even bother letting the guide writer post their own ratings in the road map in the first place? 

 

in 99% of cases guide writers aren't the "authority" on a game they write the guide for. they might just be the first person willing to take the time and effort to post all the trophy requirements in one cohesive, comprehensive format. for any given trophy there's usually dozens of YouTube videos showing how to earn it

[AihaLoveleaf]

26 minutes ago, Rebourne07 said:

• Auto-created threads for each game with polls where you can vote for difficulty ratings and time estimates and then guide writers can use those to help them determine a guide rating and players can use them to compare with the guide numbers and get a better feel of it being somewhere between the two.

I've wanted to see this idea implemented for a long time now, though not for the most noble of reasons. Those curious to know the amount of popcorn that such threads would generate on a daily basis need to look no further than this thread, if not the countless time/difficulty threads over at PST itself.

 

I hope we can at least see standardization of difficulty ratings one of these days. What truly makes a game 5/10 has never been clearly defined, nor agreed upon by the community, and we're seeing the effects of that, with many ratings landing within extremes.

[PhyrxianLibrarin]

10 minutes ago, MonaSaxPayne said:

if the ratings are so subjective as to be worthless (seems to be the general consensus from reading these, and past, replies to the topic) then why even bother letting the guide writer post their own ratings in the road map in the first place? 

 

Because they have to submit something for the guide to be published at all. The only way to make difficulty ratings optional would be for Sly to update the site itself to allow that.

 

This idea comes up every few months; there's no evidence or reason to believe that a community vote would be any more accurate than whatever the guide writer decides. Even on PST.org, you only get like 3 people voting, hardly a good sample size. It's such a subjective measure as to be meaningless.

 

The discourse around difficulty has gotten super toxic lately, to the point where I think the guide system would be greatly improved by dropping difficulty entirely.

Edited May 17, 2022 by PhyrxianLibrarin

[MonaSaxPayne]

2 minutes ago, PhyrxianLibrarin said:

Even on PST.org, you only get like 3 people voting, hardly a good sample size.

 

thats because the site is dead

 

statistically, the more ppl that contribute to a vote, the more accurate the "average rating" will be

 

which is the point of polls. to get an average of the masses. not to be correct for every individual 

[ObsiEez]

As a guide writer myself I personally don't see the point. All difficulty is subjective & length is also depends on how people play. Guides being "too high" is just that specific persons opinion on the difficulty of the game & no amount of people voting would really change the fact that those people are voting what they think would be the hardest. Plus the PST.org voting is super terrible with most people just trolling the votes giving everything a 1 or a 10 so even more recent guides on there are just throwing away the entire voting system & just rating it on their personal experience give or take a few points of difficulty then just linking the difficulty rating for people to go & rate themselves (which they never do). Overall I just think its a waste of space & memory for the site when there are so many other spots that need to be improved instead of wasting resources on something that pointless

[Rebourne07]

I think the fact that the poll shows people want it, but the comments say people don't want it proves my point. A vocal minority comes out to comment that it's useless, but more people that don't care enough to comment, but do care enough to vote, actually do want it. The same would happen with voting threads if they were created. Right now only vocal minorities that want to complain say anything on guide comment sections, however much more people would weigh in if given the chance to just vote. 

 

PST.org is dead as a doornail so their implementation isn't worth addressing. Plus I'm not advocating replacing guide ratings, only supplementing them with an additional voting system. 

[iriihutoR84]

I find the user voted completion time estimates at TrueAchievements immensely helpful, even if you of course have to consider other factors too and not take them as facts. TA has an excellent system where everyone registered to the site upon completing a game gets a message congratulating them on their completion and providing them with a link where they can vote on completion time. The displayed estimated completion time on the site is what most people have voted for. It's fantastic, super helpful and I'd love to see that on PSNP too.

 

As for difficulty, it kind of exists. Looking at completion percentages and platinum percentages does tell you something. Obviously you can't suck at fighting games and then look at completion percentages in that genre comparing them to genres you are good at. Difficulty rating will always be flawed because it very much depends on individual skill and experience. 

 

 

Edited May 17, 2022 by iriihutoR84

[ObsiEez]

7 minutes ago, Rebourne07 said:

I think the fact that the poll shows people want it, but the comments say people don't want it proves my point. A vocal minority comes out to comment that it's useless, but more people that don't care enough to comment, but do care enough to vote, actually do want it. The same would happen with voting threads if they were created. Right now only vocal minorities that want to complain say anything on guide comment sections, however much more people would weigh in if given the chance to just vote. 

 

PST.org is dead as a doornail so their implementation isn't worth addressing. Plus I'm not advocating replacing guide ratings, only supplementing them with an additional voting system. 

A majority of the people don't even vote for the guides that they use so even if, a big if, the voting system DID get added, it would end up dying off pretty quickly because people just can't be bothered to click a few buttons half the time. It's just a waste of resources when it could be used in many other places since pretty much everything on the site is outdated. If it's in actually threads, it would absolutely flood the forums with posts of just difficulty voting which would also stop people from wanting to use the forums cause all they'd see is voting threads of difficulty/time. If it's built into the trophy list itself to vote, do you have any idea how much resources & memory that would take? It would be using so much resources for such a small, meaningless feature

[DrBloodmoney]

12 minutes ago, Rebourne07 said:

I think the fact that the poll shows people want it, but the comments say people don't want it proves my point. A vocal minority comes out to comment that it's useless, but more people that don't care enough to comment, but do care enough to vote, actually do want it. The same would happen with voting threads if they were created. Right now only vocal minorities that want to complain say anything on guide comment sections, however much more people would weigh in if given the chance to just vote. 

 

There aren't really any comments here saying they don't want it - there are comments saying it would not work, or would not be of value though.

 

Another way of looking at what you're saying here, is that a majority of people who aren't giving it much thought are happy to vote "yes", but the majority of people willing to take time enough to actually think about it - enough to leave a comment/ response - have concluded that the information gained from such a poll would not really solve the toxicity of the discourse around difficulty, or give accurate representations of it.

[DaivRules]

If someone wants to move the needle on actually making difficulty information helpful and informative, they should start a thread and define the quantitative measurements and standard objective definition of “difficulty”. I look forward to the conclusions of that thread, which can be used to create applicable difficulty descriptions to game guides. 
 

 

[Bumperklever]

To me, the minimum a guide writer would need to do ,which I consider an expert in the game in question (though that statement is sometimes questionable too), is to follow some kind guideline on how to interpretend difficulty rating. 

 

I saw a few before in other threads about this subject.

 

i think the best way is to standardise a certain logic and advertise this to the guide writers.

 

I dont oppose having a voting system btw. As long as these standards are well advertised. I do think letting the guide writer do it is better.

 

Furthermore, this thread got probably partially sparked  by the fall guys 10/10 rating as well. Which I would never understand. Putting it in the same league as like crypt of the necrodancer.., what? (And im not a vocal minority there xD). Hence why there needs to be some standards. Either by voting or by the guidewriter himself.

[Mellenthin]

On 17/05/2022 at 8:47 PM, DaivRules said:

I think a lot has been said in this thread on the utility of voting for game-related stuff:
 

https://forum.psnprofiles.com/topic/69299-can-we-make-it-so-platinum-achievers-can-vote-on-the-difficulty-of-a-game-and-also-show-that-in-the-guides/

 

Stance on the question asked in the title of the thread: No. Just because they're numbers, doesn't make them science that provide any value. The readers skill level may in no way match a guide writers, nor the average of a bunch of subjective voters, voting based on their own irrational way of assigning numbers to something interpreted as "difficulty" completely differently than everyone else.

 

Having a few people say their piece and give some valuable tips is always welcome in my book. Even when just a single person or a couple is voting, that can go a long way if said persons are well-known and established hunters.

 

The system at PST is not perfect, but it doesn't have to be. The entire point with a poll is to give you a rough estimate when perhaps none previously existed.

 

On 17/05/2022 at 8:48 PM, DrBloodmoney said:

Not to be a nay-sayer - the idea comes from a good place - but I doubt such a system would work, for a couple of reasons:

 

 

 

Firstly...

... people don't actually vote their real opinions. They vote in extremes, to get the consensus to reflect their own opinions. 

 

What I mean by that is, if someone thinks a game is, say, a 4/10, but the current consensus vote is sitting at 6/10, a lot of them won't vote 4/10.

They will vote, say, 2/10, hoping to swing the consensus down closer to what their own opinion is, rather than just offering their own opinion to add to the mix.

 

You could make the voting blind to try and counter this... 

...but that is only liable to exacerbate the problem.

 

People will just assume what the consensus is, based on general chatter, (or the belief that they are so much more skilled than everyone else, because, "Hey, I'm awesome and everyone else sucks" )... and go even more extreme.

Half the votes will end up being in the 1/10 - 3/10 range, or the 8/10 -10/10 range, hoping to have the maximum impact, as they will be either assuming the consensus will be "too low," or assuming it'll be "too high".... and they know better.

 

 

Secondly...

... people tend to remember games in rose-tinted ways.

After finishing a game, people don't really remember how much they struggled, they just remember the sweet feeling of success they got at the end. Like Childbirth - or a really good shit -  the euphoric feeling of having finished softens the pain they went through in the process. 

 

You see this all the time over on .org.

They use a voting system for "time to platinum", and "difficulty".

 

If you look at comments by people made before people get platinums, vs. after they get it, they always seem to suddenly go from thinking something is harder, to easier.

 

Also, if you look at what people vote in the "Time to Platinum" ranges, then look at their actual playtimes on Exophase, you will see how wildly people misremember. (Not saying that's deliberate - it is just the nature of memory - we blot out the suffering / boredom / grind, in order to cope!)

 

 

 

 

Personally, I think a better idea is just to require guides to have a "Easier Than X / Harder than X" example from the same genre.

 

Saying, for example Elden Ring is a 6/10, or an 8/10, or a 4/10 will get some contingent riled up, no matter which one you chose.

(as we are all now, acutely aware!) 

 

That's because different people find different genres harder, or easier, depending on taste, skill, etc.

 

However, saying it is "Easier than Sekiro / Harder than Dark Souls II" is much less liable to be controversial, as it is relative, not absolute. 

 

Whether all those games are considered by the individual reader to be easy as pie, or hard as fuck is irrelevant in that scenario- they still have a basis upon which to gauge the difficulty relative to their own tastes, and a rough idea of what they are in for.

 

People don't vote according to their own actual opinions? I have some experience with agitation but it's rare to see someone who manages to insult an entire site like that.? In all honesty, do you have anything to back up those claims with or is it just conjecture?

 

Either way, in all my years of gaming I've found the polls at PST invaluable. Even when there's only one or a few comments, they can prove to be very helpful - especially so when they are from well-known hunters.

 

It's definitely one thing if the resources aren't available, but I simply cannot see that polls wouldn't have a helpful impact - unless, of course, that you're of the opinion that people in general are dishonest, severely biased and ultimately cannot be trusted.?

 

 

[DaivRules]

51 minutes ago, Mellenthin said:

 unless, of course, that you're of the opinion that people in general are dishonest, severely biased 

Confirmed. 
 

 

[Mellenthin]

42 minutes ago, DaivRules said:

Confirmed.

 

I've been using PST and later on PSNP for about 10 years, and I've seen the same pattern from the very start: people usually gets the difficulty and time required down pretty good.

 

If your experience is that what people comment outside from greenlit guides is to be seen as false until proven otherwise, then I guess I understand your comment better.

 

Yet I think it's a shame you seem so distrustful and dismissive from something that has great potential. For example, having a poll section could have saved me 70 hours or so in Pillars of Eternity. Having an otherwise detailed guide on the site but leaving out a bug that breaks in-game tracking is pretty significant in my opinion.

 

We do have the comment section but it would be much more convenient to find relevent information in the appropriate topic.

 

 

 

 

[Chaos_JB_2017]

Just do it so we keep the author of the guides scores and then a separate one made up of the average of people’s votes

 

[DaivRules]

2 hours ago, Mellenthin said:

I've been using PST and later on PSNP for about 10 years, and I've seen the same pattern from the very start: people usually gets the difficulty and time required down pretty good

 

And statistically, how often do those ratings deviate from what a guide author designated themselves, including the times they've taken into consideration feedback and adjusted?

 

For someone who accused others of being biased, your posts are full of bias.

 

You're ignoring the actual reasons already given from others (who you quoted) who don't think this adds any value, and replacing it with twisted arguments we didn't make. When you said others must be dishonest and severely biased, you're the one who's shown both of those qualities.

 

 

Edited May 19, 2022 by DaivRules

[HusKy]

No need for vote system regarding time, as this information can be now extracted from PSN data. With large enough dataset, you can get some pretty accurate numbers.

[dieselmanchild]

This is one feature this site needs very badly. I vote hell yes.

 

I pretty much use PSNP exclusively now, but I still use PST for one thing - accurate time/difficulty estimates. The PSNP estimates are based on one person’s opinion and as such, they’re often off the mark by a few points. Sometimes wayyy off the mark. I’ve seen many other people remark about this over the years and complain to me about how poor the guide estimates are in many cases.
 

A polling system based on community feedback of time/difficulty is simply far more accurate as it takes into account dozens of opinions and calculates an average based on that data, and will always more accurately reflect the experience that an average gamer is likely to have.

 

I also really like the fact that along with their vote, people on PST are allowed to leave comments elaborating on why they voted the way they did. I often spend time scrolling back through several pages reading comments on these threads, as it really helps paint an accurate picture of what you can expect to deal with when jumping into a game. It’s an incredibly helpful feature for someone like myself who does a lot of research on games before diving in.