Dificulty Rating Section

[CabDK]

10 hours ago, dieselmanchild said:

I completely support this suggestion Cab and have argued in favour of it many times, because while I absolutely love PSNP and regard it as my favourite trophy site by far, a major pet peeve I’ve always had with this site is the fact the difficulty ratings & time estimates are solely based on the guide author’s opinions.
 

Anybody who is writing a guide for a game likely knows it inside and out, and so their ratings rarely reflect the reality for the average trophy hunter and guide reader. It’s been my experience that more than half of the guides on this site are very inaccurate in both of these categories, even when the guide itself is otherwise excellent quality, and it’s a complaint I hear constantly from friends and fellow trophy hunters. You can see this play out in the comments on new guides all the time too.

 

Personally I really loved the community based polling + comment format that Playstationtrophies.org (PST) used, the data from which was inserted directly into the guides. A lot of people have brought up the problematic issue of “troll votes” and I suppose it’s a valid concern in this day and age where things like review bombing are common place, but if you go back to older games from the PS3 era on PST (when the site was at it’s peak) I found that most of the time the difficulty/time ratings were very accurate, certainly far more accurate than anything you see on PSNP.

 

Then there were the comments — arguably the best part of that system. Back in its peak, most PST users would post comments alongside their vote to elaborate about why they voted that way. In these threads you’d often find extremely useful tips and insightful commentary that would give a true idea of how long you could reasonably expect to achieve all the trophies & how difficult you could expect it to be. It was honestly kind of tragic watching PST’s user base slowly dwindle away and those threads on newer games become a ghost town, because in it’s heyday the usefulness of that place was unmatched.

 

I can understand the reservations some people have with this idea. Aside from the work of actually creating such a system for this site, the main issue for many people seems to be a lack of agreement on how it should be implemented, and who should be allowed to issue ratings etc. But I hope we can all at least agree that a system like this would be an overall improvement to the site and the user experience. 

 

 

Thank you, for a great and insightful post. I agree with all of it.

8 hours ago, ObsiEez said:

 

& that's the problem, you CAN'T be accurate with a difficulty rating as it's 100% subjective, not based in any kind of facts, yet everyone argues their OPINION of the rating as being correct. Even if you poll it, it still isn't the "right" difficulty rating because it all depends on the person playing. Going into a guide & arguing difficulty rating is just as pointless as going into a guide and arguing if the game is good or not, because it's all subjective. Guides shouldn't even have a difficulty rating to begin with as it adds 0 value & causes arguments over something that is purely subjective.

For the souls games, highest I'd rate them is a 5/10 across the board. The Evil Within I'd rate a 4/10. But for other people, both of those could very well easily be 10/10's. Even if you create a poll, there will be very little votes (rating's on guides already prove that the userbase isn't willing to spend 2 seconds of their time to rate something on a scale, so why would doing it for difficulty be any different?) Difficulty ratings overall are a pointless metric that is just a pat on the back for the people that agree with the rating. and an excuse for people to complain about a subjective rating if they don't agree. Like you said, new guides are constantly being filled with comments about the difficulty instead of what the comments SHOULD be filled with, which is feedback about the actual guide itself, not an arbitrary number that is different for everyone.

 

As a guide writer, the way I personally rate my guides when it comes to difficulty is I take my rating, then add 2 points to it. So for Callisto Protocol for example, I personally think it's a 3/10 difficulty wise, but I added 2 points to make it a 5/10 because most people would view that as "too low" for the "average player". What even is the average player? What kind of games do they play? Are they familiar with the genre? Do they play video games often? It all depends on a multitude of factors that a simple scale of 1 to 10 can never accurately measure to a player. I think it's the same thing with reviewing. Reviewers rating a game a 10/10 is like "Oh wow, that game must be good!" But why is the game good? That 10/10 means nothing without context, just the same thing with difficulty ratings. Not to mention difficulty ratings could mean it's difficult in literally so many ways that can't be conveyed by a number. Is it hard because it forces you to use your brain a lot? Is it hard because it requires really precise movements & coordination? Is it hard because it's an extremely strict time limit? Some people might find some of those things easier than others, but all of those would still be rating under the same umbrella of a difficulty rating making it difficult to gauge for players.

Basically TL;DR, a number rating is pointless, watch gameplay videos, reviews, or even play the game yourself to form your own opinion on difficulty because no one is a better gauge for how difficult a game is than you.

Your arguments against such a system, is the exact reason we need it imo.

[Shikotei-kun]

3 minutes ago, CabDK said:

Your arguments against such a system, is the exact reason we need it imo.

How so? Obsi's take is that difficulty is subjective and many faceted. It's not something that you can boil down to a single number.

 

If you were to implement it, then the difficulty rating system should incorporate these facets too. Like a spiderweb graph.

Then we'd have a discussion about which facets to include for each game. You can hardly rate a turnbased RPG based on the player's twitchy trigger finger skills. Or a shooter based on the player's logical thinking capacity.

 

And -even then- it's still subjective. How do you scale each facet? Do you take reaction tests, and quick-decision-making as a measure for shooter difficulty? How would you distinct Borderlands from Wolfenstein? IQ tests for puzzle games like The Witness?

 

I get it though: the author's rating is just one person's opinion. And more opinions means a more balanced outcome. But it's still an opinion. And anyone whose opinion differs far from the average of many will want to see theirs come out on top. Which is when discussions about difficulty start. And they'll never end. The low end of the bell curve will argue with the high end of that curve, while the top of the curve will be satisfied until one end wins and the whole cycle starts anew.

[ObsiEez]

33 minutes ago, CabDK said:

Your arguments against such a system, is the exact reason we need it imo.

My whole argument is against difficulty ratings as a piece of informative information in general, not just as an addition to the site because it's an arbitrary metric and a scale of 1/10 is pointless without context, not to mention the average persons skill at a video game will vary significantly in different genres & types of challenges. I don't see how my arguments I've made would be the reason we need it

 

5 minutes ago, Shikotei-kun said:

How so? Obsi's take is that difficulty is subjective and many faceted. It's not something that you can boil down to a single number.

 

If you were to implement it, then the difficulty rating system should incorporate these facets too. Like a spiderweb graph.

Then we'd have a discussion about which facets to include for each game. You can hardly rate a turnbased RPG based on the player's twitchy trigger finger skills. Or a shooter based on the player's logical thinking capacity.

 

And -even then- it's still subjective. How do you scale each facet? Do you take reaction tests, and quick-decision-making as a measure for shooter difficulty? How would you distinct Borderlands from Wolfenstein? IQ tests for puzzle games like The Witness?

 

I get it though: the author's rating is just one person's opinion. And more opinions means a more balanced outcome. But it's still an opinion. And anyone whose opinion differs far from the average of many will want to see theirs come out on top. Which is when discussions about difficulty start. And they'll never end. The low end of the bell curve will argue with the high end of that curve, while the top of the curve will be satisfied until one end wins and the whole cycle starts anew.

 

Pretty much my point exactly honestly. There is no accurate way to gauge a difficulty without specific context. & even then, it's still subjective to if the player will find it an accurate rating for them with or without context

Edited October 19, 2023 by ObsiEez

[DrBloodmoney]

5 minutes ago, ObsiEez said:

Pretty much my point exactly honestly. There is no accurate way to gauge a difficulty without specific context. & even then, it's still subjective to if the player will find it an accurate rating for them with or without context

 

 

This is right, and exactly why there is never any real conclusion to any discussion of difficulty ratings.

 

Fundamentally, even within the relatively small subsection of gamers on this site, there are huge variables, even in what people use as the definition of a difficulty rating, before you even get to people's individual ratings of a specific game.

 

Is a difficulty rating...

 

"The difficulty of the game for someone unfamiliar with the genre?"

"The difficulty for a person very familiar with the genre?"

"The difficulty of the game relative to other games of the genre?"

"The difficulty relative to other games of other genres?"

"The difficulty of the game while using a guide?"

"The difficulty of the game playing blind?"

"The difficulty of the game after a set number of hours?"

"The difficulty for someone familiar with the franchise already?"

"The difficulty for someone completely new to the game?"

"The difficulty of the game once familiar with the basic mechanics?"

"The difficulty of the game once the narrative is complete, and the mechanics are well known?"

"Is cerebral challenge the metric?

"Is it about twitch dexterity?"

"Is RNG technically "difficulty?"

"Is grind?"

"is length?"

"Are puzzles considered difficult, if someone can just look up the answers?"

"Does a game drop in difficulty once "best builds" are available on the internet?"

"Does a glitch or loophole that negates difficult sections lower that rating or not?"

"If a game is technically "boostable", does that affect it?"

"Does the mere existence of 45-second shovelware games force all actual games up from a 1/10 to a minimum of a 2/10, or do you simply negate those games from consideration?"

"Does the mere existence of absurdly difficult games like Crypt of the Necrodancer force all other games down to a maximum of a 9/10, or do you simply negate those "extreme" examples from consideration?"

 

 

 

TBH, I reckon that if you ask 100 people on this site for the answer to each of those questions, you'd likely get 100 different sets of answers - and they'd all be valid.

 

That means individual "ratings" from someone can be useful - providing you know the specific context for what they consider the metrics...

 

...but having a conglomerate answer, where you have no idea what metrics individual people are using just means that - aside from the very extreme ends of the spectrum - every rating ends up being largely useless, because the large discrepancies in individual metrics render any result as only really accurate to like ±5 points...

...which on a 1-10 scale, makes them virtually redundant.

 

[CabDK]

10 minutes ago, ObsiEez said:

Pretty much my point exactly honestly. There is no accurate way to gauge a difficulty without specific context. & even then, it's still subjective to if the player will find it an accurate rating for them with or without context

Because it's subjective, an average rating or more ratings makes way more sense as opposed to one person. And this is also why a comment section is important, to add some context.

 

By having a lot of vores, you can get a better understanding of the dificulty, compared to just a guide writer that votes four while 90% of people vote seven. Yes I agree it's 100% subjective and sometimes you are not going to agree with the average rating of course.

[ObsiEez]

20 minutes ago, CabDK said:

Because it's subjective, an average rating or more ratings makes way more sense as opposed to one person. And this is also why a comment section is important, to add some context.

 

By having a lot of vores, you can get a better understanding of the dificulty, compared to just a guide writer that votes four while 90% of people vote seven. Yes I agree it's 100% subjective and sometimes you are not going to agree with the average rating of course.

That's also why I personally advocate for removing difficulty ratings from the guides just in general. But lets be honest, even if we both got what we wanted, a place to vote, & a place to vote with all the context of it like @DrBloodmoney mentioned in his comment, the chances of anyone actually taking the time out of their day to vote for it, even if it takes 5 seconds, will still be very low, especially if the game is more niche, which is usually the ones you'd want more information on.

I know it's not the exact same thing, but I'll just use this as an example. My guide I wrote for Rune Factory 4 has a total of 30,686 views at the moment, but only 7 ratings. Now, that number 10x higher than the actual number of game owners, which is 3,327, so for the sake of argument, we'll use the owner numbers since we can't count for duplicate views, or unregistered views. That means out of 3,327 unique players of the game, only 7 of them took the 2 seconds it takes to rate a guide, which is usually a place someone would frequent going for the platinum & would rate it if they found it useful, or not useful. But this really isn't the case & it's very rarely users every do. So if the site actually DID add a difficulty rating section, chances are it'd almost never get any actual votes outside a very select few users, especially if it was a more in-depth rating system to add context to the actual ratings. It's the sad reality that as soon as people get the platinum/100% (which would really be the only way to lower the amount of troll votes), most people are just going to move on & not go out of their way to rate it, which is pretty much what happened with the other trophy sites that have implemented this in the past.

 

21 minutes ago, DrBloodmoney said:

Snip

I agree 100% with you there. Stuff like that is more of a useful gauge of difficulty than the regular 1/10 rating that's implemented on most sites, & usually proposed to happen here, but like I mentioned above & especially in this case if it really went to be as broad as possible to encompass all types of things that'd contribute to difficulty, people just aren't going to take the time to do it.

Edited October 19, 2023 by ObsiEez

[HusKy]

I see no issue with this -- it's just another statistic. This whole website is a giant spreadsheet, adding another metric like this would make sense.

 

On the other hand, the chances that this will get added are very slim so I suppose the discussion is indeed useless.

[Fiercethetear]

23 minutes ago, HusKy said:

I see no issue with this -- it's just another statistic. This whole website is a giant spreadsheet, adding another metric like this would make sense.

 

On the other hand, the chances that this will get added are very slim so I suppose the discussion is indeed useless.

Exactly. I dont understand why people here are finding a big issue with this literally nobody is saying remove the guidewriter's opinion just add the difficulty voting option as extra. More features does not hurt people

[CabDK]

2 hours ago, ObsiEez said:

I know it's not the exact same thing, but I'll just use this as an example. My guide I wrote for Rune Factory 4 has a total of 30,686 views at the moment, but only 7 ratings. Now, that number 10x higher than the actual number of game owners, which is 3,327, so for the sake of argument, we'll use the owner numbers since we can't count for duplicate views, or unregistered views. That means out of 3,327 unique players of the game, only 7 of them took the 2 seconds it takes to rate a guide

Its not a good comparison imo. I've been trophy hunting for 14 years, and I know how popular the dificulty polls was in the prime of .org hundreds of votes and pages of comments. It's like with the most popular thread on this forum "Post your latest platinum" it's extreamly popular because people like to show their new plats and share their thoughts on dificulty.

 

Edit: Of course you could be right and nobody cares, but I highly doubt it.

2 hours ago, HusKy said:

I see no issue with this -- it's just another statistic. This whole website is a giant spreadsheet, adding another metric like this would make sense.

 

On the other hand, the chances that this will get added are very slim so I suppose the discussion is indeed useless.

Unfortunately you are absolutely right, it is wishful thinking.

1 hour ago, Fiercethetear said:

Exactly. I dont understand why people here are finding a big issue with this literally nobody is saying remove the guidewriter's opinion just add the difficulty voting option as extra. More features does not hurt people

Exactly, I dont understand why people are against it tbh, people can just stay away from the section if they are not interested.

Edited October 19, 2023 by CabDK

[Rally-Vincent---]

11 hours ago, CabDK said:

 I've been trophy hunting for 14 years,

 

What do you need widely differing opinions of other people for? You have over a decade of experience, you are very well able to judge how difficult you are going to find a game., and your own verdict is going to be much closer to your truth then an aggregate score from other people.

[CabDK]

9 hours ago, Rally-Vincent--- said:

 

What do you need widely differing opinions of other people for? You have over a decade of experience, you are very well able to judge how difficult you are going to find a game., and your own verdict is going to be much closer to your truth then an aggregate score from other people.

I dont! Maybe other people do tho.

 

Edit: You can find something useful and interesting without needing it. I'm interested in other people's opinions regardless.

Edited October 20, 2023 by CabDK

[Rally-Vincent---]

4 hours ago, CabDK said:

I dont! Maybe other people do tho.

 

Edit: You can find something useful and interesting without needing it. I'm interested in other people's opinions regardless.

 

You can. In my opinion, site ressources were better invested in something more important like fixing auto-updating profiles rather than a heavily subjective statistic that - should the need arise - anyone could inquire via a thread in the respective game sub-forum. This suggestion feels a bit like decorating while there's still a crack in the roof.

[dieselmanchild]

On 10/18/2023 at 5:21 PM, ObsiEez said:

 

& that's the problem, you CAN'T be accurate with a difficulty rating as it's 100% subjective, not based in any kind of facts, yet everyone argues their OPINION of the rating as being correct. Even if you poll it, it still isn't the "right" difficulty rating because it all depends on the person playing. Going into a guide & arguing difficulty rating is just as pointless as going into a guide and arguing if the game is good or not, because it's all subjective. Guides shouldn't even have a difficulty rating to begin with as it adds 0 value & causes arguments over something that is purely subjective.

I take your point, but I couldn’t possibly disagree more with the idea that it’s useless or adds zero value. I think most people who make use of trophy guides find it very helpful to have a general idea of how challenging a game might be before they dive into it. I know I do, even if that number ultimately doesn’t decide whether or not I’ll play a game. I just like having a sense of what I’m getting myself into.

 

While it’s true that difficulty is entirely subjective and there will never be a one-size-fits-all number that appeases everybody, polling across a large sample size of people who actually played the game is by far the most accurate way to determine a fair and reasonable rating. Again, there will always be some people who disagree and find it easier or harder than the guide rating. But when that number is determined by the average rating submitted by all the people who voted, you’re going to find way more people who find it more or less accurate and reflective of their experience. And consensus is typically governed by what the majority thinks.

 

On 10/19/2023 at 2:03 AM, CabDK said:

Thank you, for a great and insightful post. I agree with all of it.

Thank you Cab. 🙂

[JohnCenaSong-]

Do I care about a feature like this? Nope not really. 

 

But I think there is value in a difficulty polling in the same way there's value in user voted scoring for game quality. Sure, just like review bombing and the like there's going to be troll votes even if you limit it to those who either have 100% (or an A rank or higher for extra inclusion) but for the most part you're going to get some kind of consensus.

 

Now, consensus doesn't make it a fact, just like a 9/10 user score on Metacritic doesn't mean you'll even remotely like the game. But there's value to be had in easily digestible figures for consensus. The only other alternative is to stick to the singular rating of a guide writer which would be like solely relying on IGN review scores for some marker of game quality. That's not to criticise guide writers (nor IGN actually) but that a single source isn't the best.

 

I think it's a shame people are so quick to dismiss things like this. I get that some folk are obsessed with difficulty ratings in the same way as some people are overly fixated and obsessed with review scores and that makes people feel like they want to stand in opposition to those types but I don't think that average ratings on things themselves are pointless and they do serve a value.

[JAK-KRIEG]

I don't understand why people are against this idea imo more data doesn't hurt anyone and the guide writers are still able to give their opinion. Status quo is found wanting.

[Shikotei-kun]

On 10/25/2023 at 1:56 PM, JAK-KRIEG said:

[..] imo more data doesn't hurt anyone [..]

The point is that the additional data is going to be unreliable, meaningless without context, and -if the guide-rating usage is any indication- barely populated.

Would you rate all 663 games you completed to 100% should this ever become a feature? Would you lament not being able to vote on the other 51?

Would those older ratings be even accurate enough, knowing some of them were played well over a decade ago?

 

If you want to know how difficult a game is, watch gameplay videos, let's-plays, and reviews. From multiple sources -as that's the same as multiple people voting on a scale of 1-10). Find out the genre of the game, the major gameplay mechanics, and make an estimation whether or not you'll be comfortable with those mechanics.

In other words: do research. Difficulty is relative and can't truly be a single number between 1 and 10.

[JAK-KRIEG]

1 minute ago, Shikotei-kun said:

The point is that the additional data is going to be unreliable, meaningless without context, and -if the guide-rating usage is any indication- barely populated.

Would you rate all 663 games you completed to 100% should this ever become a feature? Would you lament not being able to vote on the other 51?

Would those older ratings be even accurate enough, knowing some of them were played well over a decade ago?

 

If you want to know how difficult a game is, watch gameplay videos, let's-plays, and reviews. From multiple sources -as that's the same as multiple people voting on a scale of 1-10). Find out the genre of the game, the major gameplay mechanics, and make an estimation whether or not you'll be comfortable with those mechanics.

In other words: do research. Difficulty is relative and can't truly be a single number between 1 and 10.

Im gonna keep it short I disagree on all your points except for yes will there be games with few votes of course but an average rating on many more popular games than say niche ones still can't hurt and would that be beneficial yes of course it would imo.

[Rejim]

I think I like this idea and I don't understand with the people bashing it and finding that the data is unreliable.. is the guide info reliable? Are the forum posts reliable? Everything is subjective in the end. But I want the info! Yes, the guide writer will probably have a slight lower difficulty rating because it often suits his playstyle or knows the game inside out and be totally far from what I would mark it but still, it's helpful to have an idea. I can always dismiss it if I want to.

 

But having multitude of opinions is also useful. Only if you have the complete data of how many votes each rating has gotten, and not just the average (6.4/10 on its own means too little). Because in some cases it can even out. For example, you will have the hardcore gamers probably marking a game as 4, the normies as 6, and the casual as 7. If you check that game and you see 11 votes to 7/10, 8 votes to 6/10 and 12 votes to 4/10, you will probably get the impression that the difficulty would depend on the person's playstyle, and if you're a casual it might be a bit on the harder side. A perfect example but still, you lose nothing by having that info. And for those that don't like it, they can just don't pay attention to it.

 

As for the rules, that's where it gets a bit tricky. The plat/100% would seem the fairest but it could throw the balance, because logically it's mostly the ones that are still trying to get the plat/100% that find it more difficult than the ones that managed to finish it. It was tossed around that a rank A as a minimum could be an option and I think it would be the best way if it ever gets implemented. Means you have invested quite enough, but you can still voice your opinion if it gets out of hand.

[Mellenthin]

On 11/7/2023 at 12:27 AM, Rejim said:

I think I like this idea and I don't understand with the people bashing it and finding that the data is unreliable.. is the guide info reliable? Are the forum posts reliable? Everything is subjective in the end. But I want the info! Yes, the guide writer will probably have a slight lower difficulty rating because it often suits his playstyle or knows the game inside out and be totally far from what I would mark it but still, it's helpful to have an idea. I can always dismiss it if I want to.

 

But having multitude of opinions is also useful. Only if you have the complete data of how many votes each rating has gotten, and not just the average (6.4/10 on its own means too little). Because in some cases it can even out. For example, you will have the hardcore gamers probably marking a game as 4, the normies as 6, and the casual as 7. If you check that game and you see 11 votes to 7/10, 8 votes to 6/10 and 12 votes to 4/10, you will probably get the impression that the difficulty would depend on the person's playstyle, and if you're a casual it might be a bit on the harder side. A perfect example but still, you lose nothing by having that info. And for those that don't like it, they can just don't pay attention to it.

 

As for the rules, that's where it gets a bit tricky. The plat/100% would seem the fairest but it could throw the balance, because logically it's mostly the ones that are still trying to get the plat/100% that find it more difficult than the ones that managed to finish it. It was tossed around that a rank A as a minimum could be an option and I think it would be the best way if it ever gets implemented. Means you have invested quite enough, but you can still voice your opinion if it gets out of hand.

 

What you're saying is what I consider common sense. Unfortunately, that seems like a rare commodity nowadays.🥲

 

A rating system of this kind doesn't need to be perfect, nor does it have to be of use to literally everyone. Just because I personally see little to no value in most threads on this site, that doesn't necessarily mean they have no value for others, or for the site itself.

 

A simple solution would be to ignore site services that's of no use to you, although that might be too much to ask for, I know.😅

[Vault-TecPhantom]

Just throwing in my two pence: I've often found the polls on the Playstation Trophies website to be accurate. Of course, no system like that is perfect due to how subjective difficulty is and the occasional troll vote... but for the most part I tend to agree with the consensus. That said, the guide writers here also grade difficulty similar to how I would and, even when I disagree, I can at least see how they came to that score. There are definitely exceptions though - The Witcher 3 difficulty rating of 7/10  is waaaaaaay too high and Spyro the Dragon  2/10 difficulty rating is too low. 👀