DLC % logic?

[andshrew]

On 1/15/2023 at 7:43 AM, Dry said:

 

Sly replied to a post of mine months ago reaffirming his reason for why he disabled this, which still makes zero sense to me.

 

I'd be very happy if even once a month PSNP goes down for an entire hour to update all 5.6+ million tracked profiles (as of this post), both registered and unregistered but given the site owner's activity (or lack thereof) over the past 4-5 years aside from him making way for PS5 platform trophies and introducting a much needed "shovelware filter", such a beneficial event may never occur...

 

...Though I'd like to be proven wrong.

While sites like this operate through the back door so to speak (ie. they access Sony's APIs in the context of a regular user not as a partner or service in an official capacity with Sony) it's very unlikely this would be feasible.

 

To refresh a users data from PSN you've got to make at least 1 API request to pull in their trophy list (but potentially several if they have more than 800 titles on their list), and then 1 API request per title they've earned a trophy for (subsequent refreshes of that users data wouldn't require you to request the same title again if they haven't earned anything new for it, so the cost of this would reduce over time).

 

The PS3/PSV/PS4 era trophy API had a rate limit of ~300 requests per 15 minutes (the PS5 era API which replaced it likely has a similar limit but I haven't verified it). So based on that in a best case scenario a single PSN account can be used to make ~28,800 API requests to Sony's servers over a 24 hour period without being limited. Obviously if you run a site like this you can start stacking up how many PSN accounts you use to run your service to increase your API capacity - but perhaps you then run into IP based rate limits, or even just Sony getting angry at the number of requests you're making so they start suspending the accounts you're using.

 

When you consider how many titles 5 million PSN user accounts might have it's an obscene number of requests you would have to make to Sony's trophy API to do a complete refresh. I imagine that just trying to keep up with an active user base in the 100k region would be a struggle, let alone millions.

 

Anyway, if you're interested in what the Sony API looks like to give yourself a better understanding of the technical challenges take a look at this.

[Mellenthin]

Thank you for shedding light upon the update process.? I always thought it was due to Sly being uncaring and I've honestly been quite annoyed over that people's profiles haven't been updated as much as I wanted.

 

Perhaps I've missed the memo completely, but have Sly gone out with this info in the past?

[sweeping-lamp3]

On 1/20/2023 at 3:13 PM, andshrew said:

While sites like this operate through the back door so to speak (ie. they access Sony's APIs in the context of a regular user not as a partner or service in an official capacity with Sony) it's very unlikely this would be feasible.

 

To refresh a users data from PSN you've got to make at least 1 API request to pull in their trophy list (but potentially several if they have more than 800 titles on their list), and then 1 API request per title they've earned a trophy for (subsequent refreshes of that users data wouldn't require you to request the same title again if they haven't earned anything new for it, so the cost of this would reduce over time).

 

The PS3/PSV/PS4 era trophy API had a rate limit of ~300 requests per 15 minutes (the PS5 era API which replaced it likely has a similar limit but I haven't verified it). So based on that in a best case scenario a single PSN account can be used to make ~28,800 API requests to Sony's servers over a 24 hour period without being limited. Obviously if you run a site like this you can start stacking up how many PSN accounts you use to run your service to increase your API capacity - but perhaps you then run into IP based rate limits, or even just Sony getting angry at the number of requests you're making so they start suspending the accounts you're using.

 

When you consider how many titles 5 million PSN user accounts might have it's an obscene number of requests you would have to make to Sony's trophy API to do a complete refresh. I imagine that just trying to keep up with an active user base in the 100k region would be a struggle, let alone millions.

 

Anyway, if you're interested in what the Sony API looks like to give yourself a better understanding of the technical challenges take a look at this.

 

Thank you so much for the information.

 

I tried creating my own trophy tracking website last year using the PHP wrapper created by tustin at https://github.com/Tustin/psn-php

 

However, after creating a prototype I decided not to continue forward after I discovered a few unacceptable obstacles such as the rate limits.

 

Do you happen to know why certain users might or might not be available in the API?

 

For example, the PSN ID chalk_gaming is find-able by PSNProfiles and Exophase but not PSN100.
Another example, the PSN ID AdamLeitch is find-able by PSN Trophy Leaders but not TrueTrophies.

 

There were also a few other cases but I didn't document them at the time of experimentation.

 

I also tried checking the PSN App to see if I can find some of these users.
Based on some testing, I think the PHP wrapper by tustin is mimicking the PS App?

 

Is finding PSN IDs restricted somehow by the account being used to access the API?
or are there other endpoints being used by some trophy sites that was once publicly used by Sony but are no longer used publicly but are still active (such as the old "my.playstation.com" site where players were able look at their trophy data?

Edited January 24, 2023 by sweeping-lamp3

[andshrew]

4 hours ago, sweeping-lamp3 said:

Is finding PSN IDs restricted somehow by the account being used to access the API?
or are there other endpoints being used by some trophy sites that was once publicly used by Sony but are no longer used publicly but are still active (such as the old "my.playstation.com" site where players were able look at their trophy data?

The short answer is yes, everything should respond based on what privacy settings a user has set on their account. So if someone has their trophy list set to friends only you would only be able to see it if the account you are using is their friend. The longer answer is there's sometimes some weirdness where things don't always seem to behave how you would expect (eg. accounts displaying information they shouldn't be, or restricting it when it should be open to all). There was some discussion on different profile APIs here. I believe the old APIs still exist but it's difficult to access them now as they revoked all the methods which were used for accessing them (eg. my.playstation and the old PS App). I think most public projects like Tustin moved over to only using the current PS App APIs. 

[sweeping-lamp3]

8 hours ago, andshrew said:

The short answer is yes, everything should respond based on what privacy settings a user has set on their account. So if someone has their trophy list set to friends only you would only be able to see it if the account you are using is their friend. The longer answer is there's sometimes some weirdness where things don't always seem to behave how you would expect (eg. accounts displaying information they shouldn't be, or restricting it when it should be open to all). There was some discussion on different profile APIs here. I believe the old APIs still exist but it's difficult to access them now as they revoked all the methods which were used for accessing them (eg. my.playstation and the old PS App). I think most public projects like Tustin moved over to only using the current PS App APIs. 

 

Thank you for the response.

 

I also had another theory.  In addition to the privacy setting and older APIs some private sites may have captured the user's internal account id and saved them long ago.  Now, with the current PS App API I believe you can do a user search based on a substring or account id.  If the site does not have the user's account id previously then it will try the substring search.  I believe there is a hard limit (of 128 results? - I'm not sure. It will be almost a year since I've touched the API) on the number of names returned when searching by substring.  If the user name is not within the limited search results (especially the search checks matches for the first name and/or last name fields and seems to treat dash and/or underscore as wildcard characters) then the site is out of luck in getting the user's internal account id and associated game/trophy data.

 

Sorry for rambling on.  I may have deviated from the topic of this thread.

 

Edited January 25, 2023 by sweeping-lamp3