Re: [Enigma-devel] ratings system

[Ronald Lamprecht]

Hi,

Tacvek wrote:
> Some questions and comments about the rating system. That details behind 
> the rating system was never very clear to me,
> so that is why I wish clarification.

As just a few developers and level administrators need details on the 
rating system you will find most comments on details in the source code 
(src/lev/RatingManager.* and data/schema/ratings.xml)!

> As far as I can tell, the ratings system will eventually be some form of 
> web service, through which players can submit ratting/score information, 
> which will appear on the level inspector? The game is will also be able 
> to download updated rating from this service?

I had in mind that players can submit their scorefile, which keeps 
additionally their personal rating of levels. Some web service should 
evaluate all available scorefiles and adds the "public ratings" like 
intelligence, dexterity,... which are set manually by an level 
administrator (Andreas) for purpose of consistency. The resulting 
ratings.xml is put on a download server. If a user allows ratings update 
(Option "Auto") Enigma will download the file from time to time.

We have to consider the amount of data. As we have peek downloads of 
8888 per day, there may well be 100k players. With 1000 levels the 
ratings sums up to 100 kBytes. Thus I added the possibility to download 
incremental rating updates and introduced a time limit on rating updates.

> It appears that the ratings file will be stored in 
> "~/.enigma/ratings.xml", this file is currently generated from the 
> indices, but presumably that will be changed at some point in the future.

Existing ratings.xml entries override index values. The next Enigma 
release will distribute an up to date ratings.xml (supplied by Andreas). 
The generation of ratings from indices is just for upward compatibility. 
It remains in the next release to allow users with own indices to 
upgrade to the new system.

> The update line in the rating file will give the information nessisary 
> for enigma to download the updated file.

Yes.

> I assume that the ratings information and par information currently in 
> the indices will eventually be removed, as they would be obsolete.

The whole indices will be replaced by a xml-version I am currently 
working on. The main information in the new version will just be the 
sequence of the levels. No ratings info will be stored in the new indices.

> The Par value will apparently no longer be have a name attached, which 
> implies that it will be calulated. What sort of formula will be used? 
> 95% percentile? Average of the top 10% of submitted scores?

Please read thread "[Enigma-devel] Scores - which values make sense" 
from 24th March 2006.

> To avoid really low statitistics in the '% of users who have solved this 
> level', some sort of filtering should be applied. For example a minimum 
> number of levels be completed before the user can be considered. This 
> avoids poeple who download the game, play it a bit, and decide they 
> don't like it from tainting the percentages. It may also be wise to 
> exclude people form consideration for that statistic who have not 
> participated recently, as they
> may have abondoned the game, and would definately drag down that 
> statistics for levels introduced after they stopped submitting information.

Good additions to the thread above.

> The interface allows users to change rating values. Are those editable 
> just to support development, or will that be an interface to allow users 
> to rate the levels?

The "public ratings" are editable for the level administrator (Andreas). 
The switch is not documented. If a player edits the "public ratings" the 
next update will reset the modifications.

> Also there appears to some sort of generic rating value, that will apear 
> at the bottom of the Inspector, along with an average (from all users). 
> I assume this will be user supplied via an interface not currently 
> avalable?

Yes - it will be a value 0 - 10 which the player can set to express his 
opinion concerning the level. The value will be stored together with the 
score and the average value of user who rated the level is sent back 
with the ratings.

> Finally there is the annotation field, which I assume is intended for 
> the user's own private use.

Yes - I want to store the annotations in a private file - not within the 
score file that will be uploaded.

> The rest of the data on the inspector screen all appears to be metadata 
> that will be extracted from the XML files.

Yes.

> Is the above all correct? I dpn't intend to be a bother, but knowing the 
> purpose of that code is somewhat important should i decide to 
> participate more actively in its development. After all, I am capable of 
> doing more than merely porting Enigma to Lua 5.1. (Speaking of which, 
> lua 5.1.1 is availble. It is a simple bug fix release. It *Should* be 
> possible to use the new versions as drop-in replacements for the old 
> one. There are no language changes, and the scripts are unable to tell 
> the difference.)

As you noticed there are some points to finish for a solid next release. 
Thus any additional help is welcome.

I think we should test and upgrade to any bug fix release of Lua 5.1 
till we reach the alpha testing phase.

- Ronald