With the demise of the game mods and the constant on going grumblings of the community as a result, i thought it might be interesting to have a discussion about possible alternative ways of dealing with this issue.
From what i can gather/remember, the game mods were disbanded, in part, because members of the moderation team were themselves accused of cheating. While i can't be certain of this (as i wasn't party to the decision), it seems clear that any reintroduction of this system would most likely fall foul of the same or similar problems. While some members of this site are commonly held to be engine users, some are considered genuine, but proving this beyond doubt is problematic and i fear that questions will inevitably always be raised against moderators. This constitutes our first problem.
1. How do we recruit and vet our moderation team?
I also have questions about the previously used system of investigation. In short the system of investigation as i understand it (get ready for it to be wrong..) is that a players games are analysed for match ups with 1st 2nd 3rd choice engine moves (opening moves excluded). A percentage match for each move is found for the game. This is then repeated over many games until an average is found and a player exceeding a certain percentage match up over 20-30 games is considered dishonest.
Now, while i completely accept that a player matching up in this way should be considered highly suspicious, i do feel that it is somewhat unproven (categorically speaking). Has any genuine scientific research been done that shows this method is sound at finding engine cheats? What number of games is considered a reliable sample?
I don't want to pour scorn on this system, i'm just trying to encourage debate on the subject. Is this the best we can do? So..
2. Do we have a reliable method of detecting cheats?
Also, what sort of response do the community expect the site to take with regards to engine users? Someone pointed out to me the other day that chess.com have a forum dedicated to discussion of cheating. When players are removed they are listed in a hall of shame.
RHP started with a similar list of banned players, but it was removed (as i understand) because admin got concerned about this in some way.
But are these the only options open to us? Should we automatically ban engine users? Does that really change anything? As i understand it, banned players were reimbursed their subscription when they were banned (please correct me if i'm wrong here). It doesn't take a genius to work out that there is nothing stopping these players resubscribing and starting the whole process all over again.
Perhaps it would be simpler to have a facility that allows people to declare themselves an engine user. This would bar them from playing people who have declared the do not wish to play an engine assited opponent. Tournaments are engine free (of course) but other areas of the site could be relaxed to accomodate people who choose to accept engine using opponents, or wish to use one. Consequently, when people are found to be using an engine 'illegally' their account is then switched to the engine settings permanently.
Why do this?
Firstly, instead of forcing an engine users hand by banning them, you make it their choice to leave or stay. If they set up another account, they have to resubscribe (which is money for the site) while the site doesn't have to feel all guilty about booting them). Also, has anyone ever considered that perhaps some engine users would perhaps do it genuinely if given the option? Until it is tried we'll never really know..
3. Would making engine use permissible (with certain restriction), give the site a better chance of keeping a pool of genuine players?
Also, it strikes me that in the process of analysing people to establish if they are cheating, the game mods must have analysed many players who they found not to be cheating. When i genuine player loses to an engine, their 'match up' stats could also be used as a stamp of authenticity. If a player has 20 -30 games analysed and their stats look reasonable to the moderation team, why not give them some sort of verification to that effect that lets their opponents know they are a genuine human opponent? When a new player arrives at the site and looks at the player tables for the first time and sees that all of the top 20 (say) have a ribbon showing they have been cleared by the moderation team, it send a clear signal that game moderation is used on the site and that cheats are actively sought out. Personally i think most players would find receiving such a clearance as quite an honour and it would be something to aspire to. Rewarding fare play should, in my opinion, be acknowledged and held up as something to aspire to.
Anyway, i probably shouldn't go on and on indefinitely. What are peoples thoughts/suggestions?
Originally posted by MarinkatombI think at least 100 games should be used in the average and at least 90% of the 1st choice moves of the engine should be exceeded before anyone should be considered as an engine user.
With the demise of the game mods and the constant on going grumblings of the community as a result, i thought it might be interesting to have a discussion about possible alternative ways of dealing with this issue.
From what i can gather/remember, the game mods were disbanded, in part, because members of the moderation team were themselves accused of ...[text shortened]... robably shouldn't go on and on indefinitely. What are peoples thoughts/suggestions?
Originally posted by MarinkatombAs i understand it, a game mod at the time was banned for cheating, but that wasn't the reason the game mods disbanded. It's because the site's admin weren't banning even the most blatant cheats so the volunteer game moderators packed it in.
With the demise of the game mods and the constant on going grumblings of the community as a result, i thought it might be interesting to have a discussion about possible alternative ways of dealing with this issue.
From what i can gather/remember, the game mods were disbanded, in part, because members of the moderation team were themselves accused of ...[text shortened]... robably shouldn't go on and on indefinitely. What are peoples thoughts/suggestions?
What needs to be done is simple, do what was done before. It worked, it's what works over at chess.com, but this time ban the cheats.
Originally posted by Dewi JonesUsing only 20 specially picked games is just not enough to make a good decision. Also using 3 or 4 choices for the person to match as being right is not the way a true test is given. Even if one is given 4 choices on a test, they are always graded wrong unless it is the best choice. Anything below 70% or C- should not be considered passing.
Of course you do.
Also many people pass tests with higher than 70%. Now, if someone passes a difficult test with at least 90% or A- there might be a possibility of cheating, but it still would not be a sure thing. But I would not be opposed to banning players that can play that well for the benefit of the majority of the players, even if those high scoring players are not cheating.
But I am also not for dragging their name through the mud. It is good enough just to ban them from this website.
- Joined
- 25 Jul 07
- Moves
- 27755
Originally posted by MarinkatombI've often thought that checking for top 3 engine matches is the wrong way round. It would be very easy to cheat by using an engine as a blunder-check, where the player decides on a move in meat-space, then runs it through silicon-space just to check the tactical ramifications. A top-3 match won't necessarily catch this, as the original move the player thought of might not necessarily be in the top 3 .. but it equally might not be all that bad, so the blundercheck passes, so the player makes the move.
... In short the system of investigation as i understand it (get ready for it to be wrong..) is that a players games are analysed for match ups with 1st 2nd 3rd choice engine moves (opening moves excluded)....
In order to stop this kind of cheating, you'd need to check the number of moves that a player makes that significantly *decrease* the evaluation. I haven't checked, but around my level (1800) I'd say we're making 4 or 5 of these per game, with maybe one or two big decreases. Obviously as the player level increases, the number and magnitude of the decrease events gets less. Does it ever get to zero in top quality human play? I don't know .. but it would be quite easy to find out with a large enough sample of games.
I might even try to knock something up in Python to do this .. it would be quite a fun little project.
Originally posted by aquatabbyI would encourage you to do yo.
...cut...
I might even try to knock something up in Python to do this .. it would be quite a fun little project.
The point is in my opinion that it is impossible for a machine to decide what typical moves for machines and humans are. (otherwise humanity could be faked 🙂) This is evident when artificially decreasing the potential of a machine. It doesn't blunder humanlike.
Another point to consider in my opinion would be to see how much the first three (or four) choices differ. As I understand the calibration uses correspondence grandmaster games of before the advent of strong computers. And a 90% probability of choosing the first move can't be a regular occurence. However if I would cheat (everybody is invited to check my games) I would look at the first three moves and choose the third one if it is only negligeable under the first to mkae sure my 1st move match-up is not so high. (And I don't think I invented this, it is clear to all of us that there is no assurance whatsoever to catch every cheater. A good cenatur is probably very hard to catch, as are the aforementioned blundercheckers.)
Originally posted by PonderableUnfortunately we're not going to be able to catch every player who cheats. But at least with the top 3/4 match up method we would be able to catch the blatant cheats. That would be a start.
I would encourage you to do yo.
The point is in my opinion that it is impossible for a machine to decide what typical moves for machines and humans are. (otherwise humanity could be faked 🙂) This is evident when artificially decreasing the potential of a machine. It doesn't blunder humanlike.
Another point to consider in my opinion would be to see ...[text shortened]... r. A good cenatur is probably very hard to catch, as are the aforementioned blundercheckers.)
Originally posted by MarinkatombCompare how other sites have approached the problem of trust.
The ICCF, as far as I know, have not officially banned nor allowed engine assistance. I don’t believe this is because they or their members generally like the idea of engine assistance. I think it’s because they had to admit to no way of enforcing a ban. This is not ideal but it’s at least accepting some reality.
Or consider how Ebay or Amazon try to identity reputable sellers based on feedback following a sale. A person buys from them and then states whether they’d like to do so again, making the stats public for future potential buyers. The stats don’t ban a seller but could make it harder for the seller to make further sales.
For RHP, this would mean that we don’t try to declare someone as a cheat as such, but rather more simply whether people enjoyed playing against them. This could include suspected cheating but could also be because of abusive messages, etc. So, what does this then mean for tournaments where it’s an all-play-all scenario? I can’t comment on clan matches since I’ve never participated.
I don’t think there’s a easy answer. Maybe tournaments is where it falls down. But what if some, not all, tournaments required a certain reputation threshold to enter?! Or what if just prior to starting a tournament, a player could be voted out if more than e.g. 50% of the other players had previously gave them a bad reputation. The exact thresholds would require careful tuning. Maybe there would be cases of some genuine player becoming unfairly victimised with a bad reputation but at least that is down to the RHP community making a decision, albeit getting it wrong.
Anyway, a perfect answer doesn’t exist. RHP has to decide on the best compromise, just like the ICCF did, though I’m not suggesting RHP chooses the same compromise.
Originally posted by aquatabbyPerhaps we should just ban players that exceed a rating on here of 2400 or whatever number is determined too high for a normal RHP player to achieve on his own.
I've often thought that checking for top 3 engine matches is the wrong way round. It would be very easy to cheat by using an engine as a blunder-check, where the player decides on a move in meat-space, then runs it through silicon-space just to check the tactical ramifications. A top-3 match won't necessarily catch this, as the original move the player tho ...[text shortened]... try to knock something up in Python to do this .. it would be quite a fun little project.
I think my main point was this, does banning even work?
Controversial, yes. but even when the game mods were in full flow, they never even got close to removing the blatant cheats!
Personally, think we should consider validating the human players. Yes, this IS a human player! When you have a high score table with validated players up against a '95% win, one draw and no loss' player, it just puts a massive spot light on the BLATANT cheat...