Lately, I’ve been quoting from last year’s survey results left, right and centre, and seeing how we still don’t have a nicely linkable version online, I guess it’s high time to change that.
Around the 0.5.0 release in early August 2009, an innocuous discussion on IRC turned into an impromptu poll about Stonesoup’s player base. The survey was subsequently ported to various forums, hosted on CAO and pointed out in the mailing list.
Between August 17 and some time in early November, we received a total of 274 replies to the following 12 questions:
- What is your age?
- What is your country?
- Do you play locally, on a server, or both?
- Do you play Tiles, ASCII, or both?
- OS(es) at home?
- Roguelikes played before? (NetHack, ADOM, etc.?)
- Where did you learn about Crawl?
- And when?
- How many Crawl wins? (If none, you may specify your best game.)
- If you take part in the tournament, where did you hear about it?
- Ever recommend Crawl?
- Which computer game have you played most in the last month (July)?
The results, as posted once we had 250 replies, can be found here.
In November, I presented the complete results at the International Roguelike Development Conference in Geneva.
In the rest of this post, I will attempt to summarize the most interesting findings, but leave the more detailed numbers and in-depth discussion to the two files linked above.
The average Dungeon Crawl Stone Soup player at the time of the survey (or at least the average survey participant) can be described as follows: He (94% likelihood of being male) is between 25-29 years old (31%, average age: 27.3; most common age: 25) and lives in the USA (54%). Even if he doesn’t, he’s very likely to speak English as a primary language (75%). He uses Windows XP (52% Windows, of these 63% Win XP) at home, and only plays Crawl locally (54%) because he prefers Tiles to ASCII (60% if playing locally, 38% total). This also means that he did not take part in the tournament (39% participants total, 23% Tiles players).
He’d probably never even heard of the original Dungeon Crawl and only started playing after Stone Soup came out (65%). Crawl is not his first roguelike (91%) and he’s very likely to have at least tried NetHack before (73%) and maybe also ADOM (40%), Angband (28%) or one of the smaller roguelikes (e.g. Rogue, DoomRL, Dwarf Fortress, POWDER, various Angband variants, etc.)
At the time of taking the poll, he hadn’t won Crawl yet (62%) and probably hadn’t even found his first rune yet (79% of non-winners), but he nonetheless claims to have recommended the game at least once (83%) and also lists Crawl as the game played most “within the last month” (53% total, 27% if discounting tournament participants). Though, to be fair, hardcore players are more likely to take part in such a survey, so we should take all results with a grain of salt. Still, thank you! :)
There’s an interesting rift between ASCII players and Tiles players when it comes to percentage of winners (49% ASCII, 25% Tiles). However, interpreting this finding to mean that “ASCII players are better players” would be overly hasty. For one, Tiles players tend to be relatively new to the game (Tiles: 72% joined since Stonesoup, 34% in 2009; ASCII: 61% joined since Stonesoup, 20% in 2009) if not the genre, whereas players who knew Crawl from before tiles were introduced were already used to the console “graphics” and might have seen little reason to switch. Second, and more importantly, there’s a much stronger correlation when comparing local vs. server play (25% local winners, 68% online), possibly because players on the server are more likely to use the ##crawl channel to ask for playing tips and can receive advice from more experienced players watching them. The survey also showed that many players felt they “weren’t good enough” for online play or the tournament, so it might be that the more experienced Tiles players eventually switch to online (ASCII) play, even though they would prefer a way to continue playing the Tiles version online.
In general, Crawl’s popularity has increased significantly once tiles had been added. Downloads jumped from 2,472 and 4,321 in total (source and various platforms included) for 0.1.7 and 0.2.7, respectively, to 10,374 downloads of the Windows Tiles version alone in 0.3.4. (All numbers taken from SourceForge on March 31, 2010.)
A few conclusions
The impact of language is much stronger than we expected. It would be interesting to compare this effect with other games which rely a lot on text (e.g. MMORPGs which have not been translated or text adventures, i.e. interactive fiction).
A good many players have known the game for years, yet have never won. This is important for design purposes: adding features for the early game (e.g. new gods, shallow vaults, etc.) will be appreciated by those players.
The Crawl tournament would benefit immensely from tiles play over the internet. While some players oppose Tiles on principle, we also got a lot of praise for the quality of Crawl’s tiles and interface; for example by players who said that they play other roguelikes in ASCII, yet Crawl with Tiles. As developers, we plan to keep supporting both modes. Each of them has advantages of its own and we will continue to improve their interfaces.
1. Comment by Ostsol
4/Apr/2010 at 22:25
Crawl and Powder are the only big roguelikes I could really get into — probably because they’re pretty light on stats, unlike Nethack or the like which emulate a more D&D-style statline. This makes building your character less of a matter of juggling stat and more a matter of just having fun.
2. Comment by garron
6/Apr/2010 at 04:08
Hey, someone else played Fatal Labyrinth!
3. Comment by Anonymous
7/Apr/2010 at 18:49
>(94% likelihood of being male)
Wow, we’re a sausagefest.
Are there any plans to do another survey at some point?
4. Comment by Pratama Wirya
10/Apr/2010 at 19:49
94% likelihood of being male…. Is it that masculine, to die over and over again in a game like this? Wow!
5. Comment by Rugxulo
14/Apr/2010 at 06:19
Somebody’s stats are incomplete. If you only survey < 300 people, and only 2 use the DOS port, then you discontinue the DOS port. (Ironically, only 1 used Win2k.) And yet Sourceforge shows 2903 downloads for 0.5.2 for DOS (and twice that for 0.4.5). I know I downloaded it a handful of times, but I don't think that covers the rest of them. So somebody's wrong. (And still no 0.6.0 DOS package .ZIP online despite me putting it on a silver platter for you over a week ago.)
6. Comment by Phil Martin
19/Apr/2010 at 04:43
Have been playing crawl on and off for years, until a couple of years ago, just a very early version of Linleys that my brother downloaded, must have been in the nineties sometime. Discovered tiles and loved it as my enormous fingers led to constant key errors. Anyway, the win stats make me feel heaps better about my crawl acheivements thus far, never got close to finishing it, but at least I’ve found a few runes. And as for those ASCII purists, they sound like the same kind of people who make bizarre claims of vinyls superiority over CDs. Go the tiles.
7. Comment by nutnarukex
24/Apr/2010 at 14:08
wanna have online tiles version seriously ><
I completed the game once but I want to share my exp / rank with online profiles
8. Comment by jpeg
27/Apr/2010 at 15:55
@Rugxulo: Sorry, I didn’t hear about you (or anyone else, for that matter) offering a DOS binary. Personally, I don’t see any problem with hosting it or at least linking to it, as long as we make it clear that we don’t intend to spend any time fixing DOS specific bugs. Thanks for offering!
For the record, we stopped supporting DOS because none of the team uses it natively and there were some serious bugs consequently none of us was capable of reproducing, let alone fixing. In previous discussions, I got the impressions that most players of the DOS version actually have Windows as a OS, but used the DOS version because it was faster or something, so it’s no wonder these numbers differ wildly.
@Anonymous: Yes, we’ll be running another survey at some point, though probably with different questions and more pre-planning. :)
9. Comment by rlbond86
30/Apr/2010 at 06:59
You guys should have asked about vi-keys vs. numpad. I’d be interested to learn the difference in popularity.
10. Comment by MDFi
1/May/2010 at 02:12
As one of the majority of players who have never won, I wholeheartedly support whatever bones the devs want to throw our way. I’ve played hundreds of games through the past several releases and it seems to me the game gets incrementally more difficult with each release. In the latest version, I’m posting far fewer points per game, dying far more times in less than a few minutes, and can’t figure out why an unarmed, unarmored human slave consistently inflicts 30-40 points of damage or outright kills my fully armed, armored dwarf fighter. It seems more and more like a roll of the dice for that critical early game armor or weapon and less a thinking man’s roguelike. Why not just through 2-3 Sigmunds on level 1 and call it good?
11. Comment by Aquilifer
6/May/2010 at 16:39
As another one of the majority who’s never won a game, I sort of agree with MDFi. My biggest concern is that the devs might listen too much to the obsessive minority with inhuman talent for this game. These are people who can beat nethack, ADOM, even IVAN – making changes to suit their skill level and playstyle may risk alienating the rest of the players. For example, the Shoals is nice, but seems more difficult than Swamps and Snake Pit. Additionally, the AC change is really a big deal for less skilled players, but if only the most skilled players are the ones who talk in IRC and post on the devwiki, they’re likely to not make much noise about it.
Additionally and on a completely different note, could Jivya be a starting god choice for Chaos Knights?
12. Comment by dpeg
9/May/2010 at 12:04
We are much less influenced by player opinions than you may expect. The desire to nerf something is, generally speaking, our approach to balance. Needless to say, this also goes for buffs. (Some random examples: mephitic cloud is overpowered, so needs a nerf; as does haste, both the player and monster version; the Traps & Doors skill is too weak beyond some point, so needs a buff.)
Our goal is not, and never has been, to make the game unwinnable apart from the select few. We stand by the goals of “anyone can win, unspoiled”, yet “no-one can win reliably”.
It was a plain fact that heavy AC was overpowered. The nerf was long ordered, we took some time to finally get there. It turned out we overnerfed a little, so we’ll keep adjusting things in the future. 0.6 also showed that high EV is quite overpowered (something insightful people warned us about in advance), but you can use Ha or Sp instead of MD in 0.6.
Finally, the game getting harder with each version. This is objectively wrong: Crawl became generally easier with all releases up to 0.5 (generally, because each version contains many changes, both nerfs and buffs). It was always understood that certain nerfs would arrive later to reset balance.
13. Comment by KiwiStalk
29/May/2010 at 12:59
I find it funny that I’m (more or less) in your most common surveyed player base. Male, 25, USA, English, playes Roguelikes before (Nethack, ADOM, AND Z/Angband), prefer Tiles, haven’t won yet nor got my first rune, never heard of or played the orig Crawl, and use Windows (yay 7!). I just recommended the game to my wife and brother too. :P
I usually don’t think of myself as a statistic, but hot damn am I one right now! Oh well, glad I stumbled upon Stone Soup. Very user-friendly and feature-rich for a Rogue game, which is probably why you guys get lots of newbs and non ASCII-level hardcore players.
14. Comment by MDFi
4/Jun/2010 at 02:34
Thanks for responding dpeg. As a regular player I really appreciate the time and effort the devs give to all this, and as a software developer myself I understand all that entails.
I’m curious how the devs measure balance. You mention nerfs arriving later to reset balance but the vast majority of players can’t win. What metrics, if any, are being run to determine how a particular nerf is applied to reset balance? From my perspective, I understand the big picture vision of the game and the need to coherently balance different facets of the experience, but going from 0.5 to 0.6 meant a quantitatively significant reduction of performance in my games.
Perhaps an idea is to apply a difficult setting or some such, rather than trying to make the game balanced for everyone. There’s a clear difference between players who can win and those who can’t – why not let players adjust settings to find a pleasing balance? Presumably a major portion of the game works in such a way that particular variables could be set by players at the outset of a game.
15. Comment by dpeg
6/Jun/2010 at 19:25
MDFi: It is extremely hard to define “balance”. We don’t use any formal approach. Mostly, we are concerned with addressing what we perceive as the most glaring _im_balancies. This is often spotted simply by watching players. Some examples from history:
1. When a few players would use the staff of earth (back in 0.4) for its obscene melee damage potential, it was obviously unbalanced but not urgent. We wanted to wait with a change until we overhaul staves (which will come eventually, including randart staves and ego properties for them). When many players announced (in ##crawl) that they’re “just starting another GnEE, hoping for an early SoE”, it became urgent, and we invented an ad-hoc nerf for the enhance staves.
2. I have seen the brokeness of the Tomb of Dorokhloe spell (you can still read the discussions in old posts to the r.g.r.misc newsgroup). DCSS has tried to balance the spell, for example by raising the level. Finally, I pushed for removing the spell altogether: the effect is a great idea, it should just not be a _spell_. We delegated it to a card effect, which seems better. (Nemelex is overpowered, too, but that’s a different topic.)
3. Selfbanishment (up to 0.4 IIRC via spell, after that via distortion weapons) was overpowered: you’d get an _instant_ late-game (when Abyss would pose no serious threat anymore) life saver. On top of that, proper usage is incredibly tedious: approach target destination (e.g. a hell end), self-banish if something goes wrong, repeat. One approach would be to make selfbanishment costly but I pushed for removing the self-version of the spell altogether. (It is good for Lugonu, of course.) This nerf came before the abuse potential was fully realised by the player base, I think. But it was obvious to me, so we could nerf anyway.
4. Up to 0.5, melee with heavy AC was too strong. There was a reason why players would call MDFi broken. This was a long-standing issue and we promised to do something about it. We did not manage to deliver the nerf for 0.5, but we had one for 0.6. It is _not_ a problem that melee in heavy armour is now harder than before. If you wouldn’t feel the change, we would have undernerfed. The question is if heavy armours play about correctly now. This is very hard to judge (also for us).
On measuring “success” or “balance” as a whole: I don’t think it is a good idea to use “average number of wins” or something similar. Crawl is supposed to be hard — this is also part of the appeal for players who have never win (and perhaps never will). The Nethack threads are full of this, and last year’s poll also underlined this point.
The (harsh) difficulty has some rather enjoyable corollaries: a), players of very high skill can still be differentiated; b), replay value increases (especially with the many approaches available); c), even the progress to your second rune (say) is meaningful.
I am certain a difficulty section would mess with these without really improving matters. We try to nerf certain features because (vastly) overpowered mechanics are just no-brainers. And we boost others, because not-using something that is plain too weak is just a no-brainer itself. That’s all we are doing.