Experience System

Here is a table showing the skill points needed for gaining skill levels.

It is more or less regular up to level 14. There are some irregularities, which doesn't make any sense, but they are quite minor. On the other hand, starting at level 15, the cost suddenly doubles and then doesn't increase anymore. It doesn't help that it is also about at this stage that it becomes useless to increase some important skills (many weapon skills and charms for example). This scale pre-dates stone soup. I looked at stone soup 0.1.1, the scale was like that with comments about an older one with numbers divided by about 10, but with mostly the same progression.

This is the total progression for skill cost levels. This has some important and unexpected effects, aside from the obvious of making all your skills more expensive. Spellcasting, INV and EVO have an additional SCL multiplier of SCL/10 (min 0.5).

And it fixes at 4000/lvl from there on out.

This has all kinds of weird problems associated with it. If, when you've got ~2700 SP accumulated in desirable skills (primary skill is 6, some other skills are 2 or 3), and you've also got Armour, Stabbing or Dodge trained with about 150SP (which has no benefit at all, isn't even visible), this'll push you up to the next SCL, 150/200 of the time, then 150/350 of the time, then 150/400 of the time… the XP:SP penalty causes you to accumulate skill points about 5-10% more slowly, so eventually it evens out to some extent. — DrPraetor 2011-05-08 14:58

Whenever we are tweaking a formula involving a skill (stabbing, spell power, weapon damage,…), we are rewarding the XP the player invested in the skill by making the character more powerful. It would be easier to balance those formulas, if the skill levelling scale were more regular. Here is a proposal:

Levels 15 to 20 are cheaper, and levels 22 to 27 are more expensive. It's mostly a buff, because any skill above L14 will gain a level (a little bit also in the lower levels). The percent column represent the amount of XP invested compared to the maximum at XL27. We probably shouldn't use this number straight in formulas, but it can give us some guidance. For example, reaching L27 requires more than twice the amount of skill points than L20, so maybe reward accordingly.

Maybe this scale was set this way for balance reasons. But since then, most of the important formulas have already been changed (stabbing, spell power, armour) or need to be (weapon damage, evasion). As it is a buff (you gain between 1 and 2 levels when above L14), we'll need to balance it by nerfing the things that are already quite (too?) strong (so that at worse, they stay at the same level): evasion, spell success and maybe stabbing. — galehar 2011-01-27 17:01

For reference, some skills have a different base cost than normal skills. For a base cost of 100 for normal skills, those skills have a base cost of:

• Spellcasting: 130
• Invocation: 80
• Evocation: 80
• Stealth: 50

Can we please axe these? There are way too many variables for XP → skill level already. — eronarn 2011-04-17 02:52

It's worse than that for three of these skills (see my note on s_c_l above.) I agree in principle, but, unfortunately, “axeing” the cheapness of INV/EVO would either be a huge nerf, or a huge rewrite of big blocks of code. — DrPraetor 2011-05-08 14:58

Once the game gets going, *most* of the cost associated with increasing a skill is actually buried in the increase-in-cost from your total skill point accumulation. People don't complain about this much, because it is *extraordinarily* opaque to the player (it's not easy to deduce from the code, either.) First, we need to document the *actual experience cost* of a given heap of skills, before we talk about costs being flattened. This will require a bit of code diving. I have a couple of specific proposals:

• The total correction factor should be based more closely on the total XP invested in skills, rather than the total SP invested in skills, since the XP:SP conversion rate differs so hugely. This could be as simple as: SUM (skill level * skill points), rather than SUM (skill points). This simple change would also make my other two proposals largely moot, but in case people don't like it for some reason…
• Stabbing, Traps & Doors, and Stealth should not count towards your total SP for purposes of skill costs. This has a couple of beneficial effects: players who accidentally train Stealth or Stabbing, when they can't use them, won't be punished for it (they'll still waste whatever XP they wasted, but won't increase other costs.)
• Melee skills below your highest melee skill shouldn't count either. Right now, there is a huge hidden penalty for switching weapons in the mid-game, and people seldom notice it because it's not that bad right after you switch but gets progressively worse as you go up in levels.

— DrPraetor 2011-05-07 16:07

This system used to be based on XL but was switched to skill points because XL is gameable. Once you have gained some XP, you have to convert it to skill points. If you hadn't accidentally trained stealth, those XP would have gone to another skill and you total_skill_points would have raised anyway. Lower costs for low level skills is double edged though. It makes training them in the late game easier, but it also means the same amount of XP is going to be converted in more skill points if you train a low level skill rather than a high level one. So generalizing is easy, but it will slow down a bit your progression. So in the end, skill costs will raise a bit faster if you train many skills, I'm not sure that's a bad thing. And I wouldn't worry about accidental training. A couple of levels of stealth or dodging is hardly going to make any significant difference. — galehar 2011-05-08 08:35

I don't think it's all that small - +1 skill_cost_level tends to raise your XP:SP rate by about 20%; at *best* a level 3 skill is 1/8 s_c_l (500/4000), so ~2.5% to *everything*. That's a bigger change than your revised table makes, no? I agree that generalization slowing down your progression isn't a problem *for magic*, which is what most people generalize; an air magic of 4 is useful even with other skills in their 20s, so you should pay for it, okay. But it ends being a very nasty penalty for weapon switchers, and that 2% benefit from avoiding any Stabbing/Dodging/Stealth/Armour/Poison Magic is a big enough difference to alter behavior, and to make other things harder to balance, I'd say.

— DrPraetor 2011-05-08 14:16

The raising costs isn't small, I agree. What I meant is the effect on total skill points of training low level skills instead of high level ones is small. It will make you progress slightly faster to the next level than if you could have avoided it and put your XP in a high level skill. But in the end, it doesn't change much. skill_cost_level will raise and will eventually reach L27 for all characters. The 450 skill points from 2 levels of stealth and dodging are negligible. — galehar 2011-05-08 14:44

They're negligible if you get them *late*. Paradoxically, if you get them early, they're a big penalty that will have lasting effects. Your skills will be noticeably lower at any given future point in the game. How's about this: every class has at least four skills, yeah? So instead of SUM(SP), how about SUM(MAX(SP-250,0)), and that “first jump” is 1000 SP smaller. This means you don't pay a weird phantom penalty for skills < 1, and accidentally getting 2 points of stabbing in the Orc Mines doesn't hurt you in the mid game. On top of that, I do think that the s_c_l math should be further cleaned up and simplified. — DrPraetor 2011-05-08 16:08

Here is the source code comment about s_c_l by Peter Berger: These values were calculated by running a simulation of gaining skills. The goal is to try and match the old cost system which used the player's experience level (which has a number of problems) so things shouldn't seem too different to the player… but we still try to err on the high side for the lower levels. – bwr

Ultimately, it would probably be better if s_c_l depend on XP gained (ignoring draining) instead of SP. s_c_l would progress regularly regardless of which skills are trained. I have no idea what kind of simulation Peter ran, but I'm reluctant to blindly mess with the formula without proper tools. — galehar 2011-05-09 12:50

I once ran a merfolk stalker/melee character and realized when in my late teens or so of character level that I should've been using shields all along (by asking advice on IRC and someone noticing I didn't have any and being confused). I proceeded to grab a shield, turn off all skills except shields and try my best to train it. I cleared the entirety of Elf and a bit of other stuff and I believe I only gained a point or two (I should note that my character was already boned by this here hidden penalty because I'd made the horrid mistake of letting too many points get invested in short blades before realizing I should be training ONLY polearms; actually now that I think of it I probably made that switch somewhere around lair or a bit later. A powergamer I was not). The moral here is that the current system is way past broken in its punishment for learning new skills, especially in a game that prides itself on having no brainers; because at present, “Don't bother to learn new skills after the lair” is the no-brainer to end all no-brainers. I support any and every change to the skill system which makes it so that a character can pick up new skills late in the game and actually get them somwhere. I'm not thinking a skill you pick up at level 20 should be able to match a skill you trained all game, but it should be reasonably possible for a level 20 character who funnels all XP into a skill his race isn't bad at to get it up to half the level of his main weapon skill by level 22. So I think you should be phasing out “late in the game” penalties as much as possible in favor of “skill is already high” penalties. Increase the cost of spell schools to compensate if you have to. — Brickman 2011-05-09 00:02

In your stalker-with-shields example, are you sure your character wasn't just feeding all that xp into dodging? It still trains 25% as often even when turned off, and a typical stalker with be dodging almost every attack. If you dodge every attack before checking your shield score, your shield never gets a chance to train.
In my in-game experience, I often play casters, and my casters have some spells they want no matter what point in the game I'm at. Abjuration seems to train summoning skill to 8 or 10 pretty easily whether I get it early or late, and it doesn't seem to matter all that much whether I'm hardlining air magic for Tornado or if I've got a more generalist build. — KoboldLord 2011-05-09 01:34
I think generalizing in the late game is too easy, not too hard. I have pulled Translocations from 0 to 12 specifically to cast Controlled Blink with only the XP from Vault:8. If I'd dumped that XP into a level 20 skill, I'd get it to 21 or 22. — minmay 2011-05-09 04:09

I think that the overall difficulty of generalizing is about right, myself. Vault:8 is really scary so you should get enough XP to learn a useful secondary skill :). In regards to the problem you name - that Vault:8 only gives you enough XP to raise your primary skill by a point? Well, you're proposing to make that worse in the interest of clarity and simplicity :). — DrPraetor 2011-05-09 05:09

The current skill cost system has some great benefits. When you train low level skills early on, skill cost raise faster. But this is because you're buying cheap, so you gain more skill points. Assuming you do want the skill, it's a good thing, better buy those first points while they are cheap. Postponing will only make them more expensive until the point where they are as expensive as the ones from high level skills.
However, it does slow down the training of higher level skills, until the XP incoming rate catches up with your fast skill cost levelling. Overall it's a good thing, it means that early on, you have to choose between generalizing and specialising. You can't really do both. But if you do want to specialise, then any gain in a low level skill is going to slow you down, as explained by DrPraetor.
With the upcoming change to skill training, it will be possible to avoid training a skill by using manual mode. We don't want to force players to use manual mode, it's supposed to be a convenience, not something mandatory or strategically useful. Thus, we need to allow players to avoid being penalized for learning a new skill. I suggest we do the following:

• Full skill disabling. A disabled skill will not be trained at all.
• No penalty for learning a new skill. The first 200 SP don't count for total skill points.
• Adjust the skill_cost_level scale to compensate. The average number of starting skills is 5, so a reduction of 1000 might be good. We'll have to check that no background start with an s_c_l > 1.

— galehar 2011-05-13 10:50

I think that would mostly fix it. Certainly, it would cure the problem of reverse-victory-dancing, which annoys me. But consider the following two scenarios:
Scenario 1: Train Ice Magic to 15, Train Staves to 15, Train Ice Magic to 27
Scenario 2: Train Ice Magic to 27, Train Staves to 15
Scenario 2 is *way* more expensive than Scenario 1; and the character under scenario 1 was probably more powerful for most of the game, since skills past 15 aren't that hot to trot to begin with.
The only way to solve this problem is to alter the skill-level vs. skill_cost_level formula to avoid having it “saturate out”.

— DrPraetor 2011-05-14 16:43

I think this is the perfect timing to push this. So if you want to comment on the subject, now would be the time. — galehar 2011-04-10 22:00
It's in 0.9. — galehar 2011-05-08 08:35

I like the fact that the costs are more consistent now, but the balance is pretty bad. Prior to the change, raising skills to 27 was questionable, now it's a downright bad idea for nearly every skill. I don't know if the skill costs can be changed to accomodate this without losing the consistency that we wanted, though. It was suggested somewhere that getting a skill to 27 give you some special bonus: for example, 27 Stabbing could guarantee a successful stab against confused/distracted monsters. I think this is the perfect time to add this. — minmay 2011-05-01 15:01

As a player, the discussion of skill XP costs here kind of blows my mind. I have won the game twice (which is by no means impressive compared to the best players, but at least means I'm no stranger to the game), and I had no idea that skill costs increased with your total number of skill levels. I think this is a big problem for a game that is trying so hard to break the conventions of its genre and be more player-friendly. As others have said above, the optimal play is to never gain any levels in skills that aren't critical to your character's survival. Trying to limit XP spent to the important skills is intuitive, but not dabbling in useful but non-critical skills is not. I might think, “Spending the XP to get a few levels of Traps and Doors is probably worth it,” not knowing that I am actually increasing the cost of other, more important skills significantly by doing so. The example above of learning then same skills in different orders having different XP costs might be even worse. If redesigning the skill/experience system further doesn't fit in the schedule for 0.9, could you maybe add some kind of feedback to the 'm' screen so players can learn how the system actually works? Maybe just a line that says “Total Skill Levels: 27, Current XP Cost: 200%” or something. — wjrogers 2011-06-03 10:45

I'm playing trunk on CDO. When I select “manual” skill training, I'm basically back to traditional RPG system where you buy points on level-up. This means I can easily train skills I'm barely using. Yes, you can eliminate victory dancing this way, but along with “train by use”. It never occured to me someone would want that.

Side effects so far:

• I can train a skill I only used once or twice. For example step into a trap once, and voila, I get the T&D skill.
• I can skip costs of training skills, for example only zap a wand once and have it train. I don't have to find a ring. For gods who don't have 0 cost Invocations, this probably allows to train the skill easily.
• I can easily train skills that would otherwise be dangerous to train, like branching from caster into a melee hybrid. Pat a monster on the head, and continue to kill with spells. Suddenly you're a decent fighter. I suppose I can also start training Summoning when I only have high-level spells accessible.

In general, there's barely any relation between what I'm doing and what skills are trained. I suppose powergamers love this already. — b0rsuk 2011-07-10 08:46

This is not a solvable problem. 'Victory Dancing' is an amusing term used to describe practice, which is tedious by nature. Many people choose to play games specifically to avoid repetitive tasks that they perform without thinking, and victory dancing is and always will be such a task as long as it attempts to be realistic. Realism itself is tedious. If it doesn't benefit the game, cut it out. — KoboldLord 2011-07-10 09:26

It is, in fact, solvable. I don't think there exists a complete solution, but successful hybrid systems have been made. Programmers may dislike the idea of having separate code to handle different groups of skills, but skills just are used differently.
For example of a successful system, you may want to try ADOM. ADOM has “skills” and “weapon marks”. ADOM separates normal skills from weapon skills. Weapon skills are only trained by use, and there are multiple limitations. For example you can only learn so much by fighting a single enemy type. While in general ADOM gives you experience for killing summoned creatures, killing the same enemy over and over stops being useful eventually. As the time goes by, an ADOM character has to fight more and more dangerous creatures to advance. He will still get xp from killing weaker stuff, but no weapon skills.
Many skills in Crawl could be handled similarly, especially more primitive ones like Conjurations. Others are inherently unsolvable, like enchantments (buffs). Without artificial intelligence, there's virtually no way to determine if player is casting Haste in anticipation of a tough battle, or to victory dance. Hmm, maybe if Enchantments were trained not by casting them, but by fighting while enchanted ? But this is research topic (or 7DR topic), not a topic for serious established RL's like Crawl. — b0rsuk 2011-07-10 10:09

Having the same cost for a skill point regardless of skill level would be good. I've made a table comparing the total cost in XP to reach a skill level at each skill_cost_level.
The new system having a fixed cost, it means that to keep some balance, the lower levels are more expensive and the higher level cheaper. So it might have the tendency to nerf generalists and boost specialists. We can also adjust the cost (in skill points) of some skill levels. I did it for the first level which costs 50 instead of 200 in the new system.
I'm sorry, there's even more numbers than usual. I tried to make it easier to read and analyse by highlighting in yellow the break point at which the new system and the old system are the closest. To reach the grey area you need to invest more than half your skill points in a single skill. The new system might be a bit too cheap for high level skills or too expensive for low level ones, but I'm not sure what I should be aiming for. If anyone has an idea of where the breaking point should be placed, please let me know.
Also, while we're at it, we might want to clean up some old hacks from exercise2(). There's one in particular that I think we should get rid of. skills.cc:475

cost -= random2(5); / / XXX: what's this for?

The comment says it all. Unfortunately, removing this hack will just push it a bit further along the same axis (increase low level XP costs compared to high level ones). Might be worth it anyway, especially if we decide to adjust the skill level costs.
To gain a skill in the current system you need 20 exercises, assuming you've got enough XP in the pool to gain 10 skill points each time. I think it's ok to reduce this number, but 5 may be a bit too low. We can cap the amount of skill points gained in a single exercise to 5 for L0 skills to compensate. Meaning you'll need at least 10 exercises to gain a skill. — galehar 2011-06-14 14:55

Update: I forgot to take into account the fact that the skill cost is the cost of 10 skill points, not 1. Instead of fixing all the formulas, I simply divided by 10 the cost of the skill levels. The skill_cost_level is supposed to try to match the experience level (skills.cc:47). With the fixed cost, I can adjust the levels so that skill_cost_level equal the XL (for a human who hasn't been drained). — galehar 2011-06-15 12:07

I've changed the skill_cost_level progression so that it progress at the same rate of a human XL. It will level a bit later depending on starting background. A healer (starting total skill points: 673) will stay at scl 1 for a while until his skills have catch up with the the total of a warper (starting total skill points: 1147). The result is a much faster progression of scl compared to the old system. I've added a comparison to the table. A faster progression might be ok, but this one seems too much. So maybe we should use a race with a slower levelling as reference. I'll try using different races and will put the results in the table.
The random cost reduction hack has a huge impact on the early game: it makes most training to have a cost of 1. I'll reduce the costs of the two first levels from 3 to 1 and 2 to compensate. — galehar 2011-06-16 11:02

I'm using minotaur (exp apt 140) as a reference for s_c_l progression.

I've written a skill training simulator in lua to compare the skill progression of the new system with the old one. Results are in this table (html version). There are 3 backgrounds compared, each time with a human, and with a typical species with good aptitudes. The fighter has few skills and trains them a lot. It also passively trains a bit of evoc and T&D. The Fire Elementalist has a bit more skills and gain new skills mid-game (charms and air) which he trains a lot. The Crusader has a lot of skills and he keeps many at low or mid-level. In the end, low level skills are a bit more expensive and high level ones a bit cheaper, but the result is quite close to the old system. I think it can go in without breaking balance. — galehar 2011-06-23 11:58