Where are these data from?

[onomastikon]

I have been told that the average number of Amnesia scrolls one finds in a 3-rune game are 14. Since I am now in my fifth game in a row with 4 runes, in each game of which I have found less than 4 scrolls, I feel perplexed. I do not wish to doubt the power of RNJesus, nor do I wish to whine, I would like to know, however, the source of the data that has been given. What, exactly, has been averaged, by whom? Or is this from the source code? Thanks in advance

[Siegurt]

I have been told that the average number of Amnesia scrolls one finds in a 3-rune game are 14. Since I am now in my fifth game in a row with 4 runes, in each game of which I have found less than 4 scrolls, I feel perplexed. I do not wish to doubt the power of RNJesus, nor do I wish to whine, I would like to know, however, the source of the data that has been given. What, exactly, has been averaged, by whom? Or is this from the source code? Thanks in advance

There's a thing called "objstats" which basically lets a dev build a special version of Crawl that runs through all the item generation stuff and reports back totals/averages etc. across a large number of generated games, this takes a long time, so they don't do it very often (Plus you have to select like, what branches appear, and what portions of the game are included, so it's not like a super automated process)

One of the devs (Gammafunk) takes it upon himself to run those stats periodically, and upload them to a google docs spreadsheet here:
https://drive.google.com/drive/folders/ ... WhLVWRXbG8

That has all the data from the passes He's done to generate statistics

[damerell]

I have been told that the average number of Amnesia scrolls one finds in a 3-rune game are 14.

It's worth noting that that 14 includes 1 in Zot (where you might not have been yet), the assumption you can go in and clear every portal branch you find (about another 1 total), 0.7 of a scroll in Crypt, 0.9 of a scroll in Elf... so while you may have had poor luck (or have uncleared exclusions / runed doors?) that headline figure of 14 can be a bit misleading.

[onomastikon]

Thanks. I usually do clear all those branches first before getting further runes, so: sadly not part of the mystery.
Currently on a character with 6 runes now, and again have found a total of 3 amnesia scrolls, decided to try my zig now in the hopes of getting a few, currently on lvl 17 and gotten buckets of rarer things but not a single amnesia. Not a sausage. I must be dragging the statistics way down to the cellar!

[onomastikon]

I understand the principle of RNG being R, but I am beginning to doubt the data are coming from the same source.
Current game, I found the only amnesia at the bottom of E3 in the big vault, it was the last scroll unidentified (already got torment and acquirement before). Have 2 runes, did the standard D to 15, L to 6, both lair branches, O to 2 and E to 3 and V to 4. This is 0.23.2

[delarado]

14 sounds high to me.

I don't have a whole load of wins under my belt, but when I do win, I'd say I get an average of about 3 or 4 amnesia scrolls.

[Siegurt]

14 sounds high to me.

I don't have a whole load of wins under my belt, but when I do win, I'd say I get an average of about 3 or 4 amnesia scrolls.

You probably get 3-4 *relevant* amnesia scrolls, how many do you ignore (in shops and vaults and zot and what not) after you no longer care about them?

[Sorcerous]

I can understand the disbelief when a game decides the first ever blink scroll (regular and portal branches included) appears in U:3, and games where a grand total of 4 or 5 amnesia scrolls are found outside of procedurally infinite places like Abyss. I've certainly experienced both those, and games where a read-ID result on the largest stack is definitely not what I expect (5 immolations in D:4??).

This being said, why are we making this into a thread laden with observer bias? Even if the distribution has a (possibly large) mean area and we've all experienced the outliers at some point, why discuss those when the code for item generation is readily available? What I'm saying here is that we should find and take some of the relevant DCSS code for item generation, and then see where the noise comes from: if the scrolls of amnesia/ remove curse/ blink/ whatever else "feels" sparse compared to the expected mean result - what is generated instead that makes the numbers strange? There has to be a higher number of something else that compensates (and obscures) the numbers we can reasonably expect to see. There is something in the same item "bracket" as the amnesia scrolls in this case, and that makes us say "not enough" for one and keeps us from noticing more of the other(s).

[Siegurt]

This being said, why are we making this into a thread laden with observer bias? Even if the distribution has a (possibly large) mean area and we've all experienced the outliers at some point, why discuss those when the code for item generation is readily available? What I'm saying here is that we should find and take some of the relevant DCSS code for item generation, and then see where the noise comes from: if the scrolls of amnesia/ remove curse/ blink/ whatever else "feels" sparse compared to the expected mean result - what is generated instead that makes the numbers strange? There has to be a higher number of something else that compensates (and obscures) the numbers we can reasonably expect to see. There is something in the same item "bracket" as the amnesia scrolls in this case, and that makes us say "not enough" for one and keeps us from noticing more of the other(s).

I'm just saying that the "feels sparse" in this case *is* observer bias. They are exactly as sparse/common as objstats reports, because objstats literally generates a bunch of games and counts the number of scrolls.

There are other objects that are equally rare, and amnesia was brought up because in this case someone wanted one and didn't have it (I mean would you care/notice if you had a few extra of something you didn't need?)

What makes them "feel" sparse is that humans are bad at estimation across a large number of results, we tend to remember outliers, rather than actual average results, and we tend to remember "bad" results more than good ones.

So some more number beyond the mean which might help here:
The average amnesia scrolls across all levels in 1000, 3 rune games is around 14.6
The minimum was 4, the maximum was 27
The average/min/max scrolls generated in the *dungeon* branch was: 3.69/0/11
The average/min/max scrolls generated in the *lair* branch was: 1.36/0/7
The average/min/max scrolls generated in the *orc* branch was: 0.36/0/5
The average/min/max scrolls generated in the *vault* branch was: 1.91/0/8
The average/min/max scrolls generated in the *vault* branch was: 1.51/0/8

Some things you can see here: the average is consistently closer to the low end than the high end, meaning we don't have a evenly-distributed bell curve, we have a high-ish number of games with a low number of scrolls generated, with a long tail (a graph of distrbutions would be fatter on the low end than the high end) therefore it's very likely that if you look at the number of *games* (rather than the number of *scrolls*) you'll see a large number of games with scrolls generated in the lower range of numbers, and comparitively fewer games with numbers on the high end of the average.

This is probably why "mean number of scrolls" isn't the the most intuitive quantifier, what you really want is "median number of scrolls per game" (assuming a fairly continuous distribution, which I believe this is) That number isn't reported by the statistics gathered, but would probably be more in tune with what one would intuitively expect.

[Sorcerous]

So some more numbers beyond the mean which might help here:
The average amnesia scrolls across all levels in 1000, 3 rune games is around 14.6
The minimum was 4, the maximum was 27
The average/min/max scrolls generated in the *dungeon* branch was: 3.69/0/11
The average/min/max scrolls generated in the *lair* branch was: 1.36/0/7
The average/min/max scrolls generated in the *orc* branch was: 0.36/0/5
The average/min/max scrolls generated in the *vault* branch was: 1.91/0/8
The average/min/max scrolls generated in the *vault* branch was: 1.51/0/8

Some things you can see here: the average is consistently closer to the low end than the high end, meaning we don't have a evenly-distributed bell curve, we have a high-ish number of games with a low number of scrolls generated, with a long tail (a graph of distrbutions would be fatter on the low end than the high end) therefore it's very likely that if you look at the number of *games* (rather than the number of *scrolls*) you'll see a large number of games with scrolls generated in the lower range of numbers, and comparitively fewer games with numbers on the high end of the average.

This is probably why "mean number of scrolls" isn't the the most intuitive quantifier, what you really want is "median number of scrolls per game" (assuming a fairly continuous distribution, which I believe this is) That number isn't reported by the statistics gathered, but would probably be more in tune with what one would intuitively expect.

I thought the distribution would be biased more closely towards the middle values. This is very different, and the numbers you provided (or rather, the generator did) can show very clearly that the observed number of scrolls is in the lower range compared to the maximum values. In fact, they did average out to less than a quarter for all except D: itself. Better yet, this almost exactly coincides with the first post - "about 14" falls down to 4 or so in observed games, backed by at least three of the posters in this thread alone. This makes me feel content for some reason

[onomastikon]

Hence my original question regarding the source of the data (if the generation for the data was taken from a source which is not identical to the source which generates items in the application I am using).
I am not good at math, but from the very little I understand, the way that numbers are generated randomly might make the "average" (not the median) different for different circumstances.
For example, it seems (and this is obviously observer bias, thus "seems") that many games tend to generate similar items; when I find a potion of degeneration, I frequently find many more in levels near, and less other things. That makes me wonder if item generation is somehow type-related.
Currently, I am playing a game and am in Lair:6 and have not found a single ring, of any type, anywhere. (There was only 1 shop and it was not of jewelry). Checking my autopickup, it confirms that I have never seen a ring. I know this is an outlier. It feels odd (whatever "feels" means) to experience DCSS as having "more" "outliers" than other loot-based games of this nature. Hence my original question, and the question regarding the type and quality of the RNG.

[Siegurt]

Hence my original question regarding the source of the data (if the generation for the data was taken from a source which is not identical to the source which generates items in the application I am using).
I am not good at math, but from the very little I understand, the way that numbers are generated randomly might make the "average" (not the median) different for different circumstances.

What you appear to allude to is bias from a poor RNG that generates numbered clustered around certain results, there was a particularly notoriously bad "default" random number generator included with some popular programming languages a while ago, which has resulted in a sort of myth that RNG generate patterns of results. The ones in use today don't generate predictable discernible patterns. The "better" the RNG generator, the less predictable the random numbers generated are, but even the worst of them wouldn't generate clusters of results like you seem to allude to.

For example, it seems (and this is obviously observer bias, thus "seems") that many games tend to generate similar items; when I find a potion of degeneration, I frequently find many more in levels near, and less other things. That makes me wonder if item generation is somehow type-related.
Currently, I am playing a game and am in Lair:6 and have not found a single ring, of any type, anywhere. (There was only 1 shop and it was not of jewelry). Checking my autopickup, it confirms that I have never seen a ring. I know this is an outlier. It feels odd (whatever "feels" means) to experience DCSS as having "more" "outliers" than other loot-based games of this nature. Hence my original question, and the question regarding the type and quality of the RNG.

The RNG in crawl itself is high-quality, but portable, such that the RNG will produce the same numbers on any platform, given the same seed, it doesn't produce predictable patterns and isn't weighted towards any particular set of results.

What you're seeing is the combination of unconstrained randomness and LOTS of things being random. Most games impose strict limits on the randomness of the results "You must generate between X and Y rings in between levels 1 and N" kind of stuff, where the results of some rolls actually has an impact on later ones (limits make for easier game balancing, but less variance). DCSS doesn't have nearly as many limits on what it can produce, and there's a WHOLE LOT of things that DCSS generates randomly, that gives you a very large space in which to search for patterns, naturally if you randomly generate enough things, a pattern will naturally emerge somewhere.

I had a game recently where I did all of D, lair and orc and didn't have a single rPois item drop, it made spider much more annoying. That's just because I happened to be *looking* for the "no rPois" pattern, that's how randomness works, the odds of that happening are low, but given *enough low odds patterns to look for* one of them will happen in many games.

Human brains are high-quality pattern-identifying machines, our brains are hard-wired to pick out patterns and identify them, so when a pattern does come up, it sticks out, even if it was randomly generated.

*BUT* one of the things that DCSS *also* has is the ability to randomly pick a pattern for a level, for example, there are "shop" levels where the odds of having a shop are much higher, you'll typically get 3-4 shops on such a level, there aren't such patterns that span across levels (that I know of) however there are both pre-set content (vaults) and a few "hey this level will generate according to this slightly different set of rules" types of things. However, that doesn't explain your "feelings" (certain potions come up in large batches in certain vaults, but that doesn't impact the generation of anything anywhere outside that vault)

Without some amount of foreknowledge it's not always going to be possible to know if the pattern your spotting is a "naturally occurring, randomly generated" pattern, or a pre-determined one (other than the pre-determined patterns are all level-specific, and frequently area-of-the-level specific)

[onomastikon]

Thank you very much for taking the time to articulate all of that.
I suppose I am really not used to SO much randomness, presumably my perception has been influenced by my years and years of playing other games of the ARPG type, which must obviously have something like minimums and conditionals in the item generation. I have probably only played a bit more than 100 games (surely not enough to base real statistics upon), and (since I really like casters and hence pay more attention to how many amnesia scrolls I can get) I doubt I have ever found 14 scrolls of amnesia more than once. So be it.

[TheMeInTeam]

The "better" the RNG generator, the less predictable the random numbers generated are, but even the worst of them wouldn't generate clusters of results like you seem to allude to.

Yeah. You only get stuff like this if the devs *intentionally* add some factor to bias outcomes. Example would be troves only generating items your character can possibly use, or similar for Oka/Trog gifts. For some species you will see a strong bias in gear gifts (Og, Te, Sp for example)...just aren't as many possible slots.

It would be extremely odd to picture a dev programming clustered degen pots or something. That's non-trivial amount of work done just to mess with people, and shouldn't normally be the conclusion/suspicion. People don't usually do extra work for no or self-harmful reasons! It's possible in principle but it's not a likely guess.

As for crawl, even in the context of same-seed RNG is pretty apparent. UV4's GOTM has some pretty significantly different outcomes, which we would not expect if RNG was clustering/doing something like alluded to in this thread.

I do think it would be useful to constrain some kinds of variance in crawl, but it's not an easy thing to do well/manage w/o detracting from its decisions.