Enhancements to the sound system which I want to add

[Autofire]

Hi! I don't know if anybody uses them, but I made some enhancements to DCSS's sound system a while back. I decided to come back after a large break from DCSS and play with the sound. (I'm using my Earthbound-based sound configuration, for those interested.)

I feel like sound in a game like DCSS is really promising, because audio cues really speed things up. I don't have to scan carefully through the text log to know that I've just been hit. I also immediately know if I miscast a spell, or if my enemy (or I) resisted/dodged something. And then even small interactions (like the footsteps using stairs or the "Ka-ching!" for buying items) feel really nice and help bring the game alive.

However, this system still leaves much to be desired. Here are some of my issues with it:

• It's difficult to enable. Currently, if you want sound support in a text-based environment, you have to edit sound.h. I haven't tested it for a long time, but I think the sound system doesn't even work right when using Tiles.This is not good user experience.
  
• It largely depends on regular expressions. This is beyond terrible! For one thing, it's almost impossible to tailor sounds to certain things, like dodging attacks, because there are so many possible dodge-text strings. (Do you barely miss? Did you hit but do no damage? Do you resist? Barely resist? Easily resist?) This also sucks because this system falls apart if you try translating DCSS.*
  
• It crunches sounds all the time! It just sounds wrong to have this 3 or 4 sounds play all at the same time. It's messy, unpolished, and even dangerous. (If enough sounds play at once, they could combine and produce something far louder than originally intended.)
  
• It buries important sounds. If some powerful foe is trying to slaughter me, I care a lot more about her actions than the actions of a few lowly orcs. Thus, if she does anything major like cast a spell or attack me, I'd like to hear it! The orcs don't matter in this scenario.
  
• It limits sounds to being short ones. This one is a little weird, but we as players dictate the speed of the game. We might make several moves in one second, or several minutes to make a single move. This makes it really hard to set up any kind of fanfare or draw attention to any one event, like joining a religion, picking up a rune, leveling up, or putting on a cursed item. Sounds usually have to be short clips to avoid them spilling into many turns.**
  
• Lastly, it can be downright annoying. I have to be careful about where I put my sounds. I initially had a sound play when the user begins casting a spell. At times, it felt really good because it was a sort of a wind-up effect. However, it also gets really grating when you're casting your 30th Magic Dart, so I turned this sound off.

* I partially fixed this; for example, if you use "WIELD_WEAPON" as your sound trigger, it will play the sound whenever you equip a weapon, regardless of what weapon it is or what language you're using. See sound.h for these specific cues.
** There are times I can get around this. For example, when I get a level up, I tend to read it to see what leveled up. Thus, this gives my little fanfare audio-clip enough time to play. Of course, not all people may be like this.

But I'm more than happy to fix these issues! (Just...not right now. I'm actually really really busy atm. This is something I'll do much later. Maybe during the summer?)

I have a few ideas, but I want to know what others think about them.

• Simplify the compilation process. It shouldn't necessary to edit sound.h when compiling without tiles. If SDL can work in the text-based mode (I assume it can), then the text-based mode should use it! The external command should be a plan B for those who cannot/do not want to use SDL or whatever library we go with.
  
• Make more specific sound triggers. I don't want to go overboard with this, because it means adding function calls to basically every part of the game. However, the regular expression system is really bad.
  
• Use a sound 'buffer'. I'm using the term 'buffer' really loosely here. When a normal sound is going to play, it should get put into this buffer of sorts. Only one sound can be in the buffer at a time. Whenever control passes back to the player (i.e. the "--More--" prompt appears or all the text messages have been displayed), then the sound in the buffer gets played. This ensures that only one sound gets played a turn, instead of flooding the player with insignificant trash.
  
• Consider the importance of each sound. Sounds would be ranked in their importance. Mundane occurrences, like picking objects up, changing equipment, and getting attacked by weak monsters would get low rankings. Major things, like receiving nasty ailments, miscasting dangerous spells, or receiving divine punishments would get higher rankings. Since only one sound can be on the buffer at a time, this would be a way of deciding what sounds get priority.***
  
• Recognize "key sounds." "Key sounds" are those that denote really significant things, and the triggers are decided by the config files. If a key sound is about to be played but there is already something in the sound buffer, it would force a "--More--" prompt. When the user moves past the "--More--" prompt, then the key sound plays along with its associated message. The user cannot interrupt this, and entire sound plays out. Once the sound is done, the game continues as normal. This allows for longer fanfares. This could get annoying for some, so it's entirely up to the configuration being used.
  
• Provide options for variable sounds. I could achieve this by allowing a list of sounds to be provided. The game would pick one at random. This is a rudimentary solution, but it's easier to configure than allowing for random pitch/volume/speed shifting.

*** This ranking system could get really complex, however, if go with a dynamic ranking system. If I'm playing a stealthy character, I'm really concerned about a noisy door or a shouting monster. Fire-based attacks might be a bigger concern for those weak to fire. Followers of Ashenzari may actually be happy about putting on a cursed item. I don't know how dynamic the system should be, and I'm worried about cluttering up the code with these checks. The player's title might make it easier to gauge the player's situation, though, because titles reflect the current play style.

What are your opinions? Please, let me know! Do you have any suggestions or concerns? (Note that I'm not planning on making sound effects from scratch. I'm just going to test the system with my BindTheEarth sound configuration. I'm a coder, not a sound designer!)

EDIT: Fixed a couple typos.

[advil]

This sounds great!

My experience with all this as a dev is that the most immediate barrier is simply the packaging issue: basically no one is going to construct their own soundset, so BindTheEarth is pretty much the only option, but it could be easier to install, and it's not very obvious how to even find it unless someone tells you about it. I think it would be good to try to get it down to as few steps as possible, which might be just (a) download the sound pack and put it somewhere appropriate, (b) download the sound config file and put it somewhere appropriate, (c) crawl detects all this automatically. Also, I guess there isn't much in the way of decent CC-licensed sounds / soundsets that could be used?

One question I get occasionally is whether it's possible to use CWZ's soundset offline. I'm guessing distribution would be the main problem (since these sounds are from all over the place and fairly non-free, and you can't just download them from CWZ).

[bel]

Sounds pretty good (I have no idea about sounds in games, so take this with a boulder of salt).

Btw: "audio queues" should be "audio cues".

[Autofire]

My experience with all this as a dev is that the most immediate barrier is simply the packaging issue: basically no one is going to construct their own soundset, ... Also, I guess there isn't much in the way of decent CC-licensed sounds / soundsets that could be used?

That was one problem I faced when I first started working on this. I needed something to test out the code I was writing, and I didn't just want dummy place-holder sounds. I found all of the sounds on this Earthbound fansite (which I linked to in my repo's README.md), so it was fairly convenient to grab them and begin plugging them in. They also go fairly well with a text-based interface, though I imagine that they sound out of place with the Tiles interface.

But this obviously isn't something that could ship with DCSS. I'm hoping that, by building up a good system, it will be easier for those who are interested to actually make sounds. But the code needs to be there first.

...so BindTheEarth is pretty much the only option, but it could be easier to install, and it's not very obvious how to even find it unless someone tells you about it.

Agreed. Making it accessible wasn't my main priority, but now that I'm looking back at it, it needs some help in that area.

One question I get occasionally is whether it's possible to use CWZ's soundset offline. I'm guessing distribution would be the main problem (since these sounds are from all over the place and fairly non-free, and you can't just download them from CQZ).

CWZ? CQZ?

Btw: "audio queues" should be "audio cues".

Blegh...I always seem to screw that up somehow...

[Siegurt]

Btw: "audio queues" should be "audio cues".

Blegh...I always seem to screw that up somehow...

A cue is a prompt or notification (or a stick in pool) a queue is a line.

Just imagine someone poking you with a pool cue to get you to wake up and do something.

[advil]

One question I get occasionally is whether it's possible to use CWZ's soundset offline. I'm guessing distribution would be the main problem (since these sounds are from all over the place and fairly non-free, and you can't just download them from CQZ).

CWZ? CQZ?

Sorry, CQZ was just a typo; CWZ is a Korean webtiles server that does sound via a server-customized thing.

[aidanholm]

• Simplify the compilation process. It shouldn't necessary to edit sound.h when compiling without tiles. If SDL can work in the text-based mode (I assume it can), then the text-based mode should use it!

SDL Mixer can be used without the rest of it (opening a graphical window). I'm confident it'd be possible to build a sound system that fixes the concerns about sound crunching, burying important sounds, etc; SDL Mixer continuously mixes sounds and can alter how each sound is played as the game progress, as opposed to a fire-and-forget approach.

The external command should be a plan B for those who cannot/do not want to use SDL or whatever library we go with.

I'm not sure that this would be worth the effort, as ensuring feature parity would restrict what could be achieved with SDL, and it'd probably be tough to coordinate sound effects in a way that sounds good on both backends.

[duvessa]

My experience with all this as a dev is that the most immediate barrier is simply the packaging issue: basically no one is going to construct their own soundset, so BindTheEarth is pretty much the only option, but it could be easier to install, and it's not very obvious how to even find it unless someone tells you about it. I think it would be good to try to get it down to as few steps as possible, which might be just (a) download the sound pack and put it somewhere appropriate, (b) download the sound config file and put it somewhere appropriate, (c) crawl detects all this automatically. Also, I guess there isn't much in the way of decent CC-licensed sounds / soundsets that could be used?

I'm happy to make sounds for DCSS if needed.

[Airwolf]

I'm happy to make sounds for DCSS if needed.

You mean beyond elven face-sitting?

[aidanholm]

Also, I guess there isn't much in the way of decent CC-licensed sounds / soundsets that could be used?

opengameart.org has a lot more CC0-licensed sound effects than I expected, many of which look (sound?) reasonably high-quality and thematically appropriate.

[duvessa]

freesound.org also appears to have over 400,000 sounds at the time of writing, though I don't know how many of these are CC0-licensed (a very rough estimate from my experience is about 20% of them).