Pourush: Welcome to the game (although I have a hunch it's not the perfect target for your gaming needs).
I have initiated the philosophy section and written large chunks of it, so I feel responsible to reply. (Note 1: I am not an official developer anymore, but Crawl is still very dear to me; I follow what's going on, and I have some ideas I'd like to see get into the game. Note 2: The replies by cerebovssquire (blue) and mageykun (green) were on the point; I'll try to add something new.)
Roguelikes are developed in a larger context -- the design space is huge, considering that Rogue, Nethack, ToME, Letterhunt, Diablo, DoomRL are all roguelikes. Crawl happens to be a roguelike with a long history and a rather large crew (between four and ten active developers at any time, I would say). When I wrote the philosophy section, I was mostly concerned with trying to distill what makes Crawl work so well (compared to Nethack, where I came from). These are the major and minor goals you can see there. Later on, the philosophy section was used, first among developers but then also by players, to say that something is Crawly or un-Crawly. So it gave Crawl a more specific place in the huge design space for roguelikes (although at times it is interpreted too strictly, for my tastes at least -- the recent posting that "doing Lair is a no-brainer", for example).
A completely different way to interpret the philosophy section, which addresses Pourush's question, is that it says to some degree what's unfun to the *devteam*. (And of course, there can be tensions among developers and there is no clear procedure how to resolve those.) Fun is subjective. We have no chance to make the game universally fun for everyone. And honestly speaking, we'd be in an awkward position if we tried, what with our baroque interface, the aged graphics (don't even mention the ASCII), the turn-based and sudden death approach to game.
But even among more hardcore players, fun is diverse. There probably are players who really enjoy grinding in games. (I'd bet that many who grind don't really enjoy it but who am I to judge.) However, everyone in the devteam agrees that grinding is no fun but rather a symptom of bad design. This does not mean that Crawl is grind free but it does mean that we'll think about how to remove a source of grinding if one shows up. In other words, if you happen to like grinding, then Crawl is probably not a game for you (speaking in general, not to Pourush).
So the Crawl philosophy does not try to say what a roguelike is (it assumes that the players know that) and it does not try to say how a game can be fun (because that's impossible). It just lays down a number of basic principles that are accepted among developers (and also most players). More or less sticking to those principles makes sure that Crawl is more or less fun to those who actually play it.
Regarding Pourush's last set of questions: Crawl is not a game to teach you its inner workings in any detail (for example, because they're very formulaic -- by the way, I think I was the developer most adamant about not displaying numbers, this might change over time). However, the idea is that if you learn from your deaths, you can improve. It is true that you won't play perfectly without access to long lists of numbers, formulas, maps (i.e. spoilers) or even the source, but it should be possible to improve your game -- i.e. to get deeper and deeper, obtain a rune, and then win. The game has been won unspoiled, and that's important to us.
I read that you're bored by Crawl's large levels. That's absolutely fine but an indication that Crawl might not be for you. Crawl is one of the (rather few) huge roguelikes (I guess the most known others are Angband and derivatives, Nethack, ToME). It has a large number of those large levels. Winning game of first time victories often take 20 hours. Some players like this epic grandezza, others cannot stand it. Obviously, that's a property which won't change although steps are taken to make the game a bit slimmer.
There is a solution to all your problems, though: Give Brogue a try:
https://sites.google.com/site/broguegame/That game is much smaller in scope. I can vouch for its excellent design -- it does many things better than Crawl can do, and it explains its mechanics in a very clear way. It is also hard but in a good way (a lot to learn before you win).