Playing New Vegas makes me think about balancing the experience versus player expectation in single-player RPGs, especially when they’re classless, meaning you can morph your character into whatever you want as you play.

The core experience balancing decisions revolve around how much FOMO you want to create for the player. Examples:

How many locked chests do you put in the first couple hours of play? Do you put a chest that’s close to the max lockpicking skill they could have at that point?

Have too many things requiring a specific skill in the first couple hours and players can feel cheated by the class/skill they chose. They focus on what they CAN’T do instead of what they CAN.

Have too few items and players feel like they chose the wrong class/skill too.

And of course these decisions are closely tied to decisions about how quickly you want players to level and whether you want to directly reward specialization. Do you place a lock with a difficulty only available if players spent 90% of their points in that skill? 75%? 50%?

I learned to try to quantify as much as I can, to develop formulas rather than just rely on “feel” or vague concepts. My formulas tend to be based on “work” rather than specific interactions or time elasped because it accommodates player actions more completely.

By “instances” I mean discrete player actions like talking to an NPC, killing an enemy, etc. Conceptualize what a unit of “work” is in terms of player actions, decide how that translates into each activity, & you have a currency for measuring & normalizing difficulty and rewards.

The more of these decisions you can make up front, before you start your level design and world building, the more balanced and even your game experience will be. You can use analytics to see if gameplay sessions are hitting your targets, even if it’s just the team playing.

This is just one reason out of around 1000 for why you should a) have regular team playtests and b) instrument your analytics ASAP so team playtests give you quantitative along with qualitative data.

As a side note, the more closely you can define your systems to the player experience, the better your systems will be. Start every system’s design by asking yourself: what is the player experience I want out of this, and how can I measure objectively whether I got it right?