You’ll probably fail the first time. It’s quite difficult to discover the right cost formula a priori, so you’ll need to experiment and revise. You just want a reasonable starting point somewhere “in the ballpark.” The first thing to do is to list your “knowns.” Maybe you want a certain effect to have a certain cost, and you can use that as a benchmark. However, let’s assume you think you know nothing. How do you start to get a handle on the costs? Well, usually it turns out you know more than you think you do.
I’m going to use an example of a game with giant robots battling it out. I’m going to call it “Drone On”. In this example game, you have a resource, credits, that you use to buy “mechs.” Some mechs are “vanilla,” they only have combat related stats, some have special abilities, like the ability to generate more credits. Each mech has an attack value and a defense value, indicating how strong it is in those areas. Ok, so how do I begin to figure out what a basic mech should cost?
You want to get some benchmarks, and the best way to do that is to get them from the basic structure of the game itself, or what I want the structure to be. Let’s say I want the game to last about 20 minutes. In Drone On, let’s say, dealing an opponent 20 damage means you win. Ok, so if I think a basic turn will take 1 minute, that means the game should end in about 10 turns, giving each player 10 x 1 minute turn = 20 minutes. Ok, so that means that over 10 turns, a player should be able to generate at least 20 attack power. Now lets assume that resource generation is fixed, you get 1 credit per turn. If attack = cost, then on turn 1 (t1), I generate 1 attack, t2 I have 3 attack (1 + 2 for the turn), and so forth. I also assume some mechs will be destroyed either in combat or due to other effects, so let’s say that only half the attack power I generate actually counts. Ok, so I divide the attack generated each turn by 2. That looks to be pretty close. The way I did it, I added half the cost to the attack from the previous turn, giving me 28 attack by t10.
You could get more complicated, but the basic principle is to work from what you know. In this case, I know I want my game to be about 20 minutes. I made some assumptions about the amount of damage needed to win, resource generation, and the way destroying mechs might work. I then tried a solution, attack = cost, and it seems pretty close to what I want. You could complicate things by, for instance, making resource generation probabilistic instead of fixed. You could also tweak the attack = cost formula so it’s not linear.
While attack = cost seems a reasonable starting place in this game, I might want to do a bit more before I begin designing a lot of sample mechs. One dynamic I like in strategy games is for there to be an aggressive, short-term strategy and a more patient, long-term strategy. To do that in Drone On, I want my lower cost mechs to be more efficiently costed than my more expensive mechs, but have those expensive mechs be more powerful. In other words, what I want is not a linear cost to effect relationship, but a curve. I want something close to an exponential function, where the lower units are closer in power level than the later units. I did this in Google Docs, and just tried a few values for the exponent in attack = cost^exponent. Just eyeballing it, it looks like attack = cost^1.35 looks close to what I think I want. That means a 1 cost mech has 1 attack, a 2 cost mech has about 3 attack, a 4 cost mech has about 6 attack, and so forth.
Remember, the point isn’t to discover the right answer, just give yourself some guidelines to begin costing the mechs you want to design. The next thing to do is try to peg costs for abilities. At this point, you know how much 1 attack is worth, and hopefully, you can use that as a guide for how to cost other things. For example, you know how much attack 1 credit is worth, and maybe you can tell how much destroying a mech of cost 3 or less should be worth.
In testing, you should pay attention to what feels right. The numbers and math can hopefully get you closer, but ultimately, what makes games fun is how they feel, not how elegant your math is. So play it out! I might discover that my aggressive, short-term strategies aren’t going well, so I might want to introduce a 1 cost, 2 attack option to power that strategy up. The game might feel more fun when aggressive strategies are good.
Another thing to pay attention to is interactions. If you have a lot of effects that destroy mechs of 3 attack or less, then having 4 attack has a special premium. These kind of interactions can add a lot of satisfying texture to a game. I tend to think that basic cost to effect ratios for a game are relatively easy for players to discover. Especially in the age of the internet, if your game is successful, it will be analyzed by players, and the kind of costing math I’ve discussed is relatively transparent. Even if players don’t do any math themselves, they can intuit what’s a good deal for the cost in your game. Interactions complicate things, make things situational, and throw off the math.
So in summary, base your costs off what you know. Maybe you know some effects should have a certain cost and can base other costs off that. If you don’t, think about how long you want the game to last, how players will progress from the beginning to the end, and use basic assumptions to arrive at some initial costs.