Players in a base-builder acquire a roster and then pour investment into a few units, in an order set by how they like to play. The roadmap question is not “how much did players spend” — it is which unit gets the investment, and what happens to every other unit when we change one of them.

The estimand that makes this answerable is a share: the fraction of a player’s investment occasions that go to the target unit, with the player’s own investment as the denominator. That removes the need for any revenue-attribution system — no currency has to be assigned to a unit for the comparison to be valid — and it bounds the outcome, so one heavy investor cannot drag the mean.

1. The estimand: a share, not a level

Every unit a player owns competes for the same occasions, so every other unit is bucketed into the groups the design cares about: same-role units (the substitutes), complements (units whose value rises when the target is invested in), and other-role units. The analysis is restricted to players who own the target; ownership is fixed before the window opens, so that is not post-treatment conditioning.

## SRM: control=3349, cost_down_25=3239, cost_up_25=3168  chi-square p = 0.077 -> PASS
## owners of air_hornet with at least one investment in the window: 2,675
Investment mix by arm
variant_name share_target share_same_role share_complement share_other_role occasions
control 0.1559 0.2215 0.0435 0.5792 3.5612
cost_down_25 0.2364 0.1594 0.0326 0.5716 3.7243
cost_up_25 0.1241 0.2378 0.0357 0.6024 3.7856

2. The cost ladder moves share, both ways

Test 9014 randomizes one unit’s upgrade cost: −25%, control, +25%. The share responds in both directions — a one-sided test could not separate a real demand response from a novelty effect.

Both directions move, and the arc elasticities agree in sign
arm cost_multiplier players share_target abs_diff ci_low ci_high p_value arc_elasticity
cost_down_25 0.75 903 0.2364 0.0806 0.0572 0.1040 0.0000 -1.4487
control 1.00 923 0.1559 0.0000 -0.0212 0.0212 NA NA
cost_up_25 1.25 849 0.1241 -0.0317 -0.0524 -0.0110 0.0027 -1.0197

3. Diversion: where the share actually comes from

The own-effect is the least interesting number. What a roadmap needs is the diversion — of the share the target gains, how much came from units doing the same job, and how much from everything else.

## 77% of the target's gain came from same-role units; 9% from other roles.
bucket share_change pct_of_gain
target unit 0.0806 NA
same-role
(substitutes) -0.0621 77.0436
complements -0.0109 13.5404
other roles -0.0076 9.4160

Net complements can still be gross substitutes. The complement bucket falls too. That is not evidence against complementarity — the units are linked, but every unit competes for the same scarce occasions and the target just got cheaper. Reading levels alongside shares keeps the distinction visible.

Investment levels, not shares
bucket control cost_down_25 level_pct_change
target 0.5710 0.8693 0.5226
same_role 0.7681 0.6379 -0.1696
complement 0.1809 0.1229 -0.3206
other_role 2.0412 2.0941 0.0259

4. Who responds: strategy inferred from the pre-period

Strategy preference is latent — nobody declares it. But it is visible in behaviour: a player’s pre-window investment mix says which role they favour, and that is a legitimate pre-treatment covariate.

Small strata are kept in the table, out of the chart
pre_period_focus share_target_control share_target_cost_down_25 share_target_cost_up_25 n_control n_cost_down_25 n_cost_up_25 abs_lift
support 0.1235 0.3059 0.1598 52 33 51 0.1824
air 0.1912 0.2901 0.1571 459 425 384 0.0988
artillery 0.0998 0.1839 0.0737 142 178 155 0.0841
armor 0.1360 0.1541 0.0938 211 210 208 0.0181

5. The choice model

The share comparison is the decision-relevant read; the choice model makes it portable. Every occasion is a discrete choice among the units a player owns, with each unit’s cost and power observed, so a conditional logit recovers how players trade cost against power — and those coefficients predict what a different price would do without running the test again.

Converting a logit coefficient into an aggregate share elasticity with the flat formula coef x (1 - share) assumes substitution is proportional across all units. It is not — it is concentrated inside the role — so the flat conversion overstates the aggregate move. Quote the observed arc elasticity; the gap between the two measures how nested the substitution is.

## occasions used: 21,652
## implied share elasticity from the choice model : -2.00
## observed arc elasticity from the arms          : -1.45
Conditional logit on the choice sets
term coefficient std_err ci_low ci_high
log_cost -2.365 0.066 -2.495 -2.235
log_power 1.622 0.078 1.469 1.776
log_synergy 0.397 0.029 0.340 0.454

6. Equipment slot: diversion or expansion?

Test 9015 adds an equipment option to the same unit — more ways to invest in it rather than a cheaper one. Occasions per player is the test: if it does not move, the treatment reshuffled rather than grew.

## target share 15.9% -> 32.3%     occasions 3.38 -> 3.44 (p = 0.48)
Share moves hard; the level barely moves — diversion, not expansion
variant_name share_target share_same_role share_complement share_other_role occasions
control 0.1592 0.2094 0.0469 0.5846 3.3843
extra_slot 0.3234 0.1401 0.0433 0.4932 3.4415

7. The value lever, randomized by server

Test 9016 buffs the unit’s power instead of its price: demand moves through value, not budget. The randomization unit changes too — a per-player balance change in a PvP game is unfair and detectable, so the shard is the smallest defensible unit. Whether clustering widens the interval is an empirical question, not a given.

## servers: 32   players: 1,914
## effect on target share: -0.0599
##   server-level (cluster) 95% CI [+0.0365, +0.0826]  p = 0.000
##   player-level (naive)   95% CI [+0.0367, +0.0831]
Value moves demand too — but the unit of randomization is the shard
variant_name share_target share_same_role share_complement share_other_role occasions
control 0.1375 0.2122 0.0596 0.5907 3.0088
power_buff 0.1974 0.1977 0.0523 0.5526 2.8406

Conclusion

Changing one unit’s upgrade cost by 25% moved its share of investment up and down symmetrically — a demand response, not a novelty effect. The number that matters for a roadmap is the diversion: roughly three-quarters of the gain came from units doing the same job, and under a tenth from other roles. The discount reshuffled attention inside one slot of the roster rather than changing how players play.

The equipment slot moved share even harder while leaving occasions per player flat — so it is diversion, not expansion. The power buff moved share through value rather than price, and it has to be randomized by server.

Three things to carry into the next test:

  1. Share is the right estimand for “which unit”, and it needs no revenue attribution at all. Read levels alongside it, or a complement that merely held its ground will look like it fell.
  2. Ask for the diversion, not the lift. A cost cut that steals from substitutes is a very different decision from one that pulls from other roles.
  3. Match the randomization unit to the fairness constraint. Cost and equipment can be randomized per player; power cannot.

Generated by the Savepoint Analytics video-game A/B testing case study. All data is simulated; the demand engine is data/simulation/units.py.