Blizzard Entertainment has an unusual issue: They are scared of their own success.

In 2002, the company crafted the most important netplay upgrade since Battle.net itself, a Warcraft III matchmaking system hailed for consistency and legitimacy.   It’s been downhill in the eight years since.  Today, Starcraft II is now in beta testing.  And so far, the only nice thing to say about this matchmaking system is that I can play Starcraft II with it.

Blizzard doesn’t get it.  Matchmaking isn’t about getting your cat into games against players of equal skill.  It’s about consumer confidence.  No one cares whether the Warcraft III matchmaking system works.  Nobody thinks it does.  And if people don’t believe in Starcraft II’s smorgasbord of algorithms and placement, they won’t convert their time and effort into shaping an accurate leaderboard.

The Reign of Chaos approach worked because its transparent methodology was easy to explain: Start at level one.  Play anyone within six levels of you.  Five wins against equal competition earned you a level.  And if you win fifty percent of your games, the system will nudge you towards level ten.  Simple enough to make win-loss records mean something, simple enough to let players create personal goals.

Yup.  Casual players complained about that system.  A system similar to the one now used by TetrisFriends, a stronghold for casual gaming.  So Blizzard pressed the reset button.  In 2004, the new Warcraft III matchmaking system was built on Expected Ladder Level, where the game would guess your eventual level.  The formula for ELL was never disclosed, so nobody knew the game-to-game reward for beating other players.  What players did know is that one Azeroth player (Jubae) cracked the top five with a near-below-.500 record, that players were tanking games to launch ladder rampages, and top players couldn’t find games at all.  So rather than take the time to play the as-many-as-300 games required to reveal their actual ladder level, people quit.  Lots of them.

We now have Starcraft II.  It combines a modified Elo system with a league format.  Weaker players pad out Novice, Copper, and Silver Leagues, while better players hog Gold, Platinum, and (supposedly-invite only) Pro Leagues.

The first problem? Players in each league don’t directly compete against each other.  It’s a pseudo-league that’s just a simple size.  Instead of explaining you’re in the 97th percentile of all players in one gametype, you’ll be ranked third in a one-hundred-man league.

The bigger problem?  Let me put it this way: Know how people claim a good college basketball team would beat a bad NBA team? Rating in each level of play are independent of the other leagues.  As of this writing, the top-ranked North American player is Canadian Warcraft III semi-pro KiWiKaKi, who holds a rating close to 1900.  Meanwhile, Oakhill of the Battle.net forums (thanks to placement that was no fault of his own) has dominated his Bronze League to a 2100 rating  Great! I totally look forward to Copper League players telling me to get on their level.

Solve for x.

If you are going to create a matchmaking system that doesn’t disclose its methodology, it needs to be effective out of the box.  Even X-Box Live’s Trueskill (link credit to Veryrandom), the work of calculus hell, matches you on a fifty-level ranking system that appears to work consistently.  And right now, I’m looking at a Starcraft II ladder where nobody can explain how record correlates to rating, a team ladder where the best players can’t advance the rankings because they’re “heavily favored” to win.  You know, the same things that happened to Warcraft III.

So yeah.  Blizzard originally trashed their best ladder system to create fair matchups for weaker players.  They’re now tweaking it to give casuals false satisfaction.

What’s that saying about fixing things that aren’t broken?

This entry was posted on Sunday, February 28th, 2010 at 7:53 pm and is filed under Online Play, Starcraft II, Warcraft III. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.