Redefining “massively” to fit lots more players into MMOs

Gaming veterans like Mark Jacobs, CEO of Unchained Entertainment, have been thinking about staging massive battles in online games for a long time. Working on Final Stand: Ragnarok, Jacobs wants to see how many thousands of players can fit inside — anywhere from 1,000 to 10,000 players fighting each other — in the same fantasy warrior battle.

Jacobs talked about this topic in a session dubbed, Redefining ‘massively’ in MMOs: How many players can you fit in an online multiplayer game at our GamesBeat Summit 2024 event in Los Angeles in May. He was joined on the panel by Christopher Caen, a consultant at The Seven and a veteran of VRML in the 1990s; Rashid Mansoor, CEO of Meta Gravity, which is also working on creating massive concurrency in online games; and moderator Wanda Meloni, a longtime gaming analyst at M2 Insights. They talked about whether tech is really ready to put together the massive online gatherings we have dreamed about.

Eventually, solving this kind of challenge in online games will deliver more realism in games, resulting in fantasy battles that can be as big as the Battle of the Pelennor Fields in the Peter Jackson film, The Lord of the Rings: The Return of the King. It will also enable many of the expectations of the metaverse — the universe of virtual worlds that are all interconnected, like in novels such as Snow Crash and Ready Player One. Jacobs hopes that the tech is finally here to enable games to live up to our imaginations. (And be sure to sign up for GamesBeat Next 2024 on October 28-29 this year in San Francisco).

Here’s an edited transcript of the panel.

GamesBeat: How many players can you fit in a massively multiplayer online game? Here to redefine “massively” in MMO, please welcome Mark Jacobs of Unchained Entertainment, Rashid Mansour of Meta Gravity, Christopher Caen, and moderator Wanda Meloni of MT Insights.

Wanda Meloni: Let’s have everyone introduce themselves.

Christopher Caen: I’m a partner at a consulting agency called The Seven. We were founded by a group of marketing and metaverse veterans who kept looking out over the landscape and seeing brands and marketers making the same mistakes that we had seen back in the days of VRML in the ‘90s. The same behavioral mistakes, the misunderstandings of what comprises a community online. The Seven has basically developed a framework that allows us to look at a community, tear it down into its constituent pieces, and understand what’s ticking, what’s not working, the care and feeding of a community. And then we’ve developed a set of metrics and measurements that allow us to gauge whether we’re succeeding against that.

Unfortunately the problem today is the same as it was back in the days of VRML, which is: getting people to come to your virtual experience and virtual world is easy. Getting them to come back is hard. That’s what The Seven is all about.

Rashid Mansoor: I’m founder and CEO of Meta Gravity. I founded the company focusing on solving what I think is a very important problem in virtual worlds, games, and simulations. How do you scale a backend to make massive concurrency? Large numbers of users, AI, physics, and objects in a virtual world, how do you make that possible? I previously founded another company called Hadean, which was also focusing on that problem, but also more broadly, how do you build supercomputing capabilities using cloud hardware? Before that I founded another company, Adbrain, which was my learning curve on how the approaches we’re taking to cloud architectures were fundamentally flawed. At Meta Gravity we’re now building technology that we hope will power the next generation of games, and also these metaverse experiences everyone keeps talking about.

Mark Jacobs: I’ve been making online games literally forever. If you played Dark Age of Camelot, Warhammer Online, Aliens Online, and so on, you probably played one of my games. The last decade or so I’ve been doing what he’s doing, which is trying to build an engine to do great things. Things that for me, as a gamer, I’ve always wanted to see, which is large scale battles, up close and personal, 30 frames per second minimum, and not 50 people, not 100 people. That’s for tourists. I’m talking 500, 1,000, 2,000 and up. That’s what we do. I’ve been doing it for a while. We’re about to show you some video, one from me and then one from Rashid.

I should quickly mention, there’s a rumor going around that someone on this panel has actually done medieval swordplay. I’m here to tell you that the rumor is correct.

Mansoor: Guilty as charged.

Meloni: Those were beautiful, both of the videos, and the question then is, “Why?” What is the reasoning for wanting more and bigger?

Mansoor: As Mark was saying, this has been an obsession of his for quite some time. It’s the same with me. There are really two people you should be thinking about when you think about these technologies. It’s the designers building the game and the players playing it. It’s about serving the players. For me this is very much not even about the numbers. It’s not about 2,000 people in a battle, or 10,000, or 100,000. Those are useful benchmark numbers. But it’s really about freedom. It’s both design freedom and player freedom.

If you think about a game today, you can’t just just implement any mechanic in an online multiplayer game. You have to think about the consequences of every mechanic. If you let people create a clearing and build a house, you have to endure the consequences of there being two houses, and then a little hamlet growing into a town and into a city. Now you have hundreds if not thousands of players in a single location. No server tech today, outside of what we’re building, and no engine can handle that. In order to handle that you need to rethink the technology.

But if you could handle it, then designers have total freedom to think of intuitive mechanics that make sense in the real world, or in a story. You can bring them into games. When you do that, you also give players that freedom. If you take MMOs today, they either don’t do something like player housing, even though it makes intuitive sense, or if they do it it’s instanced, which breaks immersion. It’s about giving players freedom as well. For that, this technology has to exist. That’s really it. It’s about freedom.

Caen: I love that answer, and I love being on this panel, because I think it’s important to realize that what Rashid says is very important. I’ve been doing virtual worlds since the ‘90s. I remember the first time we got 50 people into an instance. That was the most amazing thing that had ever happened. But as we do this, it’s important to remember that Rashid said, it’s about the player. The technology will always catch up to where you want it to be. Then we’ll leap ahead in terms of playability. The technology will catch up again.

But it becomes about the things that bind the players together. It becomes about discoverability. It becomes about agency. It can be too big of a world, where you lose agency. You lose your own impact and import in the world around you, which means you lose interest in going back. You have yourself in a horrible loop.

I was watching Ready Player One on Saturday before I came down here, and there’s a line at the beginning where Parzival logs into the Oasis. He pulls up the UI and says, “Locate H.” I never really thought about that line before, but I thought, “Wow. The amount of things that have to go correct to do ‘Locate H’ in that massive space is really hard.” By the way, should he have even been allowed to interrupt someone like that? Another question. But I think this is important. Mark has really started to drill down on the importance of that view pane into what’s really happening in there.

Jacobs: In my case, what we’re trying to accomplish is pretty much along the lines of what Rashid said. But we’re also building a client, because the problem really is twofold. In order to give these kinds of experiences to players, both the networking and the rendering side have to be awesome. You can’t do one without the other. Once you have that–Rashid used the perfect word, “freedom.” As a designer myself, if I don’t have to worry about, “Hey, maybe I want to have 1,000 people in a battle, or 500 people”–as someone who’s been a huge consumer of geek IP, because I’ve been doing this forever, I want the same things that you read in books, when people used to read books, or that we’ve seen in movies.

Who wouldn’t want to see the battle on the Wall? Who wouldn’t want to see Helm’s Deep? Who wouldn’t want to be a part of it, where it’s not just the four of us here and everything else is an NPC? We want it to be all players whenever possible. If our engines can give designers that freedom, if we can give players that incredibly immersive experience, we will create something that gives the ability to designers to create something even better.

As Rashid certainly knows, and Chris knows, it’s a pain in the ass to try to do this. It has consumed companies. It’s consumed billions of dollars. Nobody has gotten it even close to right yet in a production sense. That’s what we’re hoping to change over the next few years.

Meloni: On that note, how do you manage this in terms of the UI and the discoverability? What are the keys that you’re seeing that are necessary?

Caen: Agency is important. Even back when we were doing our first–I played Destiny a lot, obviously. But in the radar in Destiny, there are the players in your fireteam who are little green dots, and then there are the LFGs who can join your team who are little blue dots. LFG players, randos in Destiny are called blueberries, because they’re blue dots on your radar. This is part of building a layered experience, building relationships unconsciously between players.

Then it becomes about–look at Helldivers. You have 100,000, 200,000 concurrent users. How do you make sure everyone feels like part of the story? The way they do it today is with, in essence, a dungeon master, a guy named Joel who basically issues orders. The bad guys have taken over this planet. All Helldivers go and conquer it. That’s great for agency and impact because if I can only spend a couple of hours a week playing, I know that if I jump in then, my impact in the world is maximized. I feel like I’ve had an impact there.

You can do that with a couple hundred thousand concurrents. What these two gentlemen are building, you can’t Joel your way out of it. It needs to become more integrated into the experience. That’s why I love that both–on the backend, Rashid’s thinking about it, and on the frontend, what’s actually in the viewport, Mark’s thinking about it. That’s incredibly important. You can’t separate these three things. They have to be considered all together.

Mansoor: It’s important to observe that we’re at the cusp of something new. I saw someone the other day who said that the majority of game genres that exist today, at least in archetype, were really established in the 1990s. That’s interesting. That decade, into the early 2000s I suppose, that was the decade where CPU clock speeds were increasing at an exponential rate. They were starting to slow down by the mid-2000s. We established CPU performance for the most part, which meant that much of what you could fit into a game simulation had become sort of crystallized. That meant that game genres became crystallized.

The next two decades were about making games look better, because GPUs were still improving. But CPUs were in the logarithmic phase of performance, let’s say. What we’re talking about here is opening Pandora’s box. It’s a completely new era of performance and simulation capability. We aren’t going to be able to predict, no matter how smart we think we are, the kind of game genres and experiences that will unleash. We have some theses. That’s the reason we’re doing it. But we could never predict what’s going to crystallize in the next decade of this endeavor.

Caen: A perfect example is what we saw in the latest Zelda game, where people were taking pieces of the world that didn’t seem to have anything to do with each other, and they were crafting mechs and spaceships and tanks. The designers said, “Yeah, we didn’t see a lot of that coming.”

To Rashid’s point, the last decades have been about building these game experiences on rails. We’re not going to let you get out of the sandbox. All shooters talk about a sandbox now. But what if there is no sandbox? What does that look like? What if you do decide to have 100,000 people attack some poor guy’s village? That’s the simulation. That’s the world this now lives in. But that layers a whole bunch of considerations in terms of UI and discoverability to at least give you some ways of working a way through that. Not around it. It’s no longer under your control.

That’s where you start thinking about what these gentlemen are building. I have the easy part on this panel, because I don’t have to build it. But how do you start managing a world where the sandbox doesn’t exist and that’s okay?

Jacobs: If you go back and look at different ages within the game industry, there has never been a time where people have said, “We don’t need more people. Eight people in multiplayer is enough. Who would want 16? Who would want 32?” That’s such a constant in the industry. It’s been going on since the MUDs. How many people could you fit in a text-based game before it explodes?

One of the challenges for people like Rashid and myself is that there are a lot of people in the industry, executives some of them, others in VC, who don’t, because they’re not designers and not visionaries and not whatever–they don’t see what designers see, which is the ability to keep going, to come up with new ideas, to create games that you couldn’t even think of creating before. One of the great proofs of that is Fortnite. The reason being, when Fortnite was made, when Tim Sweeney made the brilliant decision–if you want to look at some of the best decisions a CEO has ever made, it was when Tim made the switch from Fortnite Saves the World to Fortnite. A lot of people, even though PUBG had already come out, were saying, “100 people? We can’t wait for 100 people to start a match.”

It’s the same thing with what we’re doing. The designers, the companies, the people who will build the games, because we’re building our own as well–we’ll take this tech in a different direction than we can expect. I can guarantee one thing. It’s going to happen. People might say there will never be a 500-person game or a 1,000-person game like we’re talking about. I’m not talking about MMOs, because we’ve had thousands and thousands on a server before. But the kind of up-close battles that you saw on Rashid’s video, and you saw some of it in mine as well–the designers and the companies will take that in directions that, even if we could anticipate–it’s going to be great. It’s going to be so much fun.

That’s why crazy people like us have been chasing this. We’re not the first. We won’t be the last. But it’s for the good. It’s to make games better, to make them more interesting, more challenging, to bring out the immersive experiences that you can get in books and in film, but now we can deliver it. Partially because of what you said about CPUs and GPUs, everything getting more powerful, which allows us to make these engines that take advantage of that. Because let me tell you, our engine is the most parasitic engine on the planet. We’ll take every core, every thread. If you lean another computer toward it, there are rumors that we’ll take that too. That’s what you need to power these kinds of games.

To touch base with what you said earlier about this being difficult–yeah, the degree of difficulty on this, plus making a game like we’re both doing, that’s kind of hard-ish.

Meloni: Let’s add another layer onto that, which is discoverability.

Jacobs: Discoverability is something that, to be honest with you right now, I’m not worried about. I’m worried about the tech. I’m worried about the games we’re making. I think discoverability–I’ll leave it to people like him, who might have better ideas – and probably do – about how to make that happen.

Caen: Discoverability is huge. I’ve started to play more games, big games, different games. The one thing that’s happening is that–my wife and I talk about how all these streaming videos, if you watch a series now on Netflix or Amazon, they’re pushing the actual title card further and further into it. They’re trying to get you hooked before they give you the title. We’ve watched some shows where it’s eight or 10 minutes before the title card shows up.

Discoverability is one of those hooks. If it’s hard for you to get in and immediately find your place in the world of that moment, then it’s going to be harder to keep getting that person to come back. It also is where you need to start–I know there were conversations earlier at GamesBeat about the power of building communities, about Discord, about being able to wrap more layers around the game experience so that the discoverability happens faster. In my clan, we don’t go into the game and then figure out how we’re going to build a raid team. We build the raid team on Discord and then go into the game.

You also have to start thinking about this as more of an ecosystem. The world they’re building is not just in the actual engine. It’s all the pieces of this community that live inside and outside the game, and how that provides discoverability. It also creates pull. Again, the challenges, especially as more and more types of entertainment, digital entertainment, happen–the time that you spend playing one of these games is time you’re not doing something else. You have to make it easier. You have to make it faster and more engaging in order to hook those people in before the title card for one of these games comes up.

Meloni: In terms of engagement itself, when you have these larger worlds, how does that impact–on the designer level, how does that impact how you design?

Jacobs: It’s a similar challenge to making just an MMO. Obviously when you look at something like Dark Age of Camelot, World of Warcraft, and so on, those are all problems we’ve faced before. If you’ve made an MMO, you’ve faced that problem. How do you handle the world?

We’ll be putting on a whole new layer of that, because instead of having 2,000 or 3,000 people on a server–that term has been so abused over the years. The fact is, as you saw from Rashid’s video and my video, we’re putting that in a battle, up close and personal. That’s where things change from a design perspective. That’s going to be the real challenge. It’s not how you deal with 100,000 people all around you. From a design perspective, a technical perspective, that’s a pain in the butt. But what do you do when they’re fighting? How does everyone feel like they’re a part of it? How do you handle both the networking and rendering sides of the problem?

That’s where the brilliance has to come in. Designers have to be very clever. They have to find a way. Because if it’s just a mosh pit, but 100 times over, people aren’t going to play that. Or not very often. It’s like the old saying. Just because you can do it doesn’t mean you should do it. We as designers are going to have to figure out what is the better number. There won’t be a right number. Anyone who says they know the exact right number, they’ll also sell you beachfront property in Arizona. There’s no right number. But we’ll have to find the good numbers, where people have a balance of this scale and enjoyment. If it’s not enjoyable they won’t play. It’s that simple. If it’s just, “Oh, cool, I’m in a battle with 10,000 people,” they’ll stop if the experience sucks.

Caen: It’s so important. The idea of design being related more to the audience than it is to the environment. We played throttling games in game design for years. There’s a reason why, when you play shooters or adventure games, you play through a big space, and then you run down a hallway into another big space. The reason is, the rendering engine uses the hallway to dump the old space out of memory and bring in the new space. It’s a delaying tactic. When you go into a big circular room, you know that’s the boss. The reason they tend to be circular is that it means fewer edges and vertices, so it’s faster to process when people are spinning their eyes around.

We played these games with design all the time. But what Mark is touching on is looking at this in a more global way. Looking at environments where instead of rivers of water, it’s rivers of people. How are you going to–you can’t control the people. But you can sort of affect the design of where the clustering is going to happen. Instead of saying, “This is the boss room,” you say, “We want to drive people over here, because we can host a couple thousand people in this space and we’re good to go.” It’s important, like Mark was talking about, that environmental design is going to become environment/people capacity style. I don’t think anyone has really dredged into that.

Mansoor: It’s an interesting observation, the way you put that. At the heart of what you’re talking about is self-organization or emergent systems. A bit unlike Mark, who has very much at this from Dark Age of Camelot and Warhammer Online – I played that game quite a bit, by the way – I’ve come at this from a scientific standpoint. I was interested in simulations. I was interested in emergent, non-linear systems. It’s around the time that I was also looking at the issue with game server tech and why online multiplayer games don’t build immersive worlds.

It was intuitive to me, coming at it from the scientific and mathematical side, that emergent systems would be interesting in a game world. It would keep them fresh and exciting. But nobody really exploits it. You realize that you can’t. You accidentally stumble on it. EVE was an interesting game, and I’ve previously worked with Hilmar and the EVE team at my previous company around solving one of their problems with scale as well. But EVE didn’t set out to have these big fleet battles. The consequences of the laws of the game were such that it was inevitable that there would be massive fleet battles. Then they had to work around the edge cases.

Similarly, I think a design choice that you made in Dark Age of Camelot, realm versus realm, and then having three factions, was very interesting. That counterbalances the imbalance that happens in the world, because when you pit thousands of people against each other, how do you deal with imbalances? Three factions balance one another, like a tripod. But there’s a deeper meaning here as well. If you look at this from a scientific standpoint, there is a lot of research around how three-dimensional dynamic systems have certain types of emergent behavior that you just can’t have with a two-dimensional system. In this case the three dimensions are the three factions.

There’s a lot of inspiration from nature. There’s already a lot of established evidence, theory, computer simulations and models that tell us how some of these systems will start to behave. As long as we go into them with the understanding that we can’t completely control them–we have to influence them. We have to understand, when we start to build these big worlds, that they are emergent systems. That’s scary for a lot of people, but therein is the opportunity.

Meloni: The idea of just touching the surface of environmental design within these communities, and how that will impact the interaction within the communities–it’s completely redefined how we’re going to do that, which is really exciting.

Question: I wanted to ask about environmental design and the approaches you’re taking to partition your potentially very large player bases into sort of parallel play, and then the larger design dynamics on top of that.

Jacobs: In my case, right now we’re making both an MMO and a co-op game. The co-op game solves for itself. We don’t have to worry about that, because the size of the battles, even in a battle royale mode, will not be so huge that the environmental challenges will be any different from the kinds of stuff we dealt with in the past.

For Camelot Unchained we’re still going to have to deal with that there, because unlike the co-op game, we want to have the kind of numbers that Rashid is talking about on the server. Big numbers, not 3,000 or 5,000, because again, we did that 20 years ago. 23 years ago at this point. We want to push it. I don’t know if we’ll push it as far as Rashid will push it with the metaverse kind of function, but we want to build bigger servers. That’s something we’ll absolutely deal with.

Right now it’s not even something we’ve spent much time on. But we will. We’ve talked about it in the past for Kickstarter and other presentations. But it’s not, to be honest, something we’re spending time on right now.

Mansoor: We’ve touched on a lot of issues that arise when you start talking about these numbers. We’re almost cheating in a sense with our game design, because it’s called Edge of Chaos. It’s actually a reference to the underlying–the fact that it’s a chaotic, dynamic system that we’re building. We’re also leaning into it by really embracing history. We’re re-creating feudalism. That already gives us this nice hierarchical model that works when you look at it from the top down. But it’s something you can build from the bottom up. We’re leaning on self-organization, but a proven model of self-organization.

Question: When it comes to having 100,000 people in one game, what is your way of making sure communication stays good between players? I know even stuff like World of Warcraft, when you’re in the game chat there are so many people talking at once that it’s hard to create your own community. If you have voice chat or game chat, what is the way of spinning that up so it isn’t so spammed?

Mansoor: Again, the answer is feudalism. It’s units at the village level led by a knight, going up to lords who have many knights, who have some loose degree of control over them, up to a king. When you start reasoning about this, the math of it really works when you fully commit to the model. There was a reason that kings didn’t have complete control over their lords. You have looser coupling. And then lords didn’t have complete control over knights and barons. They in turn didn’t have complete control over their retinues. We’re just leaning into that and thinking about design from that standpoint.

Like I said, and as Mark said, this is a new frontier. We’re taking one path that may not necessarily be the best path. But it’s certainly a path that has inspiration from nature. We think that’s useful.

Jacobs: That’s beautifully said. After all, don’t we all want barons chat with 100,000 people?

Mansoor: That’s a pun, for those who didn’t get it.

Question: Trying to create a summary of this panel, can you sum up the top two things, perhaps, that people need to know?

Jacobs: Oh, that’s easy. You want to call it hyperscale, or my term–we’re using similar terms, but the idea is–I’ll do one and you can come up with the second. It’s that massive hyperscale gaming is coming, and it’s not like nuclear fusion. It’s actually coming out soon. The best part of it is you can see the proof. That’s the big thing. You can play these games. I know you’re open. You guys are taking playtests soon. We’re doing the same thing. You’ll be able to see it, participate in it, and judge for yourselves how far this technology has come over the last–Hadean obviously, and now Meta Gravity and us, how far we’ve come over this time.

Mansoor: I’d echo massive and hyperscale. We actually went back and forth on the term a little bit. We even said “pseudo-massive” for a while until it turns out there’s a company with a name and a trademark. But hyperscale is a term also used in non-gaming industries to refer to this scale of compute. It made sense to use that. Persistence is another word that’s been around in the industry. Persistence is a useful attribute of hyperscale worlds. You want the world state to persist. It’s more meaningful that way. It isn’t just about having 100,000 players in every battle. It’s about not caring anymore so that you have that freedom. Hyperscale and persistence really help deliver that freedom.

Caen: Being the humor-centered guy here, I’m going to go in the exact opposite direction. My headline for the tweet would be “Dunbar’s Number Still Matters.”

Mansoor: Hence feudalism.