Skip to content

Matchmaking and game servers

This page covers the road from a player pressing Play to packets flowing between their game and a dedicated server, and the road back when the match ends. It names every service on the way, what each one owns, and which calls carry what.

For a gentler introduction to parties, tickets and servers, read Parties and matches and Servers and containers. This page is the engineering view of the same flow.

Who owns what

Each service owns one thing and reads nothing else.

Service Owns Does not do
Session The party (the lobby): members, leader, settings, ready flags, state Pick servers or form matches of strangers
Matchmaker The queue of waiting parties, and the matches formed from them Own parties (it reads rosters from Session) or servers (it asks the Allocator)
Allocator The game-server pool, reservations (allocations), and join tickets Know who is a friend of whom
Gameplay Proxy The public UDP door and the ticket check Hold any game state
Game server The match itself (your Godot code) Talk to players except through the proxy

There are two ways into a match, and both end the same way:

Path For Entry point
Private launch A party who play together, on a server of their own The leader presses launch: POST /api/player/session/party/launch
Matchmaking Parties (or solo players) matched with strangers The leader queues: POST /api/player/match

Path A: private launch

  1. Ready up. Members mark themselves ready. The leader launches with the party's current revision (a counter that rejects stale changes with 409 revision_mismatch).
  2. Session reserves a server. The party moves to launching and Session calls the Allocator: POST /internal/allocations with the party and its players.
  3. The Allocator reserves in one transaction. It picks a free, recently-heartbeating server with enough capacity using FOR UPDATE SKIP LOCKED, so two parties can never take the same server, and signs a join ticket for each player. No free server is 503 no_capacity.
  4. Session tells the party. Each member's event stream carries the proxy's address and their own ticket; the party becomes in_game.
  5. Players connect (see "Joining" below).

Path B: matchmaking

  1. Queue. The leader calls POST /api/player/match?party_id=... with a pool and the lobby settings. The Matchmaker checks the caller is the leader (against Session's roster), checks the pool against matchmaking.rules, and writes a queue entry: one live entry per party, idempotent. The party's size comes from Session's roster, never from the request, so a client cannot queue as one player and arrive in a match of four.
  2. Form. A worker per lane (a pool plus identical lobby settings) wakes when a party queues, and on a one-second sweep as the safety net. From the oldest party it adds whole parties first-fit until the pool's match_size is exactly filled. A party that has waited expand_after_seconds may be matched in another pool it lists; one that waits max_wait_seconds fails with timeout.
  3. Claim, place, write. Formation claims the parties (FOR UPDATE SKIP LOCKED), reserves one server for the whole match from the Allocator (POST /internal/matches), reads the rosters from Session, and writes the match, its groups and its players in one transaction. A crash in the middle loses nothing: the claim expires and the parties return to the queue.
  4. Learn the result. Clients poll GET /api/player/match?party_id=.... The answer is idle, queued (ticket, position, timeout) or in_match (match id and a descriptor: where to connect). Polling is cheap and writes nothing; it has a rate-limit bucket of its own.
  5. Get a credential. POST /api/player/match/ticket asks the Allocator for a fresh join ticket for the caller. It is also how a player reconnects.

The rules, the settings_key lanes and the repair pass that reconciles a match with the Allocator after a failure are in services/matchmaker/README.md.

The Allocator and game servers

A game server is a long-running container (gs-1, gs-2) that registers itself and then heartbeats. The Allocator tracks it as free, reserved, busy or dead, and each reservation (an allocation) as reserved, active, ended or expired.

Step Call (game server key) Effect
Start POST /internal/servers/register with id, internal address, capacity free. Re-registering resets it and ends any live allocation as server_restarted
Every 5 s POST /internal/servers/{id}/heartbeat with players connected The reply carries the allocation, if any, with the player ids and expiry; the first player moves the allocation reserved to active (the server to busy)
Match over POST /internal/servers/{id}/ended Allocation ends; server returns to free

Timers, all configurable on the Allocator:

Timer Default Meaning
Join ticket lifetime 60 s Short and single-use, so a leaked ticket is nearly worthless
Reservation lifetime 60 s If nobody connects the allocation expired and the server frees
Heartbeat timeout 15 s Silence marks the server dead and ends its allocation server_dead
Reaper interval 5 s How often the above is checked

Every way an allocation can end (ended, expired, server_dead, server_restarted) writes an outbox row. A drain calls back the allocation's owner: Session for a party allocation (which puts the party back to forming and tells the members), or the Matchmaker for a match (which closes the match and tells each party match.returned). Session callbacks give up after a few attempts and fall back to a repair poll; Matchmaker callbacks retry forever, because a running match has no other repair path.

A game server exits when its match is over and Docker restarts it (restart: always), which is how the next match gets a fresh world.

The Gameplay Proxy

The proxy is the only way into a match. It publishes UDP 27000 (GAMEPLAY_PROXY_PORT); game servers publish nothing and listen on 7777 inside the Docker network.

  1. Handshake. One datagram from the local port the client will later use for ENet: the magic OTJ1, the ticket length, the ticket.
  2. Verify. The ticket is an Ed25519 JWT signed by the Allocator. The proxy checks the signature against the Allocator's JWKS, the issuer, the audience (otomo:gameserver), the expiry (5 s leeway) and that sub, alloc, srv and jti are present.
  3. Look up and guard. srv must be a server in the Allocator's directory (polled every 5 s). Each jti is accepted once; the same ticket from another address is refused.
  4. Answer. OTOK, or OTNO and a reason byte (invalid, expired, reused, unknown server).
  5. Forward. On OTOK the proxy opens a session for exactly that client address and port and forwards traffic both ways until it is idle for 30 s. Anything else from an unknown address is dropped silently, so the proxy never answers a scan.

That is why the game must send its ticket from the same local port ENet will use after.

Relay mode: peer-hosted matches

With P2P_ENABLED=true the same proxy port also relays peer-hosted matches (transport p2p_host, design 31). For such a match the Allocator does not pick a server: it picks a host among the players (opted in, able to host, a recent low-RTT, low-loss probe to the proxy, and a good record of finishing the rooms it hosted) and creates a room for the match. The descriptor every player receives names the proxy, the room and host_player_id; the player whose id that is becomes the host.

  • The host asks for a host claim with a fresh Ed25519 public key, then proves it holds the private key to the proxy (OTH1 → OTCH → OTHP → OTHK). It keeps one UDP socket to the proxy and gets every other player on it, tagged with a slot number; the proxy tells it which player is in which slot (JOIN), and that is the host's only source of player identity.
  • Every other player joins with the ordinary OTJ1 handshake. Until the host is attached they are told OTNO 5 ("host not ready") and retry.
  • No player ever learns another's address. Per-player and per-room rate limits drop floods; nothing is queued.
  • If the host goes silent for 10 s the room waits 30 s for it to come back (from any address, with the same key); otherwise the match ends host_lost and parties go back to the lobby.
  • The proxy reports each room to the Allocator every 5 s, which is how the match becomes active and how a dead relay's rooms are cleaned up.

The host is trusted for the simulation only, never for identity, currency or results that pay out. Byte-level details: the p2p wire contract; measured cost: relay capacity.

The descriptor and the handoff contract

A descriptor says where and how to connect. Different games use different transports, so the Allocator builds the descriptor with a transport module:

Transport Descriptor Path
otomo-udp (default) Proxy address and port, proxied: true Client then Gameplay Proxy then game server
photon Room server host and a port triple, proxied: false Client straight to the room server; no otomo process is in the path

The match handoff contract (design/27-match-handoff-contract.md, reference code in services/session/internal/matchhandoff) fixes the shapes three parties agree on without reading each other's data:

  • Session owns who is in the match and their credentials. It never reads inside the descriptor, a credential or a loadout.
  • The Allocator mints credentials (the join ticket). It never reads a loadout.
  • The transport and game server decide where the match runs and who is admitted. They read their own descriptor, credential and, if the game uses one, the player's loadout snapshot.

A player's loadout (the equipment they chose, a document in a schema the game publishes as session.loadout) is copied when the match launches. The match runs on that snapshot, whatever the player saves afterwards.

The contract's refusals are typed, and each says what state the match is left in, for example no_capacity (match failed, HTTP 503), claim_expired, claim_reused (HTTP 401) and match_ended (HTTP 410, never retry). A match moves placing, ready, live, ended, with failed and cancelled as other endings.

When the match ends

Three separate calls follow, by design:

Call From To Why
POST /internal/servers/{id}/ended Game server Allocator Free the server
POST /internal/matchmaker/matches/{id}/ended Game server Matchmaker Record that the platform observed the match. The Matchmaker uses its own stored roster and tells Economy, once per player
POST /internal/economy/loot Game server Economy Pay the match reward, keyed by match_id, and only for a match that was observed

The game server cannot widen a payout by sending extra player ids: the reward is built from the registered roster. See Economy and inventory.

The game boundary

design/29-game-boundary.md is the rule behind all of this, and the one most worth reading before adding a feature:

A platform service may not know a game's word.

If a table, route or config key needs one game's noun (a crate, a badge, a deck), the abstraction is one level too low. Ask: could a second game, with different nouns, use this without renaming it? The doc maps examples to the mechanic underneath: a "crate" is an entitlement, a "deck" a loadout, a "mission" a track, a "replay" recorded content, a "rating" a server-written ext field.

Three mechanisms make it hold: declarations are closed by default (an undeclared kind is refused), policy is data (caps and ladders live in versioned config documents, not constants), and value moves only through rules (a reward class and the ledger, never a free meta field). ci/scripts/game-boundary-checks.sh greps the platform for a game's vocabulary and fails the build on a hit.

What can go wrong

Symptom Where to look
503 no_capacity on launch or queue Dashboard overview: free game servers. NoFreeGameServers and MatchmakerNoCapacity alerts
Party stuck in launching Allocator callback to Session; Session's repair poll recovers it
Queue never forms a match matchmaking.rules not loaded (every enqueue is 503 queue_unavailable); MatchmakerQueueStalled alert
Client gets OTNO Ticket expired (60 s), reused, or sent from a different local port than ENet uses
Server flapping Heartbeats missed for 15 s; see the container log for the request_id