Living City AI — Realtime NPC Conversations

Walk up to any pedestrian and talk to them with your voice. They answer in about a second, in a natural spoken voice, in character — and they remember you. Persistent NPC memory, five-dimension relationships, and city-wide gossip that spreads real events.

Walk up to any pedestrian and talk to them with your voice. They answer in ~1 second, in a natural spoken voice, in character — and they remember you.

No menus, no chat windows, no push-to-talk. Your controls are never taken, so it's perfect for cinema, RP streams and content creation.

What v2 adds: the city is alive around the conversation

• Persistent NPC memory — placed NPCs (clerks, guards, staff) remember every player individually, across sessions and restarts. Argue with the clerk on Monday and on Tuesday he opens with "oh hell nah, you again."
• Five-dimension relationships — trust, respect, fear, familiarity, hostility, all independent. A gangster can respect you and still hate you.
• City events & gossip — shootings, robberies, chases, arrests ripple outward. Witnesses know instantly and can repeat concrete details; cops hear it on dispatch; the neighbourhood hears it later; big stories make the news. Peds two blocks away say "I heard shots, didn't see anything."
• Derived mood — night shifts, rain, rough neighbourhoods and nearby violence change how people talk to you, with zero extra API calls.

Core features (v1)

• OpenAI Realtime speech-to-speech (~1s to first audio, barge-in supported — interrupt them mid-sentence and they stop).
• Proximity conversations: face someone within 4m and speak. A dwell gate stops passers-by from costing you sessions.
• Physical compliance: NPCs actually DO what they agree to — follow you, stop, turn around, hands behind back, get in the car, kneel, flee, and (if you push the wrong person) pull a gun on you. Tool-call driven, not text parsing.
• 80+ ped models mapped to roles (clerk, bouncer, dealer, cop, EMS, security, dancer…), each with its own persona, all editable in roles.lua.
• Per-model accents and pinned voices; gender-split voice pools.
• Player reputation system with a public/street split and an undercover mechanic (badge item / on-duty detection).
• Escalation system: certain roles and neighbourhoods might be carrying.
• SpeakOnce / Prompt exports so YOUR scripts can make any ped deliver a live, in-character line.
• Server-side Chat/Line text bridge (cheap chat-completions) for written NPC replies in other scripts.

Requirements

• ox_lib, oxmysql (standard on QBox/QBCore servers)
• An OpenAI API key with Realtime access
• qbx_core is auto-detected for job/citizenid; falls back gracefully without it

Install

1. Drop rr-realtime into your resources and ensure rr-realtime after ox_lib and oxmysql.
2. Add your key to server.cfg:

`` set openai_api_key "sk-..." ``

1. Start the server. Tables are created automatically. Walk up to a ped and talk.

Configuration (all open files)

• config.lua — voices, session behaviour, costs/limits, player reputation, escalation, physical actions.
• config_memory.lua — city events, gossip channels (who hears what, how fast), emotion baselines, NPC memory.
• roles.lua — ped model → role/persona/accent/voice mapping.
• html/ — the (headless) NUI audio layer.

The API for your other scripts

-- Publish a world event; witnesses can repeat the facts verbatim:
exports['rr-realtime']:BroadcastEvent({
    type = 'robbery', coords = coords,
    facts = { 'Two men in ski masks.', 'They left in a red Kamacho.' },
})

-- Make a ped say one live, in-character line (opens/closes its own session):
exports['rr-realtime']:SpeakOnce(ped, 'React to the explosion you just heard.')

-- Written text replies through the same brain:
exports['rr-realtime']:Chat(messages, opts, function(err, text) ... end)

Also exported: Prompt, IsActive, IsEngaged, CurrentPed, Report, Record, Brief, NpcBrief, RememberEncounter, Line, SceneContext, DoAction, ActionList.

Commands

• /whoknows [role] — debug: which live events would a ped of that role know about where you're standing, and via which channel.
• /citytest [kind] — inject a test event.
• /freeped — release a stuck ped. /rtstart, /rtstop — manual session control. rtchat <text> (server console) — test the text bridge.

Cost controls

Audio tokens are the real cost (~40/second of speech). This resource is built around that: sessions only open when a player deliberately engages (dwell gate), idle sessions close in 12s, one-shot lines close as soon as they're spoken, a token ceiling caps runaway replies, and the entire gossip/mood/memory-recall layer is templated arithmetic — zero API calls. The only recurring model call besides live speech is one short text summarization after a conversation with a placed NPC ends (disable with Config.Memory.Npc.summarize = false).

Support

RR Interactive — see the store page for support contact.