WorldGen Pipeline
WorldGen Pipelineπ
The WorldGen pipeline transforms LocalGPT Gen from βthe agent places things one by oneβ to βthe agent designs worlds through a structured interface.β Instead of the agentβs model generating geometry directly, it generates parameters β layout specs, density hints, style references β and procedural systems handle the actual placement.
This mirrors the architecture described in Metaβs WorldGen paper: language models should generate parameters, not geometry.
Pipeline Stagesπ
1. Plan gen_plan_layout Text β BlockoutSpec (JSON)
β
2. Blockout gen_apply_blockout BlockoutSpec β terrain + regions + paths
β
3. Navmesh gen_build_navmesh Terrain β walkability grid
β
4. Populate gen_populate_region Regions β entities (hero/medium/decorative)
β
5. Evaluate gen_evaluate_scene Screenshot + highlighting β the agent's feedback
β
6. Refine gen_auto_refine Automated evaluate β adjust loop
β
7. Edit gen_modify_blockout Non-destructive region changes
β
8. Regenerate gen_regenerate Incremental update after editsTools (15)π
Layout & Blockoutπ
| Tool | Description |
|---|---|
gen_plan_layout | Generate a structured world layout plan from a text description. Returns a BlockoutSpec JSON with terrain type, regions, paths, and density hints. |
gen_apply_blockout | Apply a blockout spec to create terrain, regions with hero slots, medium density zones, and connecting paths. |
gen_modify_blockout | Add, remove, resize, or move blockout regions with incremental regeneration. Supports non-destructive editing. |
gen_regenerate | Regenerate regions after blockout changes, preserving manually placed entities. |
Placementπ
| Tool | Description |
|---|---|
gen_populate_region | Populate a region with entities using three-tier placement: hero (landmarks), medium (structures), decorative (props/vegetation). Respects collision clearance and ground snap. |
gen_set_tier | Set an entityβs placement tier (hero, medium, decorative). Affects clearance and placement priority. |
gen_set_role | Set an entityβs semantic role (ground, structure, prop, vegetation, water, sky, light, effect). Enables bulk operations by role. |
gen_bulk_modify | Modify multiple entities by role or tier. Example: recolor all vegetation, rescale all decorative props. |
Navigationπ
| Tool | Description |
|---|---|
gen_build_navmesh | Build a grid-based walkability analysis for the current terrain. Detects steep slopes, obstacles, and blocked areas. |
gen_validate_navigability | Check that key points (spawn, landmarks, exits) are reachable via A* pathfinding on the navmesh. |
gen_edit_navmesh | Manually override navmesh cells: block areas, allow previously blocked cells, add connections between disconnected regions. |
Evaluationπ
| Tool | Description |
|---|---|
gen_evaluate_scene | Take a screenshot with optional entity highlighting (red emissive overlay) and camera presets (current, top-down, isometric, front, entity-focus). Annotate with entity names and bounding boxes. For the agentβs own evaluation. |
gen_auto_refine | Automated generate β screenshot β evaluate β adjust loop. The agent evaluates its own output and makes corrections. |
Depthπ
| Tool | Description |
|---|---|
gen_render_depth | Render a depth map of the scene from a specified camera angle (top-down, isometric, front, custom). Outputs a grayscale PNG an agent can hand to its own image tools. |
Three-Tier Placementπ
The population system uses a hierarchy that matches how real environments are built:
| Tier | Role | Clearance | Examples |
|---|---|---|---|
| Hero | Landmarks, focal points | Large (8m+) | Town hall, ancient tree, monument |
| Medium | Supporting structures | Medium (3-5m) | Houses, wells, market stalls |
| Decorative | Detail and atmosphere | None/small | Flowers, barrels, rocks, grass |
Hero slots are placed first at specific positions defined in the blockout. Medium entities fill density targets. Decorative entities scatter to reach the regionβs decorative density.
BlockoutSpec Formatπ
The gen_plan_layout tool returns a JSON blockout specification:
{
"terrain": {
"type": "hills",
"verticality": 0.6,
"roughness": 0.4,
"base_height": 0.0
},
"layout": {
"style": "organic",
"density": 0.5,
"regularity": 0.3
},
"regions": [
{
"id": "village_center",
"bounds": { "center": [0, 0], "size": [20, 20] },
"type": "structured",
"density": 0.7,
"walkable": true,
"hero_slots": [
{ "position": [0, 0, 0], "size": [8, 6, 8], "role": "landmark", "hint": "town hall" }
],
"medium_density": 0.5,
"decorative_density": 0.3
}
],
"paths": [
{
"from": "village_center",
"to": "forest_edge",
"style": "cobblestone",
"width": 2.0
}
]
}Semantic Rolesπ
Every entity can be tagged with a semantic role for bulk operations:
| Role | Description | Example Bulk Operation |
|---|---|---|
ground | Terrain, floors | Retexture all ground |
structure | Buildings, walls | Scale all structures 1.2x |
prop | Furniture, objects | Hide all props |
vegetation | Trees, bushes, grass | Recolor all vegetation to autumn |
water | Rivers, lakes, ocean | Adjust all water transparency |
sky | Sky dome, clouds | Change time of day |
light | Light sources | Dim all lights by 50% |
effect | Particles, fog | Disable all effects |
Scene Decompositionπ
The pipeline analyzes imported glTF scenes to auto-assign tiers and roles:
- Mesh segmentation β splits a single glTF into connected components
- Role inference β flat/large β ground, tall/narrow β structure, small β decorative
- Name matching β entity names like βtree_01β β vegetation, βlamp_postβ β light
This enables semantic operations on imported scenes, not just procedurally generated ones.