Continent Generator

Table of Contents

Summary

Observations

Actions

Reward Function


Summary

This page documents the architecture of the Continent Builder neural network, one of several neural networks that work together to achieve Reinforcement Learning Procedural Content Generation (RL-PCG). It focuses on creating land masses that meet input parameters; for example, if the developer or player want five continents then this network generates them on the 2D World Grid. The continent generator is powered by Unity ML-Agents, as well as a randomized seed for each continent.

Note: There are three states the environment can be in: Training, Inference, and Storage. Each environmental state will result in different states for the RL_Agents.

  1. Randomly seed <desired_number_of_continents> number of land tiles in grid.
If four continents are specified, start with four randomized seeds.
  1. Allow agent to place land OR mountain tiles near the seeds and reward or punish it based on these decisions (rewards and punishments specified in the Reward Function). Helper functions process the game board and simplify the possible Actions.
    1. The cursor (current location) is the location of the last tile placed by the RL Agent. If training (or inference) has just begun, this will be at one of the seed locations.

Components

  1. Location within the Unity scene: ML_Agents/Continent_Builder

Observations

The Continent Builder receives the following parameters:

  1. The eight tiles surrounding the previously placed tile (or one of the seeds, if training/inference has just begun). Basically, the tile placed marks the next point from which observations will be taken about the surrounding tiles.
  2. Current continent parameters. This is counted by simply keeping track of the tiles that have been placed using a helper function (no complicated grid sorting here).
    1. Number of Land Tiles placed (Elevation = 0)
    2. Number of Mountain Tiles placed (Elevation = 1)
  3. Current grid parameters. This is counted by simply keeping track of the tiles that have been placed using a helper function (no complicated grid sorting here).
    1. Number of Land Tiles placed (Elevation = 0)
    2. Number of Mountain Tiles placed (Elevation = 1)
  4. Continent parameter limits. This is set by the player or the developer and is used to tell the Agent how to shape the continents (more or less mountainous).
    1. Number of Land Tiles before punishments begin (Elevation = 0)
    2. Number of Mountain Tiles before punishments begin (Elevation = 1)

Actions

  1. Place Land Tile.
  2. Place Mountain Tile.
  3. Move on to next continent (or end if last continent).

Reward Function

Rewards:

  1. (Small) Placing a Land/Mountain tile on an ocean tile.

Punishments:

  1. (Large) Placing a Land/Mountain tile on an already existing Land/Mountain tile.
  2. (Small * <number_of_mountain_tiles> / <desired_number_of_mountains>) At first punishment is very small, until the number of mountain tiles exceeds the desired number of mountain tiles.
  3. (Small * <number_of_land_tiles> / <desired_number_of_land>) At first punishment is very small, until the number of land tiles exceeds the desired number of land tiles.

Helper Functions

Helper functions are C# scripts in the Unity project that setup the environment for the Agent to operate in, handle data, etc.

  1. Generate_Continent_Seeds.cs
    1. Purpose: Generates seeds for the continents (Random points within the grid for the Agent to work off of).
    2. Inputs:
      1. grid_properties: Drag and drop the Tile_Manager_RL2D object (in scene) into the Grid_Properties variable in the inspector.
      2. total_seeds: An integer that determines how many seeds will be generated.
    3. Outputs:
      1. continent_seeds: List of the seeds randomly generated each time Play Mode is engaged. Use this to reference them.
    4. Attached to: ML_Agents/Continent_Generator/Continent_Seed_Handler
  2. General_RL_Helper_Functions.cs
    1. Purpose: Can be used by any Generator/RL Agent to get the current state given a coordinate. The current state can be considered the eight tiles surrounding a given point on the board. Note that some of these points may be considered out of bounds.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: