Terra Diem origins

Terra Diem is planned to be a hybrid sandbox/exploration/RPG game based on uncovering the secrets of the lost Terran civilization and their earth-shaping powers. Building this game is what this blog is about. This post is the development backstory leading to the present day.

Before there was a game

Learning Vulkan

Terra Diem began as a learning exercise. At the end of 2019, we were launching Stadia at work and I was personally interested in learning Vulkan as it was the graphics API developers needed to use on the platform. As a game developer in a past life, I had worked on graphics-adjacent work and understood the high level concepts, but I was by no means familiar with the nuts and bolts. What I wanted was an understanding of the basics, and what better project than a Minecraft-esque cube world to learn with.

Vulkan is a very low level graphics API and is famously tedious to get anything going with. It can take upwards of 1000 lines of code to just bring a simple triangle up on the screen. Luckily there are some great tutorials out there (the easiest to find and what I used was https://vulkan-tutorial.com/).

Making an engine

Pretty quickly I got distracted. If I was going to have a game to work in, I would need an engine to power it. Of course, there are many very good, free game engines out there. This includes open source engines (like Godot) and commercial game engines (like Unreal Engine and Unity). However, there were several things steering me away from these options:

  • I wanted to learn Vulkan by making a renderer from scratch. These engines already had very full-featured renderers. I would either have to port their certainly complex renderer to Vulkan, or it would be unnecessary if they already had one.

  • I’ve worked on game tools, engines, and platforms for my entire career. Making an engine was my happy place.

This is a really common pathology among software engineers in the game industry. While not universal, I expect there is around a 1-to-1 relationship of hobby game engine to game programmer (if not 2-to-1 or more). The ironic thing is, it is almost never the best choice if you want to focus on the game. A modern game engine is a very large and complex collection of software and tools. Luckily, I wasn’t doing this to make a game (yet), but wanted to learn and play with the tech.

Over the next few months I built some common bread-and-butter tech found in most/all game engines: game state machine, messaging system, file system, resource system, state management / validation, and various other data structures and utilities. I won’t go into the details here, as each would easily be a post on its own, but I may write some technical articles later if there is interest (message me on the Contact page if you are interested).

First light

By September of 2020, I finally had my engine and an example “Block World” game to show for it.

Calling this a game is more than a little bit of a stretch. It is mostly a tech demo of the engine, with the interaction being a fly-camera with no collision and the ability to add or remove blocks in the world.

The current engine and this first version of the project is open source and can be built and run for those interested by going to the Game Bits GitHub project.

Early days of Terra Diem

Once I had Block World up and running, a funny thing happened: I got interested in the game.

More than cubes

This started innocently enough, with the desire to extend the world beyond just cubes. I had spent some time on this earlier in a little Unity test project I made way back in May of 2019. This supported an “erosion” algorithm, which would remove any exposed blocks and corners of a cube (breaking it into 2x2x2 sub cubes). It could then be rendered as two alternatives to cubes: Marching cubes or slope mesh (what Terra Diem uses now).

Marching cubes definitely looked nicer (and is used in a lot of games now – it may look familiar). However, slope mesh looks almost as nice, and has one really nice property: All of the mesh is confined to being within a block, instead of between blocks. See the Terrain article for details on what “slope mesh” is.

This means each block may be trivially be added or removed independently of the blocks around it. This is great for a game where one adds or removes blocks. Games that use marching cubes or similar algorithms generally feel a lot “squishier”, where the mesh sort of deforms broadly when you add or remove material.

So with this idea in place, I ported my the C# code from my Unity project into Terra Diem and suddenly I had pretty cool looking terrain.

Game design

At this point, I saw the glimmer of a game I wanted to play. While it certainly had some shared heritage with Minecraft and the near-infinite set of similar indie games that came after, Terra Diem had the beginnings of its own style that I could build on.

This got me to thinking about what the game should be. I am a big fan of both exploration / sandbox games (like Minecraft) and high fantasy open world RPGs (like Witcher 3 and Skyrim). Terra Diem was going to take inspiration from these and craft (pun intended) a new experience.

A full discussion of the game design is left to a future post, but in broad strokes Terra Diem is planned have the following:

  • Large (but bounded) procedural world with crafted elements. At a macro scale the world is generated as a whole up front, with the micro scale leveraging noise algorithms (like Perlin noise) for detail. Cities, ruins, landmarks, etc. may be pre-authored and placed or adapted into the world.

  • Progression system based on gear and “terra” crystals which unlock crafting, resource gathering, and exploration. From an RPG standpoint, it is “stat light” – think more Zelda than D&D.

  • Multiplayer support with dedicated servers for groups of people.

  • Builder, sandbox, and story game modes.

  • Modding tools for both cosmetics and gameplay.

Regarding RPG elements

Building any game is a huge amount of work, but building an open world RPG is a much much bigger undertaking. As a one-man show (and not being a game designer or artist), I need to be highly restrained in how I approach the RPG elements. The approach I am taking is one of building tools for the community (such that there is one) to craft and build quests, set pieces, scripts, for the game.

I certainly plan to take some time to work toward official Terra Diem story content, but the focus of the alpha build will be on the initial tooling and sandbox / exploration gameplay.

Development history

The following is the complete-yet-condensed development history of Terra Diem (and Game Bits) to date. Most of this time was spent in fits of productive bursts (1-3 weeks at a time) with long gaps of inactivity or twiddling around. I am now in full-on development, so hopefully the pace will pick up.

  • Mar 2019 - May 2019: An idea
    • Created a test Unity project “Blockworld” to play around with voxel-style editing.
    • Editing volume in a fixed 32x32x32 grid of voxels which were “on” or “off”.
    • Support for three rendering modes:
      • Individual cubes
      • Marching cubes mesh
      • “Slope” mesh generated via an erosion algorithm (what Terra Diem uses today).
  • Jun 2019 - Sep 2019: 4+ months – Nothing happens
  • Oct 2019: Project start
    • Domain names acquired. Terra Diem and Game Bits repositories created. Maybe I’ll do something, someday.
    • Game state machine created in Game Bits.
  • Nov 2019 - Jan 2020: 3+ months – Nothing happens
  • Feb 2020: Learning Vulkan?
  • Mar 2020: Game Bits foundations
    • Explicit thread safety in classes.
    • Callback utility classes.
    • Game state machine improvements.
  • April 2020: Create game message system
    • Added cross-component message system.
  • May 2020: Create file system
    • Abstracted file system with custom protocols, supporting in-memory and on disk file systems.
  • June 2020: Rendering a triangle
    • Vulkan work restarts.
    • Vulkan triangle can be rendered using first version of renderer. Everything is hard coded still.
  • July 2021: Create render system
    • Lots of Vulkan work and generalization into a graphics API agnostic Render System.
    • Basic support for vertex/fragment shaders, textures, materials, and mesh.
  • Aug 2020: Release the render system
    • Add IFF-style chunk file support for loading / saving compound resources.
    • Render system integrated with resource system. Shaders, textures, materials, mesh, etc. can all be read or written.
    • Promoted renderer work to Game Bits.
  • Sep 2020: Block world
  • Oct 2020: Slope mesh
    • First version with “slope” mesh and terrain erosion.
  • Nov 2020: 1+ months – Nothing happens
  • Dec 2020: Fibers!
  • Jan 2021: Optimization and hardening
    • Major optimizations to the Job system. Refactored to use moodycamel::ConcurrentQueue.
    • Run entire game in job system. Terrain chunks are built asyncronously in the background.
    • Added perlin noise caves to the game.
  • Feb 2021: Lighting
    • Added per-block lighting with support for lighting from the sky and placed light sources.
    • The Terra Diem tower is created.
  • Mar 2021: Trees begin
    • Initial version of tree trunk / branch generation done. There are no leaves yet.
  • Apr 2021 - Oct 2021: 7+ months – Nothing significant happens
  • Nov 2021: Lighting improvements and terrain
    • Smooth lighting implemented across blocks. Lighting work moved to GPU.
    • Partial block support added for stairs and other “manufactured” type blocks.
    • Partial amount support for loose and liquid materials. Snow and sand support added.
    • Multiple materials and shapes can coexist in a block. For instance: piling snow on sand in a partial slope block of rock. See Terrain article.
  • Dec 2021: Making a game
    • Started this blog.
    • Commit to dedicating focused time to build and release this game.