The debug sandbox inside Detramotor

Detramotor is my Vulkan engine. Around 41k lines of first-party code, fourteen months of commits. The parts that matter:

• runtime/vl, the RHI layer, and the bulk of the engine. Barriers batched by (srcStage, dstStage) pair and flushed as one vkCmdPipelineBarrier per pair, image and buffer states as first-class values with per-mip and per-layer subresource ranges, queue family separation into graphics / compute / upload / download, thread-unique command pools, timestamp query pools, VMA underneath, lifetime sets keeping a command buffer's dependencies alive until it retires.
• IO and resources: read/write and memory-mapped IO, asset cache, slab and slotter containers, parallel and lazy loading.
• Runtime: threadpool, sync primitives, and ptr<T>, which everything is built on. There is a whole post about that one.
• GFX: rendering components on an ECS. The GPU particle system is one, so are the clouds, the model renderer, the ImGui integration, and the graph renderer behind the Detravisualizer port.

This post is about the debug sandbox, one application on top of all that, and the thing I actually stare at while working. It has views for the editor, the inspector, the scene, the component tree, the console, the log, the profiler and a spawner.

Everything in there is a live pointer into a running engine rather than a capture of one, which matters more than once.

The editor

It edits shader code with syntax highlighting, and any other text file too, because once you have an editor there is no reason to be precious about what it opens.

Completion pulls candidates from three places: the language definition's keywords, a GLSL name list, and every identifier in the buffer you are typing in (rescraped when the document revision moves, so your own uniforms complete). All the shortcuts carry ctrl, because the underlying text widget takes Enter, Tab and the arrows unmodified and there is no taking that back. It does ignore them when ctrl is held, so that is where the popup lives.

Errors show up on their own. A shader document unchanged for 0.6 seconds gets compiled without building a pipeline, and the line-tagged diagnostics go into the gutter. You can turn it off and compile by hand.

Type to find narrows the file grid to names starting with what you typed, and drops after 1.5 seconds of silence. It reads the character queue directly, guarded on the grid being focused with no widget active, so the filter box below keeps its own keystrokes. While a find is running, backspace shortens the search instead of going up a directory.

The file icons

The asset browser needs an icon per file type, and I did not want to ship a set of them, so it bakes them.

There is one 512x512 template PNG, a white page with near-black line art, and a text file naming four regions on it: the extension text, the background behind it, the folded corner, and a content area. Those four rectangles live once in C++, normalised, and go to a compute shader. The shader tints the hue regions (white becomes the type's hue, the black outline stays black) and draws a figure in the content area: lines for text, a picture for images, a wire cube for meshes.

So a file type is three values.

struct ObjPreview : public FilePreview {
  bool handles(const std::string& path) override { return io::fileGetExtension(path) == ".obj"; }
  FilePreviewIcon icon(const std::string&) override { return {"OBJ", 0.28f, FILE_PREVIEW_FIGURE_MESH}; }
  bool thumbnail(FilePreviewContext& c, const std::string& p, const FilePreviewArea& a) override { ... }
};

filePreviewRegister(new ObjPreview);   // last registered wins

The bake is one compute dispatch, recorded into the buffer the sandbox owns before the render pass opens, rate limited to one per frame. One texture per (hue, figure) pair, so a directory with a thousand files holds about a dozen icon textures.

The extension text is not baked. It is drawn on top afterwards with a plain ImGui text call, sized to the mapped text region. Two draw calls instead of one, and the text stays crisp at any cell size while the texture count stays keyed on (hue, figure) rather than exploding per extension.

Since the icon is a shader, saving the shader re-bakes every icon in the browser. The pipeline gets polled once a frame and the cached icons are retired when it changed. Retired, not freed: ImGui caches a descriptor per image address forever, so reusing an address hands you somebody else's picture.

The inspector

It shows pipelines, buffers and textures.

The pipeline list is my favourite bit of plumbing in here, because it is not a list. ptr<T> keeps a registry of live instances, so the view asks for every live ptr<vlPipeline> and there is nothing to register or keep in sync. Pipelines are created asynchronously, so an entry may still be pending, and the view checks for that without forcing it.

What I would like it to do is command buffer submissions, in the style of RenderDoc. An engine can do that more cheaply than an external tool: the capture happens in-process, so it reads the buffers and textures a submission used, and for the static ones (most of them) nothing has changed, so there is nothing to snapshot. It would suit an engine I want to keep low level and code oriented. Whether I build it is another matter, since RenderDoc exists and is more than enough for me.

The sandbox also draws the pool itself: per-thread usage averaged over time, tasks dispatched per interval, scrolling like a system monitor. Scheduler contention shows up here before it shows up as a frame time spike. And there is a pile of custom ImGui themes, which is not important but is nice.

The thumbnails

The asset browser previews every image in the directory. Project textures go up to 4096 on a side, so thumbnails load at a reduced size on a worker thread and appear when they appear. There is no loading system behind that. This is the whole thing:

ptr<vlImage> vlImageLoadThumbnail(gfx::vlQueueType type, io::DataSource source) {
  return thumbnail_cache.get(source.getName(), [type](const std::string& _source) { //
    io::DataSource source(_source);
    return sys::sched([source, type]() {
      return vlJob::runret(type, sys::getThreadIdx(), [source](ptr<vlCommandBuffer> cmd) {
        serialize::ReadingParameters params;

        params.desiredHeight = 128;
        params.desiredWidth  = 128;
        return vlImageLoadImmediate(cmd, source, params);
      });
    });
  });
}

Read it outside in. Any thread can call this. The cache gets asked for the texture, and if it is not there, creating it is deferred to a task on the pool. The caller gets a ptr<vlImage> back either way, immediately.

The awkward cases are the point. A thread asks for a texture another thread is already loading, or one being processed right now, or one about to be freed. None of those need handling, because the cache holds a weak reference that can point at a computation in flight rather than only at a finished value, and ptr knows how to promote that into an owned value. Two concurrent misses on the same key reconstruct handles onto the same pending state, which is request coalescing for free. Most engines need a separate in-flight table with its own mutex and condvars for that.

There is a decent amount of shenanigans in the ptr<T> template design. It buys one type that is a cache entry, a lazy value, an async value and a dispatch handle at once, and the call site above is what that buys.

What runs on the worker

sys::sched puts the lambda on the pool. vlJob::runret is the part that talks to Vulkan:

//vlJob
template <typename F>
auto runret(gfx::vlQueueType queue, int index, F&& function) {
  using RetType = std::invoke_result_t<F, ptr<gfx::vlCommandBuffer>>;

  ptr<gfx::vlCommandBuffer> cmd = gfx::vlGet()->getCommandPool(queue, index)->createCommandBuffer(VK_COMMAND_BUFFER_LEVEL_PRIMARY);

  if (cmd.isNull()) {
    return RetType{};
  }

  cmd->begin();
  auto retValue = function(cmd);
  cmd->end();

  {
    PROFILE_ZONE_N("[GFX/JOB] Submit");
    gfx::vlGet()->getQueue(queue, index)->submitSingleFence(cmd)->wait(true);
  }

  return retValue;
}

Record a primary command buffer, submit it, wait on a fence. The queue index is the thread index, and several threads can submit to the same queue because it sits behind a mutex. The Vulkan fence is the task completion signal, so there is nothing to invent: the job finishes when the fence signals, the ptr resolves when the job finishes.

On submitting per task

The usual advice is to minimise vkQueueSubmit calls, and it is good advice. It is just not what decides this case. A submit here sits next to reading the file off disk, decoding it, resizing it to the thumbnail size and issuing the transfer, and against all of that the submit itself is small. The textbook shape (workers recording, a fork-join, one submit at the end) would not buy anything measurable.

What it does cost is contention. Worker threads and the render thread want the same queue, and a worker holding it means the render thread waits. Which is exactly what multiple queues exist for, except this platform exposes a single queue in the graphics family.

The fix is to run these uploads on the transfer family instead, and issue a queue family ownership transfer barrier when the image comes back if one is needed, which is a flag on the vlImage. That is what I am working on now, so the model supports that mode too.

Two things that will bite you

Blocking silently is a fine default for code that wants a picture and a bad one for code that must not stall, so there are functions to ask whether a handle is still attached to an unfinished task and then choose: block, or skip and take a sentinel. The rule in the sandbox is absolute: never dereference one of these on the render thread. Ask if it is ready, draw the type icon if it is not.

The second one is nastier. The lazy variant hands the load to a pool worker and returns immediately, and its handle resolving means the upload was recorded, not executed. The worker records into a secondary buffer, and those get executed into the frame's primary at the top of the next frame. So an upload recorded during frame N runs at the start of N+1, and sampling the image in N reads whatever was there before. The image preview skips the frame a handle resolves on and records its layout barrier on the frame after, because any earlier it would sit in the primary ahead of the upload it is meant to follow.

That also decides which loader the icon bake uses. Anything that reads a texture once and keeps the result forever needs the plain cached load, whose upload lands in the caller's own buffer ahead of it. A blank read in a baked icon is cached for the life of the process rather than corrected next frame.

Is it fast enough

I ran a Cholesky factorisation in task form, once with OpenMP tasks and once with ptr<T>, and the results are similar. OpenMP builds a static dependency graph with hash tables, which gives it a real margin when tasks are genuinely tiny, on the order of microseconds. Almost everything the engine schedules is 300 microseconds or more, and at that size the difference disappears into the noise.

Here is the scheduler's own trace of one of those runs, drawn by the sandbox. 18x18 blocks, 1140 nodes, 2907 edges, 24 workers.

Scheduler trace of a blocked Cholesky factorisation. Top: the summary numbers. Middle: one lane per pool thread, hollow head is time a task spent queued before a worker picked it up. Bottom: the dependency graph laid out left to right by depth, with the critical path outlined. Open it full size.

That 12.79x wants reading against the machine rather than against the pool size. This is a 12 core CPU with hyperthreading, so 24 logical threads and 12 cores of actual throughput. An average parallelism of 12.79 is the physical core count, and the 53.3% occupancy the trace reports across 24 lanes is what a fed pool looks like here, not work being left on the table. The remaining distance to 18.15x is the tail of the factorisation, which is critical path bound: at the end there is nothing left to schedule regardless of how many threads you own.

A proper comparison against OpenMP with real numbers deserves its own post, so that is where it will go.