Back to projects
Unity / Tools2025

Unity Scene Capture API

Production-ready scene capture system with multi-buffer output and batch processing for ML dataset generation.

UnityC#HLSLCustom Editor
ML DatasetsMulti-Buffer CaptureFactory PatternBitwise FlagsIDisposableCustom InspectorComputer Vision
View on GitHub

Overview

A flexible scene capture API for generating machine learning datasets from Unity scenes. Supports simultaneous Color/Depth/Normal buffer capture, 360° orbital rotation, batch processing, and procedural scene generation—all through a clean factory-based API.

Key Features

  • Multi-channel capture: Color, Depth (eye-space grayscale), Normal (world-space RGB)
  • Batch processing with 360° orbital rotation around target
  • Random scene generation with procedural object spawning
  • Three factory method overloads for flexible camera instantiation
  • Bitwise enum flags for channel selection (Color | Depth | Normals)
  • IDisposable pattern for proper resource cleanup
  • Custom Inspector UI with folder picker and runtime controls

Technical Breakdown

How It Works

  1. 1CaptureSession creates or wraps a camera with specified resolution
  2. 2Replacement shaders (DepthGrayscale, NormalsRGB) swap rendering mode per channel
  3. 3Camera.RenderWithShader captures each buffer to RenderTexture
  4. 4Textures are encoded to PNG and saved to organized folder structure
  5. 5Coroutine-based batching captures multiple frames with WaitForEndOfFrame
  6. 6Multi-capture mode orbits camera around target at configurable radius

Architecture

  • CaptureController.cs — MonoBehaviour component with Inspector UI
  • CaptureSession.cs — Core API with factory methods and IDisposable
  • CaptureControllerEditor.cs — Custom Inspector with folder picker
  • DepthGrayscale.shader — Eye-space depth → grayscale
  • NormalsRGB.shader — World-space normals → RGB channels

Notable Details

  • •Factory pattern: Create(Camera), Create(Transform), Create(Vector3, Quaternion)
  • •Bitwise flags allow combinations: CaptureChannels.Color | CaptureChannels.Depth
  • •IDisposable destroys camera if API created it, skips if user-provided
  • •Coroutine yields ensure GPU completes rendering before capture
  • •Output organized: captures/{name}/color/, depth/, normal/

Setup & Usage

Installation

  1. 1Clone repository to Unity project
  2. 2Add CaptureController component to any GameObject
  3. 3Configure channels, resolution, and save path in Inspector
  4. 4Press Space to capture or call TakeCapture() from code

Quick Start

using SceneCapture;
using (var session = CaptureSession.Create(Camera.main, "Assets/Captures"))
{
    session.Channels = CaptureSession.CaptureChannels.All;
    session.Resolution = new Vector2Int(1920, 1080);
    session.Capture("MyScene");
}

Parameters

NameTypeEffect
ChannelsFlagsWhich buffers to capture (Color/Depth/Normal)
ResolutionVector2IntOutput image dimensions
Capture CountIntNumber of frames in batch mode
Rotation RadiusFloatOrbital distance from target object

Challenges & Learnings

  • Designed clean factory API with multiple instantiation patterns
  • Implemented bitwise enum flags for flexible feature combinations
  • Managed camera lifecycle with IDisposable ownership tracking
  • Built custom Inspector UI for enhanced usability
  • Synchronized coroutine timing with GPU rendering pipeline

Outcomes

  • ✓Production-ready tool for generating ML training datasets
  • ✓Clean separation between MonoBehaviour wrapper and core API
  • ✓Demonstrates 5 professional C# patterns in practical context