VJ Loop Station

A high-performance, zero-latency video mixing tool designed for live visual performances and hardware synchronization. Built with a modern web stack and packaged natively with Tauri, this application treats video clips as WebGL textures to enable instant, seamless switching and advanced visual effects without buffering or black frames.

Overview

VJ Loop Station bridges the gap between traditional video playback and dynamic, live visual performances. By synchronizing playback speed to an incoming hardware MIDI clock or a high-precision internal simulation, it ensures visual continuity and zero drift over long sets.

Key Engineering Achievements

• Live/Queue Strategy: Implemented a dual-deck system where clips are pre-loaded into WebGL textures, allowing instant switching or beat-synced shader transitions.
• Custom WebGL Pipeline: Engineered a Three.js rendering pipeline utilizing ping-pong buffers. This enables recursive texture lookups for visual feedback loops and data-moshing style corruption effects.
• Drift-Corrected Clocking: Developed a custom timing engine utilizing requestAnimationFrame and time accumulation. This prevents the drift commonly associated with browser-based timers and ensures tight synchronization with external hardware.
• Automated Uniform Mapping: Created a metadata-driven pipeline to dynamically bind UI controls and LFOs directly to shader uniforms, abstracting the complexity of GLSL variable management.
• Tauri Native Integration: Leveraged Rust to handle high-priority MIDI processing and OS-level file system tasks, bypassing browser sandbox limitations.

Technology Stack

• Frontend: Vue 3 (Composition API), Vite, Tailwind CSS
• Graphics Engine: Three.js (Custom WebGL Shaders)
• State Management: Pinia (with Undo/Redo architecture)
• Desktop Runtime: Tauri v2 (Rust)
• Hardware Interface: midir (Low-latency Rust MIDI implementation)

Core Mechanics

Mathematical BPM Synchronization

Playback rates are calculated dynamically rather than relying on standard video playback APIs. This guarantees that a clip fits perfectly into its designated bar length at the current global BPM.

// Dynamic playback rate calculation
Rate = originalDuration / (barLength * 4 * (60 / bpm))

Phase Sync

To maintain alignment between audio and visuals, the system features a Phase Sync mechanism. This instantly resets the visual "Beat 1" to the audio source with a single trigger, allowing performers to recover from tempo shifts instantly.

Beat-Synced Automation & Filters

Visual filters (such as RGB Split, Pixelate, and Feedback Corruption) can be dynamically modulated using the built-in automation engine. Parameters can be assigned to Low Frequency Oscillators (LFOs) with two distinct modes:

• Continuous Sweeps: Sine wave modulators mathematically locked to musical subdivisions (e.g., 1/4 beat, 1 bar, 2 bars).
• Pulse Patterns: Rhythmic trigger envelopes that fire strictly on designated beats (e.g., Beat 1, Beats 2 & 4). This generates a sharp attack with a linear decay, perfect for percussive visual hits that snap back to a baseline state in perfect synchronization with the audio.

Development Setup

To run this project locally, ensure you have Node.js and Rust toolchains installed.

# Install dependencies
npm install

# Start the desktop application in development mode
npm run tauri:dev

Performance Optimization Roadmap

Achieving professional-grade video performance within a web-tech environment requires overcoming standard composition limitations:

• Jitter Reduction: Implementation of requestVideoFrameCallback to synchronize WebGL texture uploads precisely with hardware video frame availability.
• Zero-Latency Decoding: Migration from HTML5 <video> elements to the WebCodecs API (VideoDecoder). This will enable hardware decoding directly into WebGL external textures, bypassing the CPU-to-GPU copy phase and unlocking true frame-accurate seeking.

Project Architecture

├── src/                        # Vue Frontend Environment
│   ├── components/             # Presentation (Stage, Control Panel, Multiview)
│   ├── composables/            # Core Engine (Three.js Renderer, Video Deck, MIDI)
│   ├── stores/                 # Reactive State Management
│   └── utils/                  # Shader Definitions, Automation Logic
└── src-tauri/                  # Rust Backend Environment
    ├── src/                    # Native Window Management, MIDI processing
    └── tauri.conf.json         # Build Configuration