Overview
Generative 3D engineering visualization engine. Turn voice commands, sketches, or prompts into interactive mechanical assemblies in real time.
Project Metrics
- DeepMind × Kaggle: Winner
- Components per scene: 20-30
- Generation time: 45-60s
System Architecture & Implementation
Overview
FRIDAY is a generative immersive visualization engine for engineering teams. Type a system name, upload a sketch, or speak a command, and a detailed 3D assembly compiles in front of you, ready to rotate, explode, scan, and interrogate.
It is not a chat app. It is a neural spatial architect that writes geometry the way language models write prose.
The problem it solves
Engineering teams lose days between "we need a visual of X" and actually having one. CAD is heavyweight. Drawing tools are slow. AI image generators produce inspiration, not assemblies you can rotate. FRIDAY collapses that cycle to about a minute and produces something you can actually interact with.
How it works
- Input layer captures a voice command, a text prompt, or an uploaded image.
- Generation layer runs a two-phase Gemini pipeline. Phase 1 (Pro) reasons about the whole object, picks real-world proportions, and outputs a scaffold of 24 to 28 components with exact coordinates. Phase 2 (Flash-lite, two parallel halves) fills in the primitive structure for each component so nothing clusters at origin.
- Rendering layer materializes the scene in React Three Fiber. Holographic translucent walls, cyan silhouette edges, automatic camera framing, scan planes, and a shader pass that reveals interior components through outer shells.
- Interaction layer exposes voice (Gemini Live native audio), hand gestures (MediaPipe), keyboard shortcuts, and click-to-inspect.
What it ships with
- Generative system builder with a 20-30 component target and schema-validated output
- Holographic CAD rendering with bright cyan edges, transparent surfaces, and see-through cabinet walls
- Voice-native workflow using Gemini Live with ephemeral token generation via your own API key
- Deep scan diagnostics that surface stress points per component
- Gesture layer for explode, rotate, and zoom with hand tracking
- BYOK security - your API key stays in your browser, never on a server
Technical architecture
- All Gemini calls route through Vercel Node functions as a BYOK proxy. No server-side key required.
- AudioWorklet pipeline downsamples mic input to 16 kHz Int16 PCM in a dedicated audio thread before handing frames to the Live API.
- The 3D scene auto-fits with drei's
Bounds and gracefully downgrades quality tier on low-FPS devices via PerformanceMonitor.
- Per-IP rate limiting via in-memory tracker. Saved systems are stored in browser localStorage with a shareable hash URL.
Engineering problems solved along the way
- Vercel Edge Functions have a 25 second ceiling. Switched
/api/analyze to Node runtime with maxDuration: 60 and an ESM import fix for the runtime.
- The Gemini Live WebSocket started dropping on frame 1 with close code 1007. Root cause: the
realtime_input.media field was deprecated. Switched to audio and the session stayed alive.
- Single-shot Pro for 30 component generations kept timing out. Split into a skeleton phase and parallel structure phase, both aware of the full skeleton context so primitives stay spatially coherent.
Why it matters
Most AI tools for engineering stop at generating an image. FRIDAY generates an assembly, with component names, connections, and positions you can manipulate. It is the closest I have shipped to a browser-native spatial language model.
Recognition & awards
- Winner: Google DeepMind × Kaggle Vibe Code with Gemini 3 Pro Hackathon (Developer Tools)
Selected as one of 50 winning projects worldwide. Built for generative 3D CAD visualization, voice-driven spatial assembly, and two-phase geometric reasoning.
Links