Skip to content

Google Cloud Run Infrastructure

This section documents the video processing pipeline deployed on Google Cloud Platform (GCP). This infrastructure is responsible for offloading heavy video processing tasks from our real-time servers.

Architecture Overview

We utilize a Serverless Asynchronous Pattern. Instead of keeping a server running 24/7 waiting for recordings, we use a lightweight "Listener" Service that triggers a heavy-duty "Job" only when needed.

graph TD
    LK[LiveKit Server] -- Webhook: egress_ended --> A[<b>Listener Service</b><br/>Always On / Scalable]
    A -- Trigger via API --> B[<b>Upload Job</b><br/>One-off Execution]
    B -- 1. Download --> R2[Cloudflare R2 Bucket]
    B -- 2. Upload --> YT[YouTube API]
    B -- 3. Log --> GS[Google Sheets]
    B -- 4. Notify --> SL[Slack]

    style A fill:#4285f4,stroke:#1967d2,color:#fff
    style B fill:#4285f4,stroke:#1967d2,color:#fff

The "Single Image" Strategy

To simplify our CI/CD pipeline and ensure code consistency, we build a single Docker image containing the entire codebase (qubital-pipeline).

However, we deploy this same image in two different configurations by overriding the entry point command. This allows us to use the same artifact for distinct operational roles without maintaining separate build pipelines.

Component Entry File Purpose Resources
Listener Service dist/service.js Lightweight HTTP server. Verifies Webhooks and triggers the Job via the Google Cloud API. Low (Minimal CPU/RAM)
Upload Job dist/job.js Heavy worker. Downloads video files to the filesystem, processes metadata, and performs uploads. High (2Gi+ Memory)

Prerequisites

Before attempting a deployment, ensure you have the following installed and authenticated:

  • Google Cloud CLI (gcloud): Installation Guide
  • Docker: Running locally to build images.
  • Permissions: You must have Owner or Editor role on the GCP Project video-uploader-480122.

Deployment Methods

  • CLI Deployment (Recommended)
    The standard procedure for engineers. Scriptable, reproducible, and less prone to configuration errors.

  • Console Deployment
    Visual deployment via the Google Cloud Dashboard. Useful for quick debugging, inspecting logs, or verifying secrets.