AI Crews
Project: Jetsons Living

Project: Jetsons Living

Jetsons Living is a home surveillance platform built for Jetsons Living customers. It combines a cloud-hosted web/mobile app with a self-contained on-premise edge device that runs a local NVR (Frigate), video streams, and AI event detection — all without depending on the cloud for day-to-day operation.

Stilo Solutions builds and maintains this system.


Product architecture

┌─────────────────────────────────────────────────────────────────────┐
│  USER DEVICES                                                        │
│  ┌────────────────────┐   ┌──────────────────────────────────────┐  │
│  │  Web browser       │   │  iOS / Android app (Capacitor)        │  │
│  │  (Amplify, HTTPS)  │   │  (static Next.js export)             │  │
│  └────────┬───────────┘   └──────────────┬───────────────────────┘  │
└───────────┼──────────────────────────────┼──────────────────────────┘
            │  HTTPS via Amplify                │
            ▼                                  ▼
┌────────────────────────────────────────────────────────────────────┐
│  CLOUD (AWS Amplify + MongoDB Atlas)                                │
│                                                                    │
│  Next.js app                                                       │
│  ├── Auth0 authentication                                         │
│  ├── GET /api/servers            ← lists user's paired devices    │
│  ├── POST /api/pair/generate     ← starts pairing (makes a code)  │
│  ├── GET  /api/pair/status       ← polls for device to complete   │
│  ├── POST /api/pair/claim        ← device calls this (no auth)    │
│  ├── POST /api/pair/complete     ← device finalises pairing       │
│  ├── POST /api/pair/update-url   ← device registers tunnel URL   │
│  └── GET  /api/stream/token      ← returns Auth0 JWT + edge URL   │
│                                                                    │
│  MongoDB Atlas: users, paired servers (with apiUrl), pairing codes│
└────────────────────────────────────────────────────────────────────┘

            │  Each server.apiUrl resolves to:
            │  ① LAN: https://jms-edge.home.jetsonsliving.com
            │     (DNS A record → device LAN IP, Let's Encrypt cert via DNS-01)
            │  ② Remote: https://<id>.trycloudflare.com
            │     (Cloudflare tunnel, registered by device on boot)


┌────────────────────────────────────────────────────────────────────┐
│  JMS EDGE DEVICE (on-premise, in the customer's home/building)     │
│  Hardware: mini-PC or Raspberry Pi running Docker                  │
│                                                                    │
│  jms-api (Node.js/Express) — authenticated via Auth0 JWT          │
│  ├── GET /api/events              ← grouped event feed             │
│  ├── GET /api/events/search       ← Frigate semantic search       │
│  ├── GET /api/events/clips        ← filterable paginated clips    │
│  ├── GET /api/events/stats        ← analytics aggregation         │
│  ├── GET /api/frigate/*           ← transparent Frigate proxy     │
│  ├── GET /api/camera/:id/mse      ← live video (MSE stream)       │
│  ├── GET /api/storage/*           ← thumbnails, clip files        │
│  └── POST /api/admin/*            ← reboot, factory reset         │
│                                                                    │
│  Frigate (NVR)                                                     │
│  ├── Ingests RTSP camera streams                                  │
│  ├── Runs object detection (person, vehicle, animal, etc.)        │
│  ├── Stores clips + thumbnails at /mnt/storage/frigate/           │
│  └── Semantic search via sentence-transformers (small model)      │
│                                                                    │
│  MongoDB (local)   ← eventSync.ts polls Frigate every 10s         │
│  Caddy (TLS)       ← DNS-01 cert for jms-edge.home.jetsonsliving.com│
│  Go2RTC            ← WebRTC / MSE stream relay                    │
└────────────────────────────────────────────────────────────────────┘

Setup flow — adding a new device

This is the flow a customer follows to pair their JMS device for the first time. The pairing code API is fully implemented. The device setup UI and cloud API are real.

1. Customer receives JMS device (mini-PC/Pi) pre-loaded with jms-edge firmware

2. Device boots for the first time (no /etc/jms/device.json):
   └── Broadcasts WiFi hotspot "JMS-SETUP" at 192.168.4.1
   └── Runs captive portal web server on port 80

3. Customer connects laptop/phone to JMS-SETUP WiFi

4. Browser opens http://192.168.4.1 → device's setup UI
   └── Customer is prompted for: pairing code + home WiFi credentials

5. In the JMS app (jetsons-management-system):
   └── Settings → Add Server → "Add Server" modal opens
   └── App calls POST /api/pair/generate → gets a 6-char code (e.g. "K7X2M9")
   └── Code displayed on screen for the customer to type into the device UI

6. Customer types the code into the device captive portal:
   └── Device calls POST /api/pair/claim (cloud) → validates code, creates Server record,
       returns { deviceSecret }
   └── Device configures WiFi via nmcli
   └── Device calls POST /api/pair/complete (cloud) → marks pairing complete
   └── Device saves { deviceId, deviceSecret } to /etc/jms/device.json
   └── Device connects to home WiFi, Docker stack starts

7. App polls GET /api/pair/status?code=K7X2M9 every 3 seconds
   └── When pairing completes → returns serverId
   └── App transitions to camera list, device shows up as online

8. On every subsequent boot, device calls POST /api/pair/update-url with its
   Cloudflare tunnel URL so the cloud always knows how to reach it remotely

9. Customer is now on the cameras home screen, their device is paired

Adding a camera

After a server is paired, the customer adds cameras (IP cameras on their home network):

Settings → Configuration → Add Camera
→ Frigate RTSP stream integration
→ Camera name, stream URL, group assignment
→ jms-edge applies the new Frigate config and restarts the detection pipeline

Currently mock: AddCameraModal shows fake network-discovered cameras (MOCK_DISCOVERED). Real implementation requires a ONVIF discovery endpoint on jms-edge and a Frigate config update API. This is tracked as a Gitea issue.


Offline access

The device works without internet once paired:

  • On LAN: frontend connects to https://jms-edge.home.jetsonsliving.com — a real domain with a Let's Encrypt cert (provisioned via DNS-01 Cloudflare challenge). The DNS A record points to the device's LAN IP. No internet required once the cert is cached.
  • Remote: Cloudflare tunnel (*.trycloudflare.com) provides HTTPS access from anywhere.
  • URL resolution: lib/jms-api.ts in the frontend races both URLs — the LAN URL responds in ~1-5ms on local network, so it wins naturally. Off LAN, it's unreachable so the tunnel wins.
  • Auth0: login requires internet (Auth0 is cloud-hosted). Once logged in, the Auth0 JWT is valid for the session and edge API calls work offline.

Frontend app (jetsons-management-system)

Stack: Next.js 16, App Router, TypeScript, Tailwind CSS, Shadcn UI, Auth0 v4, Capacitor 8

Deployments:

  • prod-t branch → Amplify pre-release (testing)
  • main branch → Amplify production

Mobile: Capacitor exports a static Next.js build to iOS/Android. Set IS_CAPACITOR=true to enable static export mode.

Key architecture files:

FilePurpose
proxy.tsAuth0 middleware — guards all routes, redirects unauthenticated users
lib/jms-api.tsResolves edge device URL (races LAN vs tunnel, caches 60s)
app/api/stream/token/route.tsReturns { token, apiUrl } for client-side edge API calls
app/api/pair/*Server pairing lifecycle (generate, status, claim, complete, factory-reset)
app/api/servers/*List + manage paired servers (reads from MongoDB Atlas)
lib/models/Server.tsServer schema: { id, name, apiUrl, status, deviceInfo, location }
hooks/useClipSearch.tsSearch page clips hook (currently mock — crew replaces this)
hooks/useTagSearch.tsSearch page LPR hook (deferred — needs Frigate ALPR)

Edge device (jms-edge)

Stack: Node.js, TypeScript, Express, Mongoose, Frigate, Go2RTC, MongoDB, Caddy

Source layout:

api/src/
  index.ts          — Express app, DB connection, route mounting
  config.ts         — Env vars (FRIGATE_URL, MONGO_URI, AUTH0_DOMAIN, etc.)
  middleware/auth.ts — tokenFromQueryToHeader + checkJwt (Auth0 JWT validation)
  routes/
    events.ts       — GET /api/events, GET /api/events/search
    frigate.ts      — Transparent proxy: GET /api/frigate/* → Frigate API
    camera.ts       — Live streams: MSE, WebRTC, MJPEG
    storage.ts      — Authenticated file access (thumbnails, clips)
    admin.ts        — Reboot, restart-service, factory-reset
  models/Event.ts   — Mongoose schema for Frigate events
  services/eventSync.ts — Polls Frigate every 10s, upserts to MongoDB

setup/              — Captive portal server (runs before Docker on first boot)
  src/server.ts     — HTTP server at 192.168.4.1 (setup UI + POST /api/setup)
  src/provision.ts  — Calls cloud pair/claim + pair/complete
  src/network.ts    — WiFi config via nmcli
  start.sh          — Boot entrypoint (checks device.json → setup or normal)

Event data model:

{
  _id: string;           // Frigate event ID
  cameraId: string;      // Camera name from Frigate config (e.g. "front_door")
  labels: string[];      // ["person", "car"] — from Frigate detection
  startTime: Date;
  thumbnailPath: string; // Relative path under /mnt/storage/
}

What the crew is building

The frontend UI was built with placeholder data. The crew connects it to the real edge API.

Mock inventory

AreaFileMockReal source
Server checkpage.tsxMOCK_HAS_SERVERS = trueGET /api/servers (already real)
Camera gridpage.tsx34 fake camerasGET {apiUrl}/api/frigate/api/cameras
Live event feedpage.tsx6 hardcoded eventsGET {apiUrl}/api/events?limit=10
Clips searchuseClipSearch.ts96 fake clipsGET {apiUrl}/api/events/clips (new)
Analyticswidgets.tsxHardcoded 65%/1.2kGET {apiUrl}/api/events/stats (new)

Deferred (issues created, not implemented)

AreaFileWhy deferred
LPR searchuseTagSearch.tsRequires Frigate ALPR configuration on device
Camera discoveryAddCameraModal.tsxRequires ONVIF/Frigate discovery API (jms-edge work)
Camera groupsCameraList.tsx, AddCameraModal.tsxNo groups model yet in either repo

New backend endpoints

GET /api/events/clips — paginated, filterable clips

ParamTypeDescription
camera[]stringFilter by cameraId
labels[]stringFilter by label
dateStart / dateEndISO dateDate range on startTime
timeStart / timeEndHH:MMTime-of-day filter
querystringSemantic search via Frigate
pagenumberDefault 1
limitnumberDefault 24

Response: { clips: ClipDto[], total, page, totalPages }

interface ClipDto {
  id: string;           // Frigate event ID
  cameraId: string;
  labels: string[];
  startTime: string;    // ISO 8601
  thumbnailPath: string;
}

GET /api/events/stats — event aggregation for analytics

ParamValue
period24h | 7d | 30d

Response:

interface StatsResponse {
  total: number;
  labelCounts: Record<string, number>;  // { person: 780, vehicle: 300 }
  byHour: Array<{ hour: number; count: number }>;
  byCamera: Array<{ cameraId: string; count: number }>;
}

The crew

AgentModelTask
Project Analystclaude-sonnetFull audit of both repos, creates 9 Gitea issues
Integration Architectclaude-sonnetDesigns all 4 API contracts before code starts
Backend Developerqwen3-coderImplements /api/events/clips and /api/events/stats
Backend QAqwen3-coderTypeScript compilation check on jms-edge
Search Developerqwen3-coderRewrites useClipSearch.ts to call the real clips API
Dashboard Developerqwen3-coderWires home page: real servers check, cameras list, event feed
Analytics Developerqwen3-coderWires EventAnalyticsWidget to real stats endpoint
Frontend QAqwen3-coderTypeScript check across all frontend changes
Integration Testerclaude-sonnetCross-validates all integrations, pushes branches, writes report

Running the crew

Make sure qwen3-coder is loaded:

docker compose --profile coder ps

Then run:

cd /srv/shared/projects/jetsons-living
source .venv/bin/activate
PYTHONPATH=src python -m jetsons_crew.main

Runtime: 30–60 minutes.

The crew writes to your Obsidian vault as it works:

  • projects/jetsons-living/gap-analysis.md — full audit of all mock data
  • projects/jetsons-living/api-contracts.md — all API contracts with TypeScript types
  • projects/jetsons-living/integration-report.md — final report + deployment steps

To use only cloud models (if local GPU is busy):

# Edit .env and set all *_MODEL values to claude-sonnet, then run normally

Deploying crew output

Once you've reviewed the PRs on Gitea:

Backend (jms-edge):

# On the JMS device (SSH in)
git pull origin feat/clips-and-stats-api
docker compose up -d --force-recreate --build jms-api

Frontend (jetsons-management-system):

  1. Merge feat/real-api-wiringprod-t on Gitea
  2. Push the same merge to GitHub — Amplify webhook triggers automatically
  3. Or trigger manually in the AWS Amplify console

Manual verification after deploy:

  1. Home page shows camera_sim instead of 34 fake cameras
  2. Activity feed shows real Frigate events (person/vehicle detections)
  3. Search page returns real clips with date/label filters working
  4. Analytics widget shows real detection counts for the last 24h