Backend Setup

Overview

The Omi backend powers all AI capabilities including transcription, conversation processing, chat, and integrations. This guide will help you set up a local development environment.

If you just want to build apps or test features, you can use Omi’s development backend instead. See the App Setup Guide for the automatic setup option.

Video Walkthrough

Prerequisites

Before starting, gather these API keys and credentials:

Required Services

Service	Purpose	Get Key
OpenAI	AI language models	Get Key
Modulate	Speech-to-text fallback	Provision `MODULATE_API_KEY`
Parakeet	Self-hosted speech-to-text	Configure `HOSTED_PARAKEET_API_URL`
Redis (Upstash)	Caching & sessions	Get Key
Pinecone	Vector database	Get Key
Hugging Face	Voice detection	Get Key

Optional Services

Service	Purpose	Get Key
Modal	Serverless deployment	Get Key
GitHub	Firmware updates	Get Key
Google Maps	Location features	Get Key
Typesense	Search functionality	Get Key
Stripe	Payment processing	Get Key

New to backend development? Install Homebrew (macOS/Linux) or Chocolatey (Windows) first - they make installing tools much easier.

1. Google Cloud & Firebase Setup

Install Google Cloud SDK

macOS
Windows
Linux/Nix

brew install google-cloud-sdk

choco install gcloudsdk

Enable Required APIs

Go to the Google Cloud Console and enable these APIs:

Cloud Resource Manager API

Required for project management

Firebase Management API

Required for Firebase integration

Authenticate with Google Cloud

Run these commands in your terminal, replacing <project-id> with your Google Cloud project ID:

gcloud auth login
gcloud config set project <project-id>
gcloud auth application-default login --project <project-id>

This installs Application Default Credentials (ADC) for the Google client libraries. Do not copy a service-account JSON key into the repo — long-lived keys are the #6800 anti-pattern. Prefer plain user ADC for local work.Impersonation is an advanced, time-limited option (ADC tokens from --impersonate-service-account expire in about an hour). Only use it when you already have an org-provisioned lower-privilege SA and hold roles/iam.serviceAccountTokenCreator on that account — replace the placeholder below with a real SA email from your team; there is no default dev-readonly@… account in this repo:

gcloud auth application-default login \
  --impersonate-service-account=<YOUR_DEV_SA>@<project-id>.iam.gserviceaccount.com

Firebase Admin and Firestore discover ADC automatically when SERVICE_ACCOUNT_JSON / GOOGLE_APPLICATION_CREDENTIALS are unset.

2. OAuth Authentication Setup

OAuth is required for user authentication. You need to configure both Google and Apple sign-in.

Google OAuth
Apple OAuth

Create OAuth 2.0 Client

Go to Google Cloud Console → Credentials
Click Create Credentials → OAuth 2.0 Client ID
Select Web application as the application type
Set a name (e.g., “Omi Backend Auth”)

Configure Authorized Origins

Under Authorized JavaScript origins, add:

https://your-domain.com (production)
https://your-ngrok-domain.ngrok-free.app (local development)

Configure Redirect URIs

Under Authorized redirect URIs, add:

https://your-domain.com/v1/auth/callback/google
https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/google

Save Credentials

Click Create and save your Client ID and Client Secret for the .env file.

Configure Consent Screen

Go to APIs & Services → OAuth consent screen:

Fill in required app information
Add your domain to Authorized domains
Add scopes: openid, email, profile

Apple OAuth requires a paid Apple Developer account ($99/year).

Create App ID

Go to Apple Developer Console → Identifiers
Create a new App ID with Sign In with Apple capability enabled
Note your Bundle ID

Create Services ID

Create a new Services ID (this becomes your APPLE_CLIENT_ID)
Configure Sign In with Apple for this Services ID
Add authorized domains and return URLs:
- https://your-domain.com/v1/auth/callback/apple
- https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/apple

Create Private Key

Go to Keys in Apple Developer Console
Create a new key with Sign In with Apple enabled
Download the .p8 file and note the Key ID
Note your Team ID from your Apple Developer account

Configure Firebase

Go to Firebase Console → Authentication → Sign-in method
Enable Apple and add your configuration:
- Client ID: Your Services ID
- Team ID: Your Apple Developer Team ID
- Key ID: From your private key
- Private Key: Contents of your .p8 file

Your Apple environment variables will be:

APPLE_CLIENT_ID: Your Services ID
APPLE_TEAM_ID: Your Apple Developer Team ID
APPLE_KEY_ID: The Key ID from step 3
APPLE_PRIVATE_KEY: The full contents of your .p8 file (including BEGIN/END lines)

3. Backend Installation

Clone the Repository

git clone https://github.com/BasedHardware/Omi.git
cd Omi/backend

Install System Dependencies

The backend Docker images run Python 3.11, so use Python 3.11 locally as well. If your package manager installs a newer default Python, install or select Python 3.11 before creating the virtual environment.

macOS
Windows
Linux/Nix

brew install python@3.11 git ffmpeg opus uv
uv --version

Install Python 3.11 from python.org, then install the remaining system tools and uv:

choco install git.install ffmpeg
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
python --version  # should print 3.11.x
uv --version

The Python opuslib package still needs a native libopus DLL on PATH. One common setup is MSYS2 UCRT64: install MSYS2, run pacman -S mingw-w64-ucrt-x86_64-opus, add C:\msys64\ucrt64\bin to PATH, then open a new shell and verify with where.exe opus.dll.

Python, Git, and FFmpeg are typically pre-installed. Install opus and uv via your package manager if needed, or use the standalone installer:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv --version

Set Up Python Virtual Environment

The backend uses uv, .python-version, and pylock.toml for reproducible local and CI installs.

macOS/Linux
Windows PowerShell

# From backend/
./scripts/sync-python-deps.sh
source .venv/bin/activate

# From backend/
bash ./scripts/sync-python-deps.sh
.venv\Scripts\Activate.ps1

You should see (.venv) at the beginning of your command prompt.

Update Python Dependencies

When intentionally changing backend dependencies, edit the relevant requirements*.txt input, then refresh the locks:

./scripts/update-python-lock.sh

The refresh preserves already-locked package versions by default. Use PYLOCK_UPGRADE=1 ./scripts/update-python-lock.sh only when intentionally upgrading dependency versions.

Create Environment File

cp .env.template .env

Edit .env and fill in your API keys (see Environment Variables below).For a self-hosted backend used by an existing Omi mobile build, set the Firebase token audience separately from your data project:

# Stock development/community Omi mobile builds mint tokens for this project.
FIREBASE_AUTH_PROJECT_ID=based-hardware-dev

# Your own Google Cloud/Firebase project for Firestore and other backend data.
FIREBASE_PROJECT_ID=your-self-hosted-project

Firebase ID tokens are verified with the issuer’s public signing keys, so FIREBASE_AUTH_PROJECT_ID does not require Omi service-account credentials. When it is unset, Firebase Admin uses its default project selection instead; set the auth project explicitly when it differs from your data-project setup, or stock mobile-build tokens will be rejected. Do not enable LOCAL_DEVELOPMENT on an internet-accessible deployment: that mode intentionally bypasses authentication for local harnesses.

4. Optional Services

Pusher Service (for webhooks)

Skip this if you don’t need webhook functionality.

cd pusher
cp .env.template .env

Prefer ADC from gcloud auth application-default login (leave SERVICE_ACCOUNT_JSON unset). Only use a scoped secret for Modal/CI compatibility paths that have not yet moved to Workload Identity (#6800).Start the service:

cd ..
uvicorn pusher.main:app --reload --env-file .env --port 8000

Optionally expose via Ngrok for external access.

Typesense (for search)

Skip this if you don’t need search functionality.

Create an account on Typesense
Create a collection named conversations using the schema in typesense/conversations.schema
Install the Firebase Typesense extension

Configuration for the extension:

Setting	Value
Firestore Collection Path	`users/{userId}/conversations`
Firestore Collection Fields	`structured,created_at,discarded,started_at,id,finished_at,geolocation,userId`

If you have existing data, create a typesense_sync collection and add a document named backfill with {'trigger': true}.

5. Running the Backend

Set Up Ngrok Tunnel

Ngrok exposes your local backend to the internet so the Omi app can connect.

Sign up at ngrok.com and install Ngrok
Authenticate with your account
Start the tunnel:

ngrok http --domain=your-domain.ngrok-free.app 8000

Note the public URL (e.g., https://your-domain.ngrok-free.app).

Start the Backend Server

uvicorn main:app --reload --env-file .env --port 8000

Flag	Purpose
`--reload`	Auto-restart on code changes
`--env-file .env`	Load environment variables
`--host 0.0.0.0`	Listen on all interfaces (optional)
`--port 8000`	Port to listen on

Connect the Omi App

In your Omi app’s .dev.env file, set:

API_BASE_URL=https://your-domain.ngrok-free.app/

Don’t forget the trailing / in the URL!

Make sure your OAuth redirect URIs in Google Cloud Console and Apple Developer Console include your Ngrok URL.

When You're Done

Deactivate the virtual environment:

deactivate

To resume later, just activate the virtual environment again.

Troubleshooting

SSL certificate errors during model downloads

Add this to utils/stt/vad.py:

import ssl
ssl._create_default_https_context = ssl._create_unverified_context

API key issues

Double-check all API keys in your .env file
Ensure there are no trailing spaces or quotes around values
Verify keys are active and not expired

Ngrok connection issues

Ensure your Ngrok tunnel is running
Verify the URL is correctly set in the Omi app
Check that OAuth redirect URIs match your Ngrok URL

Module not found errors

Try reinstalling dependencies:

./scripts/sync-python-deps.sh

Virtual environment not activating

On Windows, you may need to enable script execution: Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
Make sure you’re using Python 3.11

Environment Variables

Complete reference for all .env variables:

Required
OAuth
Firebase projects
Storage
Optional

Variable	Description
`OPENAI_API_KEY`	OpenAI API key for AI models
`MODULATE_API_KEY`	Modulate API key for transcription fallback
`HOSTED_PARAKEET_API_URL`	Parakeet endpoint for transcription
`HUGGINGFACE_TOKEN`	Hugging Face token for voice detection models
`REDIS_DB_HOST`	Redis host address
`REDIS_DB_PORT`	Redis port number
`REDIS_DB_PASSWORD`	Redis password (blank if none)
`PINECONE_API_KEY`	Pinecone API key for vector database
`PINECONE_INDEX_NAME`	Name of your Pinecone index
`GOOGLE_APPLICATION_CREDENTIALS`	Optional. Leave unset to use ADC from `gcloud auth application-default login`. Do not point this at a long-lived service-account JSON key (#6800).
`ADMIN_KEY`	Temporary key for local dev (e.g., `123`)
`ENCRYPTION_SECRET`	At least 32 bytes for encrypting user data

Variable	Description
`GOOGLE_CLIENT_ID`	Google OAuth 2.0 Client ID
`GOOGLE_CLIENT_SECRET`	Google OAuth 2.0 Client Secret
`APPLE_CLIENT_ID`	Apple Services ID
`APPLE_TEAM_ID`	Apple Developer Team ID
`APPLE_KEY_ID`	Apple private key ID
`APPLE_PRIVATE_KEY`	Apple .p8 file contents (with BEGIN/END lines)
`BASE_API_URL`	Your backend URL (e.g., Ngrok URL)

Variable	Description
`FIREBASE_AUTH_PROJECT_ID`	Firebase project whose ID-token audience the backend accepts; set this explicitly when it differs from the data project.
`FIREBASE_PROJECT_ID`	Google Cloud/Firebase project used for the backend’s Firestore and other data services.

Variable	Description
`BUCKET_SPEECH_PROFILES`	GCS bucket for speech profiles
`BUCKET_PLUGIN_LOGOS`	GCS bucket for app logos
`BUCKET_BACKUPS`	GCS bucket for backups

Variable	Description
`GITHUB_TOKEN`	GitHub token for firmware updates
`HOSTED_PUSHER_API_URL`	URL of your pusher service
`TYPESENSE_HOST`	Typesense server URL
`TYPESENSE_API_KEY`	Typesense API key
`NO_SOCKET_TIMEOUT`	Set `True` to disable socket timeout

.env.template intentionally leaves ENCRYPTION_SECRET blank. Generate a unique securely managed value of at least 32 bytes for every real environment.

Code Formatting

We use black for code formatting with a line length of 120 characters.

source .venv/bin/activate
uv pip install black

To auto-format on commit, install the pre-commit hook from the repository root:

ln -s -f ../../scripts/pre-commit .git/hooks/pre-commit

App Setup

Set up the Flutter app for development

Backend Deep Dive

Understand the backend architecture

Chat System

Learn how the chat feature works

Storing Conversations

How conversations and memories are stored

Need Help?

GitHub Issues

Report bugs or request features

Discord Community

Get help from the community

Get Started

Core Development

DIY Guide

Info

Backend Setup

Overview

Video Walkthrough

Prerequisites

Required Services

Optional Services

1. Google Cloud & Firebase Setup

Cloud Resource Manager API

Firebase Management API

2. OAuth Authentication Setup

3. Backend Installation

4. Optional Services

5. Running the Backend

Troubleshooting

Environment Variables

Code Formatting

App Setup

Backend Deep Dive

Chat System

Storing Conversations

Need Help?

GitHub Issues

Discord Community

​Overview

​Video Walkthrough

​Prerequisites

Required Services

Optional Services

​1. Google Cloud & Firebase Setup

Cloud Resource Manager API

Firebase Management API

​2. OAuth Authentication Setup

​3. Backend Installation

​4. Optional Services

​5. Running the Backend

​Troubleshooting

​Environment Variables

​Code Formatting

​Related Documentation

App Setup

Backend Deep Dive

Chat System

Storing Conversations

​Need Help?

GitHub Issues

Discord Community

Overview

Video Walkthrough

Prerequisites

1. Google Cloud & Firebase Setup

2. OAuth Authentication Setup

3. Backend Installation

4. Optional Services

5. Running the Backend

Troubleshooting

Environment Variables

Code Formatting

Related Documentation

Need Help?