​
Prerequisites 📋

Before you start, make sure you have the following: Note: If you are not very experienced in backend development, we recommend installing Homebrew (for macOS or Linux) or Chocolatey (for Windows).

​
Video Walkthrough

​
I. Setting Up Google Cloud & Firebase ☁️

  1. Install Google Cloud SDK:
    • Mac (using brew): brew install google-cloud-sdk
    • Nix Envdir: The SDK is usually pre-installed
    • Windows (using choco): choco install gcloudsdk
  2. Enable Necessary APIs: 🔧
  3. Authenticate with Google Cloud: 🔐
    • Open your terminal
    • Run the following commands one by one, replacing <project-id> with your Google Cloud project ID: 
      gcloud auth login
gcloud config set project <project-id>
gcloud auth application-default login --project <project-id>
    • This process generates an application_default_credentials.json file in the ~/.config/gcloud directory. This file is used for automatic authentication with Google Cloud services in Python.
    • Copy the credentials file to your backend directory and rename it: 
      # For macOS/Linux:
cp ~/.config/gcloud/application_default_credentials.json ./google-credentials.json

# For Windows:
copy %APPDATA%\gcloud\application_default_credentials.json google-credentials.json

​
II. Setting Up OAuth Authentication 🔐

​
Google OAuth Setup

  1. Create OAuth 2.0 Client Credentials:
    • Go to the Google Cloud Console
    • Navigate to APIs & Services -> Credentials
    • Click Create Credentials -> OAuth 2.0 Client ID
    • Configure the OAuth consent screen if prompted
    • Select Web application as the application type
    • Set the Name (e.g., “Omi Backend Auth”)
    • Under Authorized JavaScript origins, add:
      • https://your-domain.com (replace with your actual domain)
      • https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/google (for local development)
    • Under Authorized redirect URIs, add:
      • https://your-domain.com/v1/auth/callback/google (replace with your actual domain)
      • https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/google (for local development)
    • Click Create
    • Save the Client ID and Client Secret - you’ll need these for your environment variables
  2. Configure OAuth Consent Screen:
    • Go to APIs & Services -> OAuth consent screen
    • Fill in the required information about your app
    • Add your domain to Authorized domains
    • Add the necessary scopes: openid, email, profile

​
Apple OAuth Setup

  1. Prerequisites:
    • You need a paid Apple Developer account
    • Your app must be registered in the Apple Developer Console
  2. Create App ID:
    • Go to Apple Developer Console
    • Create a new App ID with Sign In with Apple capability enabled
    • Note your App ID (Bundle ID)
  3. Create Services ID:
    • In Apple Developer Console, create a new Services ID
    • This will be your APPLE_CLIENT_ID
    • Configure Sign In with Apple for this Services ID
    • Add your authorized domains and return URLs:
      • https://your-domain.com/v1/auth/callback/apple
      • https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/apple
  4. Create Private Key:
    • Go to Keys in Apple Developer Console
    • Create a new key with Sign In with Apple enabled
    • Download the .p8 file and note the Key ID
    • You’ll need the Team ID from your Apple Developer account
  5. Prepare Environment Variables:
    • APPLE_CLIENT_ID: Your Services ID
    • APPLE_TEAM_ID: Your Apple Developer Team ID
    • APPLE_KEY_ID: The Key ID from step 4
    • APPLE_PRIVATE_KEY: The contents of your .p8 file as a string (keep the -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY----- lines)
  6. Configure Firebase for Apple Sign-In:
    • Go to the Firebase Console
    • Select your project
    • Navigate to Authentication -> Sign-in method
    • Click on Apple and enable it
    • Add the same configuration values from your Apple Developer setup:
      • Client ID: Your Services ID (same as APPLE_CLIENT_ID)
      • Team ID: Your Apple Developer Team ID (same as APPLE_TEAM_ID)
      • Key ID: The Key ID from your private key (same as APPLE_KEY_ID)
      • Private Key: The contents of your .p8 file (same as APPLE_PRIVATE_KEY)
    • Click Save

​
III. Backend Setup 🛠️

  1. Install Python & Dependencies: 🐍
    • Mac (using brew): brew install python
    • Nix Envdir: Python is pre-installed
    • Windows (using choco): choco install python
    • Install pip (if not present):
    • Install Git and FFmpeg:
      • Mac (using brew): brew install git ffmpeg
      • Nix Envdir: Git and FFmpeg are pre-installed
      • Windows (using choco): choco install git.install ffmpeg
    • Install opus:
      • Mac (using brew): brew install opus
      • Windows: You should already have it installed if you are on Windows 10 version 1903 and above
    • Install PyOgg:
      • All Platforms: pip install PyOgg
    • Install All Required Dependencies:
      • All Platforms: pip install -r requirements.txt
  2. Clone the Backend Repository: 📂
    • Open your terminal and navigate to your desired directory
    • Clone the Omi backend repository: 
      git clone https://github.com/BasedHardware/Omi.git
cd Omi
cd backend
  3. Set up Pusher Service: 📡 [Optional]
    • You don’t need to have the Pusher Service running if you do not intend to use the webhooks feature
    • Navigate to the pusher directory: 
      cd pusher
    • Create a copy of the .env.template file and rename it to .env: 
      cp .env.template .env
    • Set the SERVICE_ACCOUNT_JSON environment variable in the .env file to the string representation of your Google Cloud service account credentials (google-credentials.json). This is used to authenticate with Google Cloud
    • Move back to the backend directory and run the following command to start the Pusher service: 
      uvicorn pusher.main:app --reload --env-file .env --port 8000
    • Optionally you can expose the Pusher endpoint using Ngrok or a similar service
  4. Set up Typesense: 🔎 [Optional]
    • You don’t need to setup Typesense if you do not intend to use the search functionality
    • Create an account on Typesense
    • Create a new collection in Typesense with the name conversations and use the schema provided in the typesense/conversations.schema file
    • Install the Firebase Typesense extension from here
      • While setting up the extension, use the following values for the configuration:
        • Firestore Collection Path: users/{userId}/conversations
        • Firestore Collection Fields: structured,created_at,discarded,started_at,id,finished_at,geolocation,userId
      • Create typesense_sync collection and add a document named backfill with data {'trigger' : true} (required only if you already have memories in Firestore and want to sync them to Typesense)
    • Set the TYPESENSE_HOST, TYPESENSE_HOST_PORT and TYPESENSE_API_KEY environment variables in the .env file to the host URL and API key provided by Typesense
  5. Set up the Environment File: 📝
    • Create a copy of the .env.template file and rename it to .env: 
      cp .env.template .env
    • Open the .env file and fill in the following:
      • OPENAI_API_KEY: Obtained from your OpenAI API Settings
      • DEEPGRAM_API_KEY: Obtained from your Deepgram Console
      • Redis Credentials: Host, port, username, and password from your Upstash Redis Console
      • Modal API Key: Obtained from your Modal Dashboard
      • ADMIN_KEY: Set to a temporary value (e.g., 123) for local development
      • ENCRYPTION_SECRET: The .env.template provides a default key suitable for local development (e.g., omi_ZwB2ZNqB2HHpMK6wStk7sTpavJiPTFg7gXUHnc4tFABPU6pZ2c2DKgehtfgi4RZv). For production, you must generate a new secure key.
      • OAuth Authentication Credentials: From Section II above
        • GOOGLE_CLIENT_ID: Your Google OAuth 2.0 Client ID
        • GOOGLE_CLIENT_SECRET: Your Google OAuth 2.0 Client Secret
        • APPLE_CLIENT_ID: Your Apple Services ID
        • APPLE_TEAM_ID: Your Apple Developer Team ID
        • APPLE_KEY_ID: Your Apple private key ID
        • APPLE_PRIVATE_KEY: Your Apple private key content (as a string with newlines)
      • HOSTED_PUSHER_API_URL: Endpoint of your hosted pusher service (if you are using it, see step 3)
      • Typesense Credentials: Host, port, and API key from your Typesense Cloud Dashboard
      • NO_SOCKET_TIMEOUT: (Optional) Set to True to disable the socket timeout for the backend server (socket will stay connected for as long as the app is open)
      • Other API Keys: Fill in any other API keys required by your integrations (e.g., Google Maps API key)
  6. Install Python Dependencies: 📚 You have two options for installing the required Python packages: Option A: Using a Virtual Environment (Recommended) 🐍
    • It’s recommended to use a virtual environment to isolate your project dependencies and avoid conflicts
    • Create a new virtual environment in the backend directory: 
      # Create a new virtual environment
python -m venv venv

# Activate the virtual environment
# On Windows:
venv\Scripts\activate
# On macOS/Linux:
source venv/bin/activate
    • You should see (venv) at the beginning of your command prompt, indicating that the virtual environment is active
    • Install dependencies within the virtual environment: 
      pip install -r requirements.txt
    • All packages will be installed isolated from your system’s Python installation
    Option B: Direct Installation
    • If you prefer not to use a virtual environment, you can install the dependencies directly: 
      pip install -r requirements.txt
    • Note that this approach may lead to conflicts with other Python projects on your system
  7. Code Formatting: 💅
    • To maintain code quality, we use black for formatting with a line length of 120 characters.
    • You must have black installed: 
      pip install black
    • To automatically format your code on commit, install the pre-commit hook from the repository root: 
      # From the root of the repository
ln -s -f ../../scripts/pre-commit .git/hooks/pre-commit

​
III. Running the Backend Locally 🏃‍♂️

  1. Set up Ngrok for Tunneling: 🚇
    • Sign up for a free account on https://ngrok.com/ and install Ngrok
    • Follow their instructions to authenticate Ngrok with your account
    • During the onboarding, Ngrok will provide you with a command to create a tunnel to your localhost. Modify the port in the command to 8000 (the default port for the backend). For example: 
      ngrok http --domain=example.ngrok-free.app 8000
    • Run this command in your terminal. Ngrok will provide you with a public URL (like https://example.ngrok-free.app) that points to your local backend
  2. Start the Backend Server: 🖥️
    • In your terminal, run: 
      uvicorn main:app --reload --env-file .env
      • --reload automatically restarts the server when code changes are saved, making development easier
      • --env-file .env loads environment variables from your .env file
      • --host 0.0.0.0 listens to every interface on your computer so you don’t have to set up ngrok when developing in your network
      • --port 8000 port for backend to listen
  3. Troubleshooting SSL Errors: 🔒
    • SSL Errors: If you encounter SSL certificate errors during model downloads, add this to utils/stt/vad.py: 
      import ssl
ssl._create_default_https_context = ssl._create_unverified_context
    • API Key Issues: Double-check all API keys in your .env file. Ensure there are no trailing spaces
    • Ngrok Connection: Ensure your Ngrok tunnel is active and the URL is correctly set in the Omi app
    • Dependencies: If you encounter any module not found errors, try reinstalling dependencies: 
      pip install -r requirements.txt --upgrade --force-reinstall
  4. Connect the App to the Backend: 🔗
    • In your Omi app’s environment variables, set the BASE_API_URL to the public URL provided by Ngrok (e.g., https://example.ngrok-free.app/) (don’t forget the trailing / )
    • Make sure your OAuth redirect URIs in Google Cloud Console and Apple Developer Console are updated to use your Ngrok URL for local development
Now, your Omi app should be successfully connected to the locally running backend.
  1. When You’re Done: 🛑
    • If you used a virtual environment, when you’re finished working with the backend, deactivate it by running: 
      deactivate
    • This command returns you to your system’s global Python environment
    • To reactivate the virtual environment later, just run the activation command again (source venv/bin/activate on macOS/Linux or venv\Scripts\activate on Windows)

​
Environment Variables 🔐

Here’s a detailed explanation of each environment variable you need to define in your .env file:
  • HUGGINGFACE_TOKEN: Your Hugging Face Hub API token, used to download models for speech processing (like voice activity detection)
  • BUCKET_SPEECH_PROFILES: The name of the Google Cloud Storage bucket where user speech profiles are stored
  • BUCKET_PLUGIN_LOGOS: The name of the Google Cloud Storage bucket where to put logos for uploaded apps
  • BUCKET_BACKUPS: The name of the Google Cloud Storage bucket used for backups (if applicable)
  • GOOGLE_APPLICATION_CREDENTIALS: The path to your Google Cloud service account credentials file (google-credentials.json). This is generated in step 3 of I. Setting Up Google Cloud & Firebase
    • By default, the backend expects to find a file named google-credentials.json in the same directory where the application is running
    • If you’ve followed Option 1 in step 3 of the Google Cloud setup, this will be already set correctly
    • If you prefer to use the default location of the credentials, set this to the full path of your application_default_credentials.json file (e.g., ~/.config/gcloud/application_default_credentials.json on macOS/Linux or %APPDATA%\gcloud\application_default_credentials.json on Windows)
  • PINECONE_API_KEY: Your Pinecone API key, used for vector database operations. Storing Memory Embeddings: Each memory is converted into a numerical representation (embedding). Pinecone efficiently stores these embeddings and allows Omi to quickly find the most relevant memories related to a user’s query
  • PINECONE_INDEX_NAME: The name of your Pinecone index where memory embeddings are stored
  • REDIS_DB_HOST: The host address of your Redis instance
  • REDIS_DB_PORT: The port number of your Redis instance
  • REDIS_DB_PASSWORD: The password for your Redis instance (blank by default)
  • DEEPGRAM_API_KEY: Your Deepgram API key, used for real-time and pre-recorded audio transcription
  • ADMIN_KEY: A temporary key used for authentication during local development (replace with a more secure method in production)
  • OPENAI_API_KEY: Your OpenAI API key, used for accessing OpenAI’s language models for chat, memory processing, and more
  • ENCRYPTION_SECRET: A secret key used to encrypt sensitive user data, such as conversations, memories, and chat messages. The key must be at least 32 bytes long. The .env.template file includes a default key (e.g., omi_ZwB2ZNqB2HHpMK6wStk7sTpavJiPTFg7gXUHnc4tFABPU6pZ2c2DKgehtfgi4RZv) which is sufficient for local development. For a production environment, it is crucial to replace this with your own securely generated random string.
  • GITHUB_TOKEN: Your GitHub personal access token, used to access GitHub’s API for retrieving the latest firmware version
  • WORKFLOW_API_KEY: Your custom API key for securing communication with external workflows or integrations
  • HOSTED_PUSHER_API_URL: URL of your omi-pusher service if you’re using one. Example: http://omi-push:8081
  • TYPESENSE_HOST: URL of your typesense server.
  • TYPESENSE_API_KEY: Typesense API key
  • GOOGLE_CLIENT_ID: Your Google OAuth 2.0 Client ID from Google Cloud Console, used for Google authentication
  • GOOGLE_CLIENT_SECRET: Your Google OAuth 2.0 Client Secret from Google Cloud Console, used for Google authentication
  • APPLE_CLIENT_ID: Your Apple Services ID from Apple Developer Console, used for Apple Sign In authentication
  • APPLE_TEAM_ID: Your Apple Developer Team ID, found in your Apple Developer account
  • APPLE_KEY_ID: The Key ID of your Apple private key for Sign In with Apple
  • APPLE_PRIVATE_KEY: The contents of your Apple private key (.p8 file) as a string, including the -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY----- lines
  • BASE_API_URL: The base URL of your backend API (e.g., https://your-domain.com or https://your-ngrok-domain.ngrok-free.app for local development)
Make sure to replace the placeholders (<api-key>, <bucket-name>, etc.) with your actual values.

​
Contributing 🤝

We welcome contributions from the open source community! Whether it’s improving documentation, adding new features, or reporting bugs, your input is valuable. Check out our Contribution Guide for more information.

​
Support 🆘

If you’re stuck, have questions, or just want to chat about Omi:
  • GitHub Issues: 🐛 For bug reports and feature requests
  • Community Forum: 💬 Join our community forum for discussions and questions
  • Documentation: 📚 Check out our full documentation for in-depth guides
Happy coding! 💻 If you have any questions or need further assistance, don’t hesitate to reach out to our community.