Skip to main content

Developing Integration Apps for OMI

Integration apps allow OMI to interact with external services and process data in real-time. This guide will walk you through creating both Memory Creation Triggers and Real-Time Transcript Processors.

Types of Integration Apps​

1. 👷 Memory Creation Triggers​

These apps are activated when OMI creates a new memory, allowing you to process or store the memory data externally.

Memory trigger app

Example Use Cases​

  • Update project management tools with conversation summaries
  • Create a personalized social platform based on conversations and interests
  • Generate a knowledge graph of interests, experiences, and relationships

2. 🏎️ Real-Time Transcript Processors (not stable)​

These apps process conversation transcripts as they occur, enabling real-time analysis and actions.

Memory trigger app

Example Use Cases​

  • Live conversation coaching and feedback
  • Real-time web searches or fact-checking
  • Emotional state analysis and supportive responses

Creating an Integration App​

Step 1: Define Your App 🎯​

Decide whether you're creating a Memory Creation Trigger or a Real-Time Transcript Processor, and outline its specific purpose.

Step 2: Set Up Your Endpoint 🔗​

Create an endpoint (webhook) that can receive and process the data sent by OMI. You can create a test webhook. The data structure will differ based on your app type:

For Memory Creation Triggers:​

Your endpoint will receive the entire memory object as a JSON payload, with a uid as a query parameter. Here is what to expect:

POST /your-endpoint?uid=user123


{
  "id": 0,
  "created_at": "2024-07-22T23:59:45.910559+00:00",
  "started_at": "2024-07-21T22:34:43.384323+00:00",
  "finished_at": "2024-07-21T22:35:43.384323+00:00",
  "transcript": "Full transcript text...",
  "transcript_segments": [
    {
      "text": "Segment text",
      "speaker": "SPEAKER_00",
      "speakerId": 0,
      "is_user": false,
      "start": 10.0,
      "end": 20.0
    }
    // More segments...
  ],
  "photos": [],
  "structured": {
    "title": "Conversation Title",
    "overview": "Brief overview...",
    "emoji": "🗣️",
    "category": "personal",
    "action_items": [
      {
        "description": "Action item description",
        "completed": false
      }
    ],
    "events": []
  },
  "apps_response": [
    {
      "app_id": "app-id",
      "content": "App response content"
    }
  ],
  "discarded": false
}

Your app should process this entire object and perform any necessary actions based on the full context of the memory.

Check the Notion CRM Python Example and it's respective JSON format here.

For Real-Time Transcript Processors:

Your endpoint will receive a JSON payload containing the most recently transcribed segments, with both session_id and uid as query parameters. Here's the structure:

POST /your-endpoint?session_id=abc123&uid=user123

[
  {
    "text": "Segment text",
    "speaker": "SPEAKER_00",
    "speakerId": 0,
    "is_user": false,
    "start": 10.0,
    "end": 20.0
  }
  // More recent segments...
]

Key points for Real-Time Transcript Processors:

  1. Segments arrive in multiple calls as the conversation unfolds.
  2. Use the session_id to maintain context across calls.
  3. Implement smart logic to avoid redundant processing.
  4. Consider building a complete conversation context by accumulating segments.
  5. Clear processed segments to prevent re-triggering on future calls.

Remember to handle errors gracefully and consider performance, especially for lengthy conversations!

Check the Realtime News checker Python Example and it's respective JSON format here.

Step 3: Test Your App 🧪​

Time to put your app through its paces! Follow these steps to test both types of integrations:

  1. Open the OMI app on your device.
  2. Go to Settings and enable Developer Mode.
  3. Navigate to Developer Settings.

For Memory Creation Triggers:​

  1. Set your endpoint URL in the "Memory Creation Webhook" field. If you don't have an endpoint yet, create a test webhook
  2. To test without creating a new memory:
    • Go to any memory detail view.
    • Click on the top right corner (3 dots menu).
    • In the Developer Tools section, trigger the endpoint call with existing memory data.

Memory trigger app

For Real-Time Transcript Processors:​

  1. Set your endpoint URL in the "Real-Time Transcript Webhook" field.
  2. Start speaking to your device - your endpoint will receive real-time updates as you speak.

Memory trigger app

Your endpoints are now ready to spring into action!

For Memory Creation Triggers, you can test with existing memories or wait for new ones to be created.

For Real-Time Processors, simply start a conversation with OMI to see your app in action.

Happy app crafting! We can't wait to see what you create! 🎉

Step 4: Prepare Your App for Submission​

Create a JSON object defining your app:

{
  "id": "your-app-id",
  "name": "Your App Name",
  "author": "Your Name",
  "description": "Brief description of your app",
  "image": "/apps/logos/your-app-logo.jpg",
  "capabilities": [
    "external_integration"
  ],
  "external_integration": {
    // "memory_creation" | "transcript_processed"
    "triggers_on": "memory_creation",
    // a POST request with the memory object will be sent here as a JSON payload
    "webhook_url": "https://your-endpoint-url.com",
    // GET endpoint, that returns {'is_setup_completed': boolean} (Optional) if your app doesn't require setup, set to null.
    "setup_completed_url": "https://your-setup-completion-url.com",
    // Include a Readme with more details about your app in the PR
    "setup_instructions_file_path": "/apps/instructions/your-app/README.md"
  },
  "deleted": false
}

Integration Instructions Documentation​

Create a markdown file at setup_instructions_file_path with:

  1. Step-by-step setup guide
  2. Screenshots (if applicable)
  3. Authentication steps (if required)
  4. Troubleshooting tips

Example structure:

# Setting Up [Your App Name]

1. [First step]
   ![Step 1](assets/step_1.png)

2. [Second step]
   ![Step 2](assets/step_2.png)

## Authentication (if required)

If your app requires user-specific authentication (e.g., connecting to a user's Notion table):

1. In your authentication flow, use the `uid` query parameter we append this as query param to all your links in your
   README, so you can map user credentials.
2. [Authentication steps]
3. After successful authentication, return `{"is_setup_completed": true}` from your `setup_completed_url`.

If your app doesn't require user authentication (e.g., it's a purely LLM-based service), you can skip this section.

## Troubleshooting

[Common issues and solutions]

---

> Experimental feature. Feedback: [your email]

Notes:

  • Authentication is not needed for all apps. Include only if your app requires user-specific setup or credentials.
  • For apps without authentication, users can simply enable the app without additional steps.
  • All your README links, when the user opens them, we'll append a uid query parameter to it, which you can use to associate setup or credentials with specific users.