Developing Integration Apps for OMI

Integration apps allow OMI to interact with external services and process data in real-time. This guide will walk you through creating both Memory Creation Triggers and Real-Time Transcript Processors.

Types of Integration Apps

1. 👷 Memory Creation Triggers

Video Walkthrough

Running a FastAPI app locally, not on AWS:

These apps are activated when OMI creates a new memory, allowing you to process or store the memory data externally.

Example Use Cases

• Update project management tools with conversation summaries
• Create a personalized social platform based on conversations and interests
• Generate a knowledge graph of interests, experiences, and relationships

2. 🏎️ Real-Time Transcript Processors

Video Walkthrough:

These apps process conversation transcripts as they occur, enabling real-time analysis and actions.

Example Use Cases

• Live conversation coaching and feedback
• Real-time web searches or fact-checking
• Emotional state analysis and supportive responses

Creating an Integration App

Step 1: Define Your App 🎯

Decide whether you're creating a Memory Creation Trigger or a Real-Time Transcript Processor, and outline its specific purpose.

Step 2: Set Up Your Endpoint 🔗

Create an endpoint (webhook) that can receive and process the data sent by OMI. You can create a test webhook. The data structure will differ based on your app type:

For Memory Creation Triggers:

Your endpoint will receive the entire memory object as a JSON payload, with a uid as a query parameter. Here is what to expect:

POST /your-endpoint?uid=user123

{
  "id": 0,
  "created_at": "2024-07-22T23:59:45.910559+00:00",
  "started_at": "2024-07-21T22:34:43.384323+00:00",
  "finished_at": "2024-07-21T22:35:43.384323+00:00",
  "transcript_segments": [
    {
      "text": "Segment text",
      "speaker": "SPEAKER_00",
      "speakerId": 0,
      "is_user": false,
      "start": 10.0,
      "end": 20.0
    }
    // More segments...
  ],
  "photos": [],
  "structured": {
    "title": "Conversation Title",
    "overview": "Brief overview...",
    "emoji": "🗣️",
    "category": "personal",
    "action_items": [
      {
        "description": "Action item description",
        "completed": false
      }
    ],
    "events": []
  },
  "apps_response": [
    {
      "app_id": "app-id",
      "content": "App response content"
    }
  ],
  "discarded": false
}

Your app should process this entire object and perform any necessary actions based on the full context of the memory.

Check the Notion CRM Python Example and it's respective JSON format here.

For Real-Time Transcript Processors:

Your endpoint will receive a JSON payload containing the most recently transcribed segments, with both session_id and uid as query parameters. Here's the structure:

POST /your-endpoint?session_id=abc123&uid=user123

[
  {
    "text": "Segment text",
    "speaker": "SPEAKER_00",
    "speakerId": 0,
    "is_user": false,
    "start": 10.0,
    "end": 20.0
  }
  // More recent segments...
]

Key points for Real-Time Transcript Processors:

1. Segments arrive in multiple calls as the conversation unfolds.
2. Use the session_id to maintain context across calls.
3. Implement smart logic to avoid redundant processing.
4. Consider building a complete conversation context by accumulating segments.
5. Clear processed segments to prevent re-triggering on future calls.

Remember to handle errors gracefully and consider performance, especially for lengthy conversations!

Check the Realtime News checker Python Example and it's respective JSON format here.

Step 3: Test Your App 🧪

Time to put your app through its paces! Follow these steps to test both types of integrations:

1. Open the OMI app on your device.
2. Go to Settings and enable Developer Mode.
3. Navigate to Developer Settings.

For Memory Creation Triggers:

4. Set your endpoint URL in the "Memory Creation Webhook" field. If you don't have an endpoint yet, create a test webhook
5. To test without creating a new memory:
   • Go to any memory detail view.
   • Click on the top right corner (3 dots menu).
   • In the Developer Tools section, trigger the endpoint call with existing memory data.

For Real-Time Transcript Processors:

4. Set your endpoint URL in the "Real-Time Transcript Webhook" field.
5. Start speaking to your device - your endpoint will receive real-time updates as you speak.

Your endpoints are now ready to spring into action!

For Memory Creation Triggers, you can test with existing memories or wait for new ones to be created.

For Real-Time Processors, simply start a conversation with OMI to see your app in action.

Happy app crafting! We can't wait to see what you create! 🎉

Step 4: Submit Your App

Submit your app using the Omi mobile app.

The webhook URL should be a POST request in which the memory object is sent as a JSON payload.

The setup completed URL is optional and should be a GET endpoint that returns {'is_setup_completed': boolean}.

The auth URL is optional as well and is utilized by users to setup your app. The uid query paramater will be appended to this URL upon usage.

The setup instructions can be either a link to instructions or text instructions for users on how to setup your app.

Setup Instructions Documentation

When writing your setup instructions, consider including:

1. A step-by-step setup guide
2. Screenshots (if applicable)
3. Authentication steps (if required)
4. Troubleshooting tips

Example structure:

Notes:

• Authentication is not needed for all apps. Include only if your app requires user-specific setup or credentials.
• For apps without authentication, users can simply enable the app without additional steps.
• All your README links, when the user opens them, we'll append a uid query parameter to it, which you can use to associate setup or credentials with specific users.