​
🔄 Overview of the Process

  1. Memory object is processed

  2. Object is converted to a dictionary

  3. Data is organized into specific fields

  4. Memory is saved to Firestore

    Backend Memory Storage

​
🧠 Detailed Steps

​
1. 📥 Memory Object Received

  • The process_memory function in utils/memories/process_memory.py processes a new or updated memory
  • The complete Memory object is then sent to the upsert_memory function in database/memories.py

​
2. 🔄 Convert to Dictionary

  • The upsert_memory function converts the Memory object into a Python dictionary
  • This conversion is necessary because Firestore stores data in a JSON-like format

​
3. 📊 Data Fields

The dictionary contains the following key fields:

FieldDescription
idUnique ID of the memory
created_atTimestamp of memory creation
started_atTimestamp when the associated event started
finished_atTimestamp when the associated event ended
sourceSource of the memory (e.g., “omi”, “openglass”, “workflow”)
languageLanguage code of the conversation
structuredDictionary of structured information (see below)
transcript_segmentsList of transcript segments (see below)
geolocationLocation data (if available)
plugins_resultsResults from any plugins run on the memory
external_dataAdditional data from external integrations
postprocessingInformation about post-processing status
discardedBoolean indicating if the memory is low-quality
deletedBoolean indicating if the memory has been deleted
visibilityVisibility setting of the memory

​
📋 Structured Information

The structured field contains:

  • title: Topic of the memory
  • overview: Summary of the memory
  • emoji: Representing emoji
  • category: Category (e.g., “personal”, “business”)
  • action_items: List of derived action items
  • events: List of extracted calendar events

​
🗣️ Transcript Segments

Each segment in transcript_segments includes:

  • speaker: Speaker label (e.g., “SPEAKER_00”)
  • start: Start time in seconds
  • end: End time in seconds
  • text: Transcribed text
  • is_user: Boolean indicating if spoken by the user
  • person_id: ID of a person from user’s profiles (if applicable)

​
🔄 Postprocessing Information

The postprocessing field contains:

  • status: Current status (e.g., “not_started”, “in_progress”)
  • model: Post-processing model used (e.g., “fal_whisperx”)
  • fail_reason: (Optional) Reason for failure

​
4. 💾 Save to Firestore

  • database/memories.py uses the Firebase Firestore API to store the memory data dictionary
  • Data is saved in the memories collection within the user’s document

​
📁 Firestore Structure

Users Collection

User Document
└── memories Collection
├── Memory Document 1
├── Memory Document 2
└── ...

This structure allows for efficient querying and management of user-specific memory data.