diff options
| -rw-r--r-- | docs/dev/roadmap.md | 388 | ||||
| -rw-r--r-- | docs/manual/features.md | 17 |
2 files changed, 46 insertions, 359 deletions
diff --git a/docs/dev/roadmap.md b/docs/dev/roadmap.md index 1cd3e99..ea9a70d 100644 --- a/docs/dev/roadmap.md +++ b/docs/dev/roadmap.md @@ -1,370 +1,48 @@ -# Note Feature Implementation Roadmap +# Kioku Development Roadmap -This document outlines the implementation plan for adding Anki-compatible "Note" concept to Kioku. +## Issue #3: Introduce CRDT Library for Conflict Resolution -## Overview +Replace the current Last-Write-Wins (LWW) conflict resolution with Automerge CRDT for better offline sync. -Currently, Kioku uses a simple Card model with `front` and `back` text fields. To improve Anki interoperability, we will introduce: +**Decisions:** +- Library: Automerge +- Text conflicts: LWW Register (simple, predictable) +- Migration: Clean migration (no backward compatibility) -1. **Note** - A container for field values (similar to Anki's Note) -2. **NoteType** - Defines the structure of notes (fields and card templates) -3. **Card** - Generated from Notes, holds FSRS scheduling state +### Phase 1: Add Automerge and Core Types -### Key Concepts +- [ ] Install dependencies: `@automerge/automerge`, `@automerge/automerge-repo`, `@automerge/automerge-repo-storage-indexeddb` +- [ ] Create `src/client/sync/crdt/types.ts` - Automerge document type definitions +- [ ] Create `src/client/sync/crdt/document-manager.ts` - Automerge document lifecycle management +- [ ] Create `src/client/sync/crdt/index.ts` - Module exports -- **One Note → Multiple Cards**: A "Basic (and reversed)" note type creates 2 cards (front→back, back→front) -- **One Note → One Card**: A "Basic" note type creates 1 card -- **Shared Content**: When note content is edited, all generated cards reflect the change -- **Independent Scheduling**: Each card has its own FSRS state (due date, stability, etc.) +### Phase 2: Create CRDT Repository Layer -## Data Model Design +- [ ] Create `src/client/sync/crdt/repositories.ts` - CRDT-aware repository wrappers +- [ ] Create `src/client/sync/crdt/sync-state.ts` - Sync state serialization -### New Entities +### Phase 3: Modify Sync Protocol -``` -NoteType -├── id: UUID -├── user_id: UUID (FK → users) -├── name: string -├── front_template: string (mustache template, e.g., "{{Front}}") -├── back_template: string (mustache template, e.g., "{{Back}}") -├── is_reversible: boolean (if true, creates reversed card too) -├── created_at: timestamp -├── updated_at: timestamp -├── deleted_at: timestamp (soft delete) -└── sync_version: number +- [ ] Modify `src/client/sync/push.ts` - Add crdtChanges to push payload +- [ ] Modify `src/client/sync/pull.ts` - Handle crdtChanges in pull response +- [ ] Modify `src/client/sync/conflict.ts` - Replace LWW with Automerge merge +- [ ] Modify `src/client/sync/manager.ts` - Integrate CRDT sync flow -NoteFieldType -├── id: UUID -├── note_type_id: UUID (FK → note_types) -├── name: string (e.g., "Front", "Back") -├── order: number (display order) -├── field_type: enum ("text") // Fixed to "text" for now -├── created_at: timestamp -├── updated_at: timestamp -├── deleted_at: timestamp -└── sync_version: number +### Phase 4: Server-Side CRDT Support -Note -├── id: UUID -├── deck_id: UUID (FK → decks) -├── note_type_id: UUID (FK → note_types) -├── created_at: timestamp -├── updated_at: timestamp -├── deleted_at: timestamp -└── sync_version: number +- [ ] Install server dependency: `@automerge/automerge` +- [ ] Create `src/server/db/schema-crdt.ts` - CRDT document storage schema +- [ ] Create database migration for crdt_documents table +- [ ] Modify `src/server/routes/sync.ts` - Handle CRDT changes in API +- [ ] Modify `src/server/repositories/sync.ts` - Store/merge CRDT documents -NoteFieldValue -├── id: UUID -├── note_id: UUID (FK → notes) -├── note_field_type_id: UUID (FK → note_field_types) -├── value: text -├── created_at: timestamp -├── updated_at: timestamp -└── sync_version: number +### Phase 5: Migration -Card (modified) -├── id: UUID -├── deck_id: UUID (FK → decks) -├── note_id: UUID (FK → notes) [REQUIRED] -├── is_reversed: boolean [REQUIRED] (false=normal, true=reversed) -├── front: text [CACHED - rendered from template for performance] -├── back: text [CACHED - rendered from template for performance] -├── (FSRS fields unchanged) -├── created_at: timestamp -├── updated_at: timestamp -├── deleted_at: timestamp -└── sync_version: number -``` +- [ ] Create `src/client/sync/crdt/migration.ts` - One-time migration script +- [ ] Create server migration script to convert existing data -### Card Display Logic +### Phase 6: Testing and Cleanup -Template rendering uses a custom mustache-like renderer (no external dependencies). - -Syntax: `{{FieldName}}` is replaced with the field value. - -When displaying a card: -- **Normal card** (`is_reversed = false`): Render `front_template` on front, `back_template` on back -- **Reversed card** (`is_reversed = true`): Render `back_template` on front, `front_template` on back - -Example templates: -- Simple: `{{Front}}` -- With text: `Q: {{Front}}` -- Multiple fields: `{{Word}} - {{Reading}}` - -### Built-in Note Types - -Create these as default note types for each user: - -1. **Basic** - - Fields: Front, Back - - front_template: `{{Front}}`, back_template: `{{Back}}` - - is_reversible: false - -2. **Basic (and reversed card)** - - Fields: Front, Back - - front_template: `{{Front}}`, back_template: `{{Back}}` - - is_reversible: true - -### Behavior Rules - -**Updating `is_reversible`:** -- Changing `is_reversible` does NOT affect existing cards -- Only affects new note creation (whether to create 1 or 2 cards) -- Existing reversed cards remain even if `is_reversible` is set to `false` - -**Deletion Constraints (enforced by FK constraints + application logic):** -- **NoteType**: Cannot delete if any Notes reference it -- **NoteFieldType**: Cannot delete if any NoteFieldValues reference it -- **Note**: Deleting a Note cascades soft-delete to all its Cards -- **Card**: Deleting a Card also deletes its Note (and all sibling Cards via cascade) - -**Required Tests for Deletion:** -- [x] Attempt to delete NoteType with existing Notes → should fail -- [x] Attempt to delete NoteFieldType with existing NoteFieldValues → should fail -- [x] Delete Note → verify all related Cards are soft-deleted -- [x] Delete Card → verify Note and all sibling Cards are soft-deleted - -## Implementation Phases - -### Phase 1: Database Schema - -**Tasks:** -- [x] Add `note_types` table schema (Drizzle) -- [x] Add `note_field_types` table schema -- [x] Add `notes` table schema -- [x] Add `note_field_values` table schema -- [x] Modify `cards` table: add `note_id`, `is_reversed` columns (nullable initially) -- [x] Create migration file -- [x] Add Zod validation schemas - -**Files to modify:** -- `src/server/db/schema.ts` -- `src/server/schemas/index.ts` -- `drizzle/` (new migration) - -### Phase 2: Server Repositories - -**Tasks:** -- [x] Create `NoteTypeRepository` - - CRUD operations - - Include fields when fetching -- [x] Create `NoteTypeFieldRepository` - - CRUD operations - - Reorder fields -- [x] Create `NoteRepository` - - Create note with field values (auto-generate cards based on `is_reversible`) - - Update note (updates field values) - - Delete note (cascade soft-delete to cards) -- [x] Modify `CardRepository` - - Fetch card with note data for display - - Support note-based card creation - -**Files to create/modify:** -- `src/server/repositories/noteType.ts` (new) -- `src/server/repositories/note.ts` (new) -- `src/server/repositories/card.ts` (modify) -- `src/server/repositories/types.ts` (modify) - -### Phase 3: Server API Routes - -**Tasks:** -- [x] Add NoteType routes - - `GET /api/note-types` - List user's note types - - `POST /api/note-types` - Create note type - - `GET /api/note-types/:id` - Get note type with fields - - `PUT /api/note-types/:id` - Update note type (name, front_template, back_template, is_reversible) - - `DELETE /api/note-types/:id` - Soft delete -- [x] Add NoteFieldType routes (nested under note-types) - - `POST /api/note-types/:id/fields` - Add field - - `PUT /api/note-types/:id/fields/:fieldId` - Update field - - `DELETE /api/note-types/:id/fields/:fieldId` - Remove field - - `PUT /api/note-types/:id/fields/reorder` - Reorder fields -- [x] Add Note routes - - `GET /api/decks/:deckId/notes` - List notes in deck - - `POST /api/decks/:deckId/notes` - Create note (auto-generates cards) - - `GET /api/decks/:deckId/notes/:noteId` - Get note with field values - - `PUT /api/decks/:deckId/notes/:noteId` - Update note field values - - `DELETE /api/decks/:deckId/notes/:noteId` - Delete note and its cards -- [x] Modify Card routes - - Update GET to include note data when available -- [x] Modify Study routes - - Fetch note/field data for card display - -**Files to create/modify:** -- `src/server/routes/noteTypes.ts` (new) -- `src/server/routes/notes.ts` (new) -- `src/server/routes/cards.ts` (modify) -- `src/server/routes/study.ts` (modify) -- `src/server/index.ts` (register new routes) - -### Phase 4: Client Database (Dexie) - -**Tasks:** -- [x] Add `LocalNoteType` interface and table -- [x] Add `LocalNoteFieldType` interface and table -- [x] Add `LocalNote` interface and table -- [x] Add `LocalNoteFieldValue` interface and table -- [x] Modify `LocalCard` interface: add `noteId`, `isReversed` -- [x] Update Dexie schema version and upgrade handler -- [x] Create client repositories for new entities - -**Files to modify:** -- `src/client/db/index.ts` -- `src/client/db/repositories.ts` - -### Phase 5: Sync Logic - -**Tasks:** -- [x] Add sync for NoteType, NoteFieldType -- [x] Add sync for Note, NoteFieldValue -- [x] Update Card sync to include `noteId`, `isReversed` -- [x] Define sync order (NoteTypes → Notes → Cards) -- [x] Update pull/push sync handlers - -**Files to modify:** -- `src/server/routes/sync.ts` -- `src/server/repositories/sync.ts` -- `src/client/sync/push.ts` -- `src/client/sync/pull.ts` - -### Phase 6: Frontend - Note Type Management - -**Tasks:** -- [x] Create NoteType list page (`/note-types`) -- [x] Create NoteType editor component - - Edit name - - Manage fields (add/remove/reorder) - - Edit front/back templates (mustache syntax) - - Toggle `is_reversible` option -- [x] Add navigation to note type management - -**Files to create:** -- `src/client/pages/NoteTypesPage.tsx` -- `src/client/components/NoteTypeEditor.tsx` -- `src/client/components/FieldEditor.tsx` - -### Phase 7: Frontend - Note CRUD - -**Tasks:** -- [x] Update CreateCardModal → CreateNoteModal - - Select note type - - Dynamic field inputs based on note type - - Preview generated cards -- [x] Update EditCardModal → EditNoteModal - - Load note and field values - - Update all generated cards on save -- [x] Update DeckDetailPage - - Group cards by note - - Show note-level actions (edit note, delete note) - - Display whether card is normal or reversed - -**Files to modify:** -- `src/client/components/CreateCardModal.tsx` → `CreateNoteModal.tsx` -- `src/client/components/EditCardModal.tsx` → `EditNoteModal.tsx` -- `src/client/pages/DeckDetailPage.tsx` - -### Phase 8: Frontend - Study Page - -**Tasks:** -- [x] Create custom template renderer utility -- [x] Update StudyPage to render cards based on note data - - Fetch note field values - - Use `isReversed` to determine which template to use for front/back - - Render templates with custom renderer - - Maintain front/back flip behavior -- [x] Handle both legacy cards (direct front/back) and new note-based cards - -**Files to modify/create:** -- `src/client/utils/templateRenderer.ts` (new) -- `src/client/pages/StudyPage.tsx` - -### Phase 9: Cleanup & Documentation - -**Tasks:** -- [x] Make `note_id` and `is_reversed` NOT NULL (schema migration) -- [x] Update architecture.md with Note-based data model -- [ ] E2E tests for study flow with note-based cards -- [ ] Performance testing with multiple cards per note - -**Note:** Data migration is not needed since production has no legacy data. - -## Design Notes - -### Card front/back Fields - -Cards retain `front` and `back` text fields as **cached rendered content**. When a card is created from a note: -1. Templates are rendered with field values -2. Rendered content is stored in `front`/`back` for quick display -3. When note content is updated, all related cards are re-rendered - -This approach avoids rendering templates on every card display while maintaining the Note as the source of truth. - -## API Examples - -### Create Note - -```http -POST /api/decks/:deckId/notes -Content-Type: application/json - -{ - "noteTypeId": "uuid-of-basic-reversed", - "fields": { - "field-uuid-front": "What is the capital of Japan?", - "field-uuid-back": "Tokyo" - } -} -``` - -Response: -```json -{ - "note": { - "id": "note-uuid", - "noteTypeId": "...", - "deckId": "...", - "fieldValues": [...] - }, - "cards": [ - { "id": "card-1-uuid", "isReversed": false }, - { "id": "card-2-uuid", "isReversed": true } - ] -} -``` - -### Get Cards for Study - -```http -GET /api/decks/:deckId/study -``` - -Response: -```json -{ - "cards": [ - { - "id": "card-uuid", - "noteId": "note-uuid", - "isReversed": false, - "fieldValues": { - "Front": "What is the capital of Japan?", - "Back": "Tokyo" - }, - "state": 0, - "due": "2024-01-15T00:00:00Z", - ... - } - ] -} -``` - -Note: Templates (`front_template`, `back_template`) are fetched separately via NoteType API and cached on the client. The client renders the card display using the template and field values. - -## Testing Checklist - -- [x] Unit tests for all new repositories -- [x] Integration tests for note CRUD -- [x] Test card generation from different note types -- [x] Test sync with note data -- [ ] E2E tests for study flow with note-based cards +- [ ] Add unit tests for CRDT operations +- [ ] Add integration tests for concurrent edit scenarios +- [ ] Remove legacy LWW code after validation diff --git a/docs/manual/features.md b/docs/manual/features.md index 81c81ca..cc5db35 100644 --- a/docs/manual/features.md +++ b/docs/manual/features.md @@ -8,11 +8,20 @@ A list of features available in Kioku. - View all decks at a glance - Confirmation dialog before deletion -## Card Management +## Note Types -- Create, edit, and delete cards within decks -- Browse all cards in a deck -- Front/back text fields for flashcards +- Define custom note structures with configurable fields +- Built-in note types: "Basic" and "Basic (and reversed card)" +- Mustache-style templates for card rendering (e.g., `{{Front}}`, `{{Back}}`) +- Reversible option: automatically generate both normal and reversed cards + +## Note & Card Management + +- Create notes with dynamic fields based on note type +- One note can generate multiple cards (e.g., front→back and back→front) +- Edit note content and all generated cards update automatically +- Browse cards grouped by note in deck view +- Independent scheduling: each card maintains its own FSRS state ## Study Session |