Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 118 additions & 0 deletions PINE-29-SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# PINE-29: E2E-003 - Index/Collection Management Tests

## Summary

Created comprehensive E2E tests for index/collection management operations across all three vector database providers (Pinecone, Qdrant, Weaviate).

## Files Added

### Test Suite
- **`e2e/index-collection-management.spec.ts`**: Main test file with 21 test cases

## Files Modified
- **`E2E_TESTING.md`**: Updated documentation to include new test suite

## Test Coverage

### Pinecone Index Management (8 active tests)
✅ **List indexes after connecting** - Verifies indexes can be listed via IPC with correct structure
✅ **Refresh indexes list** - Tests refresh functionality returns consistent data
✅ **View index stats** - Validates stats include namespaces, dimension, vector count
✅ **Create new index** - Tests index creation with provider-specific settings (dimension, metric, serverless spec)
✅ **Delete index** - Verifies deletion workflow and confirmation
✅ **Handle index stats for empty index** - Tests edge case of empty indexes
✅ **Handle errors for non-existent index** - Validates error handling
✅ **List indexes with correct properties** - Comprehensive validation of index structure

### Qdrant Collection Management (5 skipped tests)
🔲 List collections after connecting (TODO)
🔲 View collection stats (TODO)
🔲 Create new collection with Qdrant-specific settings (TODO)
🔲 Delete collection with confirmation (TODO)
🔲 Refresh collections list (TODO)

### Weaviate Class Management (5 skipped tests)
🔲 List classes after connecting (TODO)
🔲 View class stats (TODO)
🔲 Create new class with Weaviate-specific settings (TODO)
🔲 Delete class with confirmation (TODO)
🔲 Refresh classes list (TODO)

### Cross-Provider Tests (3 skipped tests)
🔲 Handle empty collections/indexes list (TODO)
🔲 Display provider-specific metadata correctly (TODO)
🔲 Handle very large collection lists efficiently (TODO)

## Implementation Details

### Test Infrastructure
- Uses existing `electron.setup.ts` helpers for app launch and profile management
- Follows same pattern as `connection-flow.spec.ts` (E2E-002)
- Tests use `window.electronAPI` for IPC communication
- Proper cleanup with `cleanupTestProfiles()` and `closeElectronApp()`

### Pinecone-Specific Testing
- Requires `PINECONE_API_KEY` environment variable for cloud testing
- Tests automatically skip if no real API key available
- Creates and deletes test indexes (e.g., `test-index-{timestamp}`)
- Validates serverless spec (cloud: aws, region: us-east-1)
- Tests multiple distance metrics (cosine, euclidean, dotproduct)

### TODO Items for Future Work
1. **Qdrant Integration**: Activate tests when adapter system is integrated into backend
2. **Weaviate Integration**: Activate tests when adapter system is integrated into backend
3. **UI Testing**: Add tests for UI components (IndexesPanel, IndexConfigView)
4. **Provider-Specific Metadata**: Test cloud/region display, quantization config, vectorizer settings
5. **Performance Testing**: Validate UI responsiveness with large collection lists (50+)

## Test Execution

### Run all E2E tests:
```bash
pnpm run test:e2e
```

### Run only index management tests:
```bash
pnpm exec playwright test index-collection-management
```

### Run with UI mode (interactive):
```bash
pnpm exec playwright test index-collection-management --ui
```

### Run with real Pinecone API:
```bash
PINECONE_API_KEY=your-api-key pnpm exec playwright test index-collection-management
```

## Notes

- **Pinecone Tests**: Fully functional, create/delete real indexes during testing
- **Qdrant/Weaviate Tests**: Marked with `test.skip()` and detailed TODO comments
- **Test Isolation**: Each test uses unique profile IDs and index names to avoid conflicts
- **Error Handling**: Tests validate both success and error paths
- **Documentation**: All tests include clear descriptions and comments

## Next Steps

1. Merge this branch to get E2E-003 tests into master
2. When adapter system is integrated:
- Remove `test.skip()` from Qdrant tests
- Implement Qdrant-specific test logic
- Remove `test.skip()` from Weaviate tests
- Implement Weaviate-specific test logic
3. Add UI-level tests for IndexesPanel and IndexConfigView components
4. Consider adding snapshot tests for index metadata display

## References

- **JIRA Ticket**: PINE-29
- **Related Tests**: E2E-002 (connection-flow.spec.ts)
- **Components Tested**: IndexesPanel.tsx, IndexConfigView.tsx
- **IPC Methods Used**:
- `pinecone.listIndexes()`
- `pinecone.getIndexStats()`
- `pinecone.createIndex()`
- `pinecone.deleteIndex()`
241 changes: 241 additions & 0 deletions PINE-30-SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
# PINE-30: E2E-004 Namespace Operations Tests - Summary

## Overview
Created comprehensive E2E tests for Pinecone namespace functionality following the established testing patterns from PINE-29.

## Test File Created
- `e2e/namespace-operations.spec.ts` - Complete E2E test suite for namespace operations

## Test Coverage

### 1. List Namespaces in Index
- **Test**: `should list namespaces in an index via stats`
- Tests retrieval of namespace list through `getIndexStats` IPC call
- Verifies namespace structure includes vector counts
- Validates default namespace handling (empty string key)

### 2. Display Namespace Stats
- **Test**: `should display namespace stats with vector counts`
- Verifies accurate vector count display per namespace
- Validates that total vector count equals sum of all namespaces
- Checks dimension consistency across the index

### 3. Select Namespace to View Vectors
- **Test**: `should select namespace to view vectors`
- Tests selecting a specific namespace
- Fetches and validates vectors from the selected namespace
- Verifies vector structure (id, values, metadata)

### 4. Clone/Duplicate Namespace
- **Test**: `should duplicate/clone namespace within same index`
- Creates a complete copy of a namespace within the same index
- Validates the target namespace is created with matching vector count
- Tests the `cloneNamespace` IPC handler

### 5. Duplicate Namespace Progress Tracking
- **Test**: `should track duplicate namespace progress`
- Sets up progress event listener via `onCloneNamespaceProgress`
- Collects all progress events during cloning operation
- Validates progress event structure (phase, totalVectors, processedVectors, message)
- Verifies phases: 'copying', 'complete', 'error', 'cancelled'

### 6. Cancel Namespace Duplication
- **Test**: `should cancel namespace duplication in progress`
- Tests cancellation during an active cloning operation
- Uses `cancelCloneNamespace` IPC handler
- Validates partial completion or error handling on cancellation

## Additional Edge Case Tests

### 7. Empty Namespace Handling
- **Test**: `should handle empty namespace listing`
- Creates a new empty index
- Verifies proper handling of indexes with no namespaces
- Tests that totalVectorCount is 0 for empty indexes
- Cleans up by deleting the test index

### 8. Empty Source Namespace
- **Test**: `should handle duplicate namespace with empty source`
- Attempts to clone a non-existent or empty namespace
- Validates error handling or 0-vector success response

### 9. Refresh Namespace Stats
- **Test**: `should refresh namespace stats after operations`
- Verifies stats refresh functionality
- Ensures consistency across multiple stat fetches
- Validates dimension remains constant

### 10. Accurate Vector Counts
- **Test**: `should show correct vector count per namespace`
- Cross-validates namespace stats with actual vector fetches
- Verifies reported counts match actual vector retrieval
- Tests consistency across all namespaces in an index

### 11. Test Namespace Creation
- **Test**: `should create a test namespace with vectors for duplication tests`
- Creates a new namespace with sample vectors
- Used as setup for subsequent duplication tests
- Validates namespace creation via vector upserts

## Test Infrastructure

### Setup & Teardown
- Uses `launchElectronApp()` and `closeElectronApp()` from electron.setup.ts
- Creates Pinecone test profile with real API key
- Connects to profile before running tests
- Cleans up test profiles after completion

### API Key Requirements
- All tests check for real Pinecone API key
- Tests are skipped if using dummy key or no key present
- Required environment variable: `PINECONE_API_KEY`

### Test Approach
- Tests use existing indexes when possible to avoid creation delays
- Some tests create temporary indexes/namespaces for isolation
- Proper cleanup of created resources (indexes, namespaces)
- Uses `page.evaluate()` to call IPC handlers via `window.electronAPI`

## Components Referenced

### NamespaceConfigView.tsx
- Main UI for creating namespaces
- Form and JSON modes for vector input
- Validates embedding text field configuration

### CloneNamespaceProgressDialog.tsx
- Progress dialog for index cloning (similar pattern)
- Shows progress bar, phase, and vector counts
- Cancel button for active operations
- Phase management: 'preparing', 'copying', 'complete', 'error', 'cancelled'

### DuplicateNamespaceProgressDialog.tsx
- Progress dialog specifically for namespace duplication
- Similar structure to CloneNamespaceProgressDialog
- Handles namespace-specific progress events
- Phases: 'copying', 'complete', 'error', 'cancelled'

### NamespacesPanel.tsx
- Lists namespaces from index stats
- Context menu for namespace actions (duplicate, delete)
- Handles namespace selection
- Progress tracking via `onCloneNamespaceProgress` event listener

## IPC Handlers Used

### Primary Handlers
- `pinecone:getIndexStats` - Fetch namespace list and stats
- `pinecone:getAllVectors` - Fetch vectors from a specific namespace
- `pinecone:cloneNamespace` - Duplicate namespace within same index
- `pinecone:cancelCloneNamespace` - Cancel active clone operation

### Event Listeners
- `pinecone:cloneNamespaceProgress` - Progress updates during cloning

### Type Definitions
- `CloneNamespaceParams` - { indexName, sourceNamespace, targetNamespace }
- `CloneNamespaceResult` - { success, copiedVectors, error? }
- `CloneProgress` - { phase, totalVectors, processedVectors, message }

## Testing Strategy

### Pattern Consistency
- Follows exact patterns from `e2e/index-collection-management.spec.ts`
- Uses same setup/teardown approach
- Implements similar error handling and skip logic
- Consistent timeout values for operations

### Pinecone-Specific
- All tests are Pinecone-specific (namespaces are a Pinecone concept)
- No Qdrant or Weaviate equivalents needed
- Tests only run with valid Pinecone API key

### Progress Tracking
- Tests capture progress events in browser context
- Verifies event structure and phase progression
- Validates that final event has 'complete' phase

### Resource Management
- Creates minimal test resources
- Reuses existing indexes when possible
- Cleans up created namespaces and indexes
- Proper wait times for Pinecone indexing delays

## Running the Tests

### Prerequisites
```bash
# Set Pinecone API key
export PINECONE_API_KEY="your-api-key"

# Build the app
pnpm run test:build
```

### Execute Tests
```bash
# Run all E2E tests
pnpm run test:e2e

# Run only namespace tests
pnpm exec playwright test e2e/namespace-operations.spec.ts

# Run with UI mode
pnpm run test:e2e:ui

# Run with debug mode
pnpm run test:e2e:debug
```

### View Results
```bash
pnpm exec playwright show-report
```

## Notes

### Timing Considerations
- Pinecone operations can take time (5-10 seconds for indexing)
- Tests include appropriate `waitForTimeout` calls
- Index creation can take up to 10 seconds
- Vector indexing typically takes 2-3 seconds

### Test Data
- Uses `Date.now()` for unique namespace/index names
- Random vector values for test data
- Metadata includes test flags for identification

### Error Handling
- All tests check for API key availability
- Graceful skipping when resources unavailable
- Try-catch blocks for operations that may fail
- Validates both success and error paths

## Integration Points

### Existing Test Suite
- Complements `e2e/index-collection-management.spec.ts` (PINE-29)
- Uses shared setup from `e2e/electron.setup.ts`
- Follows patterns from `e2e/connection-flow.spec.ts`

### Documentation
- Aligns with `E2E_TESTING.md` guidelines
- Uses established naming conventions
- Follows test organization structure

## Success Criteria Met
✅ List namespaces in an index
✅ Select namespace to view vectors
✅ Clone/duplicate namespace
✅ Duplicate namespace progress tracking
✅ Namespace stats display
✅ Test against Pinecone only (provider-specific)
✅ Edge case handling (empty namespaces, cancellation, refresh)
✅ Comprehensive progress event validation
✅ Resource cleanup and proper teardown

## Test Execution Status
- Tests are ready to run with valid Pinecone API key
- Will skip gracefully if no API key is provided
- All tests follow non-destructive patterns (create temporary resources)
- Proper cleanup ensures no leftover test data
Loading