Feature identity recognition 2100793612207388938#158
Conversation
This change introduces face recognition capabilities to track specific people across project photos. - Added TensorFlow Lite dependencies and a dummy MobileFaceNet model. - Created `FaceRecognitionHelper` to generate face embeddings. - Updated `ProjectEntity` and database schema to store target embeddings. - Enhanced `ProjectViewModel` to support "Track This Person" and smart re-alignment using cosine similarity. - Updated UI to expose the new tracking feature. - Optimized threading (Dispatchers.Default/IO) and memory management (Bitmap recycling) for heavy TFLite operations. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
This change introduces face recognition capabilities to track specific people across project photos. - Added TensorFlow Lite dependencies and a dummy MobileFaceNet model. - Created `FaceRecognitionHelper` to generate face embeddings. - Updated `ProjectEntity` and database schema to store target embeddings. - Enhanced `ProjectViewModel` to support "Track This Person" and smart re-alignment using cosine similarity. - Updated UI to expose the new tracking feature. - Optimized threading (Dispatchers.Default/IO) and memory management (Bitmap recycling, single load per photo) for heavy TFLite operations. - Refactored `FaceDetectorHelper` to support bitmap input for efficiency. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
This change addresses review feedback by fixing a syntax error, preventing memory leaks, optimizing embedding storage, and improving code readability. - Fixed syntax error in `ProjectViewModel.kt` (extra brace). - Implemented `close()` in `FaceRecognitionHelper` and ensured `GpuDelegate` is closed to prevent memory leaks. - Optimized embedding storage: Changed `targetEmbedding` from String to BLOB (ByteArray) in Entity and Database, using `ByteBuffer` for efficient FloatArray conversion. Updated `MIGRATION_4_5`. - Refactored `ProjectViewModel.kt`: Split `processFacesInternal` into `processFacesWithTarget` and `processFacesSpatial` for better readability and maintainability. - Updated `Project` domain model to use `FloatArray` for embeddings. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
This update addresses critical review feedback by fixing syntax errors, optimizing performance, and ensuring thread safety. - Fixed `ProjectViewModel` syntax (removed extra brace). - Refactored `FaceRecognitionHelper` to include a thread-safe `suspend fun close()` using `Mutex` for proper cleanup of TFLite resources. - Optimized database storage by switching `targetEmbedding` to `BLOB` (ByteArray) using `ByteBuffer`, replacing inefficient String storage. Updated migrations and mappers. - Parallelized face processing in `ProjectViewModel` using `coroutineScope` and `async/awaitAll` for improved performance on large datasets. - Refactored logic into `processFacesWithTarget` and `processFacesSpatial` for better readability. - Corrected status handling to ensure failed photo loads do not mark photos as processed. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
This update addresses critical review feedback by fixing syntax errors, optimizing performance, and ensuring thread safety. - Fixed `ProjectViewModel` syntax (removed extra brace). - Refactored `FaceRecognitionHelper` to include a thread-safe `suspend fun close()` using `Mutex` for proper cleanup of TFLite resources. - Optimized database storage by switching `targetEmbedding` to `BLOB` (ByteArray) using `ByteBuffer`, replacing inefficient String storage. Updated migrations and mappers. - Parallelized face processing in `ProjectViewModel` using `coroutineScope` and `async/awaitAll` with a `Semaphore(4)` to improve performance on large datasets while preventing OOM. - Refactored logic into `processFacesWithTarget` and `processFacesSpatial` for better readability. - Corrected status handling to ensure failed photo loads do not mark photos as processed. - Removed `@Singleton` from `FaceRecognitionHelper` and implemented cleanup in `ProjectViewModel.onCleared()`. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
This update addresses critical review feedback by fixing syntax errors, optimizing performance, and ensuring thread safety. - Fixed `ProjectViewModel` syntax (removed extra brace). - Refactored `FaceRecognitionHelper` to include a thread-safe `suspend fun close()` using `Mutex` for proper cleanup of TFLite resources. - Optimized database storage by switching `targetEmbedding` to `BLOB` (ByteArray) using `ByteBuffer`, replacing inefficient String storage. Updated migrations and mappers. - Parallelized face processing in `ProjectViewModel` using `coroutineScope` and `async/awaitAll` with a `Semaphore(4)` to improve performance on large datasets while preventing OOM. - Refactored logic into `processFacesWithTarget` and `processFacesSpatial` for better readability. - Corrected status handling to ensure failed photo loads do not mark photos as processed. - Removed `@Singleton` from `FaceRecognitionHelper` and implemented cleanup in `ProjectViewModel.onCleared()`. - Replaced `semaphore.withPermit` with `acquire`/`release` to support suspending calls. - Fixed `processFacesSpatial` loop compilation error. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
This update addresses critical review feedback by fixing syntax errors, optimizing performance, and ensuring thread safety. - Fixed `ProjectViewModel` syntax (removed extra brace). - Refactored `FaceRecognitionHelper` to include a thread-safe `suspend fun close()` using `Mutex` for proper cleanup of TFLite resources. - Optimized database storage by switching `targetEmbedding` to `BLOB` (ByteArray) using `ByteBuffer`, replacing inefficient String storage. Updated migrations and mappers. - Parallelized face processing in `ProjectViewModel` using `coroutineScope` and `async/awaitAll` with a `Semaphore(4)` to improve performance on large datasets while preventing OOM. - Refactored logic into `processFacesWithTarget` and `processFacesSpatial` for better readability. - Corrected status handling to ensure failed photo loads do not mark photos as processed. - Removed `@Singleton` from `FaceRecognitionHelper` and implemented cleanup in `ProjectViewModel.onCleared()`. - Replaced `semaphore.withPermit` with `acquire`/`release` to support suspending calls. - Fixed `processFacesSpatial` loop compilation error. - Fixed initialization race condition in `FaceRecognitionHelper`. - Used `runBlocking` in `onCleared` to ensure TFLite cleanup. - Ensured loading state is reset when project data is missing. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
…l.kt Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…l.kt Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…l.kt Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…reen.kt Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…er.kt Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This update fixes the specific build errors reported in the previous run: - Defined `NUM_CPU_THREADS` in `FaceRecognitionHelper.kt` and used it. - Defined `MAX_CONCURRENT_FACE_PROCESSING` in `ProjectViewModel.kt` and used it. - Added `action_re_align_smart` to `strings.xml` and updated `ProjectDetailScreen.kt` to use it. - Verified compilation with `./gradlew assembleDebug`. This builds upon previous optimizations for memory, concurrency, and storage. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
This update finalizes the Face Recognition feature implementation: - Fixed build errors by defining missing constants (NUM_CPU_THREADS, MAX_CONCURRENT_FACE_PROCESSING) and string resources (action_re_align_smart). - Cleaned up the repository by removing temporary build output files and the debug script count_braces.py. - Updated .gitignore to exclude build output files. - Refactored ProjectViewModel to clean up imports and usage of runBlocking. - Verified compilation with ./gradlew assembleDebug. This completes the optimization for memory, concurrency, and storage efficiency. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
This update addresses final review comments: - Removed temporary build output files and debug script from version control. - Updated `.gitignore` to prevent build logs from being tracked. - Refactored `ProjectViewModel.onCleared()` to use `runBlocking` for ensuring native resource cleanup without unnecessary full qualification. - Verified that all components are correctly integrated and the project builds successfully. This concludes the optimization and refactoring of the Face Recognition feature. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
This update completes the Face Recognition feature by addressing all feedback: - Removed temporary build output files and debug scripts from version control. - Updated `.gitignore` to exclude build artifacts. - Refactored `ProjectViewModel.onCleared` to use `runBlocking` for safe resource cleanup. - Verified build and integration. Co-authored-by: harrydbarnes <145344818+harrydbarnes@users.noreply.github.com>
Summary of ChangesHello @harrydbarnes, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the face alignment capabilities of the application by introducing intelligent face recognition. By integrating TensorFlow Lite, users can now select a specific face to track, and the system will use machine learning to identify and align that person across a series of photos. This provides a more robust and accurate alignment experience, moving beyond simple spatial tracking to identity-based recognition. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a significant new feature for face identity recognition using TensorFlow Lite. The changes include adding TFLite dependencies, a new FaceRecognitionHelper for inference, and updating the database and domain models to store face embeddings. The UI is enhanced to allow users to select a target face, and the ProjectViewModel is refactored to support both identity-based and spatial face tracking, with performance improvements like parallel processing. The implementation is solid, but I have identified a couple of important areas for improvement related to concurrency and coroutine best practices to enhance robustness and maintainability.
| @OptIn(kotlinx.coroutines.DelicateCoroutinesApi::class) | ||
| override fun onCleared() { | ||
| super.onCleared() | ||
| GlobalScope.launch(Dispatchers.IO) { | ||
| faceRecognitionHelper.close() | ||
| } | ||
| } |
There was a problem hiding this comment.
Using GlobalScope is highly discouraged as it creates top-level coroutines that are not tied to any component's lifecycle. In onCleared, this can lead to work continuing after the ViewModel is destroyed, potentially causing memory leaks or other issues.
The idiomatic way to handle resource cleanup in a ViewModel is to use addCloseable. You can make FaceRecognitionHelper implement java.io.Closeable and then call addCloseable(faceRecognitionHelper) in the ViewModel's init block. This ensures that faceRecognitionHelper.close() is called automatically when the ViewModel is cleared.
With this change, this onCleared override can be removed entirely. You would need to modify FaceRecognitionHelper to implement Closeable and change its close() method to be a non-suspending function, potentially using runBlocking internally for the mutex lock.
…er.kt Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…l.kt Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
harrydbarnes
deleted the
feature-identity-recognition-2100793612207388938
branch
1 participant
No description provided.