WhatsThat?

WhatsThat? is an Android app that identifies what's in a picture. Snap a photo or pick one from your gallery, and the app sends it to the Google Cloud Vision API, which returns a ranked list of labels describing the image's contents.

It started life as a college project built on top of Google's official Cloud Vision Android sample and was extended with a Lottie splash animation and an animated floating-action-button (FAB) menu for choosing the image source.

Features

• 📷 Capture or pick an image — take a new photo with the camera or choose an existing one from the gallery
• 🔍 Label detection — uploads the image to the Cloud Vision API's LABEL_DETECTION feature and returns up to 10 labels with confidence scores
• ✨ Animated UI — a Lottie-powered splash animation and a rotating, expandable FAB menu (open/close, clockwise/anticlockwise)
• 🔐 Runtime permission handling — requests camera and storage permissions on demand (Android 6.0+ runtime permission model)
• 🖼️ In-app preview — displays the selected/captured image alongside the detected labels in a CardView

Architecture

The app is a single-Activity Android application. MainActivity owns the UI and orchestrates image selection, permission checks, and the network call; two small utility classes handle permissions and app-signature lookups required by the restricted Cloud Vision API key.

flowchart TB
    subgraph UI["UI Layer (activity_main.xml / content_main.xml)"]
        Lottie["Lottie splash\n(security_scan.json)"]
        Start["START button"]
        FAB["Expandable FAB menu\n(camera / gallery)"]
        Card["CardView\n(image preview + labels)"]
    end

    subgraph Activity["MainActivity"]
        UploadImage["uploadImage()"]
        ScaleBitmap["scaleBitmapDown()"]
        CallVision["callCloudVision()\n(AsyncTask)"]
        ConvertResp["convertResponseToString()"]
    end

    subgraph Utils["Utility classes"]
        PermUtils["PermissionUtils\n(runtime permission requests)"]
        PkgUtils["PackageManagerUtils\n(SHA-1 signature for API key)"]
    end

    subgraph Android["Android system"]
        Camera["Camera app\n(MediaStore.ACTION_IMAGE_CAPTURE)"]
        Gallery["Gallery / Document picker\n(ACTION_GET_CONTENT)"]
        FileProvider["FileProvider\n(provider_paths.xml)"]
    end

    subgraph Cloud["Google Cloud"]
        VisionAPI["Cloud Vision API\n(images.annotate)"]
    end

    Start --> FAB
    FAB -->|"Camera"| PermUtils
    FAB -->|"Gallery"| PermUtils
    PermUtils --> Camera
    PermUtils --> Gallery
    Camera --> FileProvider
    FileProvider --> UploadImage
    Gallery --> UploadImage
    UploadImage --> ScaleBitmap
    ScaleBitmap --> CallVision
    CallVision --> PkgUtils
    CallVision -->|"HTTPS: base64 JPEG +\nX-Android-Cert / X-Android-Package"| VisionAPI
    VisionAPI -->|"BatchAnnotateImagesResponse"| ConvertResp
    ConvertResp --> Card

Loading

User & data flow

The sequence below traces a single "identify this photo" round trip, from tapping a FAB button to seeing labels on screen.

sequenceDiagram
    actor User
    participant UI as MainActivity (UI)
    participant Perm as PermissionUtils
    participant OS as Android OS
    participant Task as AsyncTask
    participant Vision as Cloud Vision API

    User->>UI: Tap START, then FAB (Camera or Gallery)
    UI->>Perm: requestPermission(CAMERA / READ_EXTERNAL_STORAGE)
    alt Permission not yet granted
        Perm->>OS: requestPermissions()
        OS-->>UI: onRequestPermissionsResult()
        UI->>UI: retry startCamera() / startGalleryChooser()
    else Already granted
        Perm-->>UI: true
    end
    UI->>OS: startActivityForResult(Camera or Gallery intent)
    OS-->>UI: onActivityResult(image Uri)
    UI->>UI: uploadImage(uri) -> scaleBitmapDown(1200px)
    UI->>Task: callCloudVision(bitmap)
    Task->>Task: encode JPEG as Base64,\nbuild BatchAnnotateImagesRequest
    Task->>Vision: POST images:annotate\n(LABEL_DETECTION, maxResults=10)
    Vision-->>Task: BatchAnnotateImagesResponse
    Task->>UI: onPostExecute(labels + scores)
    UI-->>User: Render image + "I found these things: ..."

Loading

Tech stack

Layer	Technology
Language	Java
Platform	Android (minSdkVersion 22, targetSdkVersion 25, compileSdkVersion 26)
Build	Gradle 3.3, Android Gradle Plugin 2.3.3
UI toolkit	AppCompat, Design Support Library, CardView
Animation	Lottie (com.airbnb.android:lottie:2.2.5)
Image intelligence	Google Cloud Vision API via google-api-services-vision
Networking	google-api-client-android, google-http-client-gson

Project structure

WhatsThat/
├── app/
│   ├── build.gradle                # app module config + dependencies
│   ├── proguard-rules.pro
│   └── src/main/
│       ├── AndroidManifest.xml     # permissions, activity, FileProvider
│       ├── assets/                 # Lottie animation JSON files
│       │   ├── security_scan.json  # splash animation (used)
│       │   ├── video_cam.json
│       │   └── wave.json
│       ├── java/com/google/sample/cloudvision/
│       │   ├── MainActivity.java          # UI, image capture/pick, Vision API call
│       │   ├── PermissionUtils.java       # runtime permission helpers
│       │   └── PackageManagerUtils.java   # app signature for the Vision API key
│       └── res/
│           ├── layout/              # activity_main.xml, content_main.xml
│           ├── anim/                # FAB open/close/rotate animations
│           ├── drawable-*/          # FAB icons
│           ├── mipmap-*/            # launcher icons
│           ├── values/              # strings, colors, dimens, styles
│           └── xml/provider_paths.xml
├── build.gradle                     # root/project-level Gradle config
├── settings.gradle
└── gradlew / gradlew.bat            # Gradle wrapper

Permissions used

Permission	Purpose
INTERNET	Send images to the Cloud Vision API
READ_EXTERNAL_STORAGE	Read images picked from the gallery / stored camera output
CAMERA	Capture a new photo

Getting started

Prerequisites

• Android Studio (a version compatible with Gradle 3.3 / AGP 2.3.3 — e.g. Android Studio 2.3–3.0 — is recommended given the project's current toolchain)
• A Google Cloud project with the Cloud Vision API enabled and an API key

Build & run

1. Clone the repository and open it in Android Studio.
2. Let Gradle sync and download dependencies.
3. Supply a Cloud Vision API key (see below) and connect a device/emulator running API 22+.
4. Run the app module.

Configuring your own API key

MainActivity currently references a Cloud Vision API key as a constant (CLOUD_VISION_API_KEY). Because this repository is public, treat any key committed to it as compromised — generate a fresh, restricted API key in the Google Cloud Console and swap it in before building your own copy. For anything beyond local experimentation, prefer keeping the key out of source control entirely (e.g. inject it from local.properties/gradle.properties via BuildConfig at build time, or via an environment variable) rather than hardcoding it.

License

The original code is derived from Google's Cloud Vision Android sample, licensed under the Apache License 2.0.