This is a TensorFlow.js web application where users can classify images selected locally or taken with their device's camera. The app uses a neural network, which can be changed according to your needs to solve your problem.
If you're interested in making this to benefit your business, please contact me on LinkedIn.
- A pre-trained Keras/TensorFlow model is converted to the TensorFlow.js web friendly format and integrated with app.
- User launches progressive web application.
- App assets and TensorFlow.js model files are downloaded from the web.
- Assets and model are stored locally using browser cache and IndexedDB storage.
- User takes photo with device camera or selects local image.
- Image is sent through the model for inference and top predictions are given.
- React: A JavaScript library for building user interfaces.
- TensorFlow.js: A JavaScript library for training and deploying ML models in the browser and on Node.js.
Data remains on-device and classification is performed locally
No image is ever uploaded to the server because with TensorFlow.js, inference is done locally, and
user data is kept private. There is no need for a persistent network connection to continue performing inferences.
Assets are stored in browser cache and storage
On the user's first visit, a service worker is used to cache page resources (i.e. HTML, CSS, and JS files).
Each device must have network connectivity for this first visit, but on subsequent visits, the app
will still load and work as assets will be served from the cache. Similarly on the first visit,
the pre-trained model is downloaded and saved in IndexedDB,
a browser API for client-side storage. Subsequent loads to the page will retrieve the model from IndexedDB if
it is available. This saves from having to continually re-download the model.
App can run on desktop and be 'installed' on mobile
Regardless of what platform the user is on, as long as the app is run on a modern browser, everything
should work. With the use of our manifest file,
the app can be 'installed' on mobile devices, making it look like a native app with its own app icon
on the home screen.
Content can still be updated by prompting the user
Since content is served cache/storage first, we need a way to serve new content to the end-user.
For this, when new content is available a new service worker is ready to be installed, the user is
notified with a prompt to reload the page and get the latest changes. For updating the
pre-trained model, we use a server API endpoint to query the date the model on the server was last
updated. If the app can hit the endpoint and detects the locally saved model is older than the model on
the server, the user is given a prompt with the option to update.
The app allows you to either use your device's camera to snap an image or select a local image from the device's filesystem. Select an image of an object or put the object in frame using your camera, then click classify. Local inference will then be performed, and the top five results will be given.
- Original repo
- TensorFlow.js
- React
- Progressive Web Apps
- Service Workers
- Web App Manifest
- IndexedDB
- React Bootstrap
This code pattern is licensed under the Apache Software License, Version 2. Separate third party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 (DCO) and the Apache Software License, Version 2.