Running a computer vision model on a cell phone or mobile device is a powerful tool that can enable real-time analysis of images and videos, which can be useful in a variety of applications. While there are challenges to streaming computer vision models on small devices, CrowdAI has developed a roadmap of techniques and tools to overcome these challenges. By leveraging cloud driven API connections for invoking inference from a trained model, CrowdAI sees a pathway to real-time analysis of imagery and video on small devices operating at the edge. Additionally, the geospatial benefits of building models from media captured on cell phones can offer unique advantages for training, monitoring, and analyzing objects of interest.

In today's world, where we are surrounded by computers and cameras of all types and sizes, it's essential for machine learning services to be deployment-agnostic and camera-agnostic. Being able to work in any cloud, hardware, or software environment; and to use any camera or sensor is an invaluable advantage that has become increasingly important in recent years as the use of cameras has exploded in various industries. These features allow for greater flexibility and ease of use—exactly what CrowdAI strives to provide—enabling ML to be used in a wider range of applications.

Going from a complex segmentation model to a simpler bounding box object detection model using SAM may seem like a bit of overkill, but there are some instances where an object detection model is favored over a segmentation model. For example, if we have a photo of a street with a bunch of pedestrians, a detection model can provide insight into how many people are there, their location in the frame, and how they interact with each other; segmentation masks wouldn’t give us as useful information since they would just be silhouettes of standing or walking people. Another benefit is that object detection models are designed to be more robust to variations in object size, rotation, and aspect ratio, making them ideal for identifying objects with diverse geometries. Lastly, when computational resources are limited, object detection models tend to be less computationally intensive than segmentation models, which can require more processing power and memory to run efficiently. 

To test out quickly moving from an object detection model to segmentation model using the SAM integration in the CrowdAI platform, we revisited a detection model we had built in the past. CrowdAI worked on automating visual inspection for a real estate developer. Their goal was to have tenants take photos of damage in their apartments and have our models automatically identify what the potential problem could be and on what appliance. The examples below are from a quick study CrowdAI completed to identify if a shower head was leaking – for the purpose of the study we used images of shower heads running and not running to train a model to identify water ‘leaking’ from the shower head. 

Segmentation is a core computer vision architecture - training a model based on the exact pixels which make up an object of interest. This computer vision architecture typically requires a time consuming annotation process because a large number of carefully labeled media are needed to train a model, with the labeling work often needing to be done by subject matter experts. The caveat being, most subject matter experts don’t have the time to be meticulously labeling, nor do they have hundreds of examples of each object, defect or anomaly they are aiming to detect. This creates a huge roadblock for producing quick, off the shelf or domain specific customized segmentation models. 

In our no-code platform, CrowdAI has added what we call Persistent Analytic PipelinesTM. This feature enables users to automate the end-to-end GEOINT analysis workflow, known as Processing, Exploitation, and Dissemination.