{"id":70538,"date":"2022-08-19T20:42:26","date_gmt":"2022-08-19T12:42:26","guid":{"rendered":"https:\/\/www.seeedstudio.com\/blog\/?p=70538"},"modified":"2022-08-22T14:55:34","modified_gmt":"2022-08-22T06:55:34","slug":"computer-vision-101-what-is-computer-vision-and-how-to-implement-cv-with-edge-devices","status":"publish","type":"post","link":"https:\/\/www.seeedstudio.com\/blog\/2022\/08\/19\/computer-vision-101-what-is-computer-vision-and-how-to-implement-cv-with-edge-devices\/","title":{"rendered":"Computer Vision 101: what is computer vision, and how to implement CV with edge devices?"},"content":{"rendered":"\n

Computer vision is what enables computers and systems to derive meaningful information from digital images, videos, and other visual inputs. You can think of AI as the brain of computers and computer vision being the eyes of computers. It would take action or make recommendations based on the information received, and it has been rapidly developing and even surpassing humans in solving visual tasks. It is currently vital in many industries, such as medical diagnosis, autonomous driving, video monitoring, etc. <\/p>\n\n\n\n

<\/div>\n\n\n\n

But what exactly is computer vision? <\/strong><\/p>\n\n\n\n

Let’s walk you through the following concepts in this article:<\/p>\n\n\n\n

  1. Definition of computer vision<\/li>
  2. How does computer vision work<\/li>
  3. Computer vision tasks, hello world of CV<\/li>
  4. Popular Computer Vision frameworks, libraries, and dev platform<\/li>
  5. Computer vision applications and tools<\/li>
  6. Implement CV at the edge<\/li><\/ol>\n\n\n\n

    Hopefully, by the end of this article, you will understand what computer vision is and even try your hand at some projects yourself!<\/p>\n\n\n\n

    <\/div>\n\n\n\n

    How does Computer Vision Work?<\/h2>\n\n\n\n

    Computer vision requires a lot of data to function effectively, it analyses data repeatedly until it is able to discern distinctions and recognize images. The algorithmic models will allow the machine to learn by itself rather than manually having someone program it to recognize an image. <\/p>\n\n\n\n

    The picture below is a simple illustration of the greyscale image buffer of Abraham Lincoln. Each pixel is represented by a single-8bit number, ranging from 0 (black) to 255 (white). On the right<\/strong> is what the software would read when u input an image.<\/strong><\/p>\n\n\n\n

    \"\"\/
    Source: DataRobot<\/a><\/figcaption><\/figure><\/div>\n\n\n\n

    MNIST handwritten digit classification: \u201cHello World\u201d of Computer Vision<\/h2>\n\n\n\n

    The MNIST data set comes from the National Institute of Standards and Technology in the United States and is a reduced version of NIST (National Institute of Standards and Technology). The training set consists of handwritten numbers from 250 different people, 50% of which are high school students. 50% were from the Census Bureau staff, and the test set was the same proportion of handwritten digit data.<\/p>\n\n\n\n

    The MNIST handwritten digits dataset <\/a>has a training set of 60,000 examples and a test set of 10,000 examples. It is a subset of the larger set provided by NIST. Figures have been size normalized and centered in a fixed-size image. The original creator of the MNIST kept a list of some of the methods tested on it. In their original paper, they used support-vector machines(SVM) to get an error rate of 0.8%.<\/p>\n\n\n\n

    \"\"\/
    Sample images from MNIST test dataset<\/a>, Wikipedia<\/figcaption><\/figure><\/div>\n\n\n\n
    <\/div>\n\n\n\n

    Computer Vision Tasks<\/h2>\n\n\n\n

    Image Classification<\/h3>\n\n\n\n

    Image classification is a basic computer vision task where once the program detects an image, it is able to accurately predict the given image and classify it accordingly to its certain class (cat, orange, a human face). <\/p>\n\n\n\n

    Image Localization<\/h3>\n\n\n\n

    Image localization is a basic computer vision task that finds the object and draws a bounding box around it. The following image shows the difference between classification and localization.<\/p>\n\n\n\n

    \"\"\/
    Source: Medium<\/a><\/figcaption><\/figure><\/div>\n\n\n\n

    Object Detection<\/h3>\n\n\n\n

    Object detection classifies and detects all objects in the image. However, it assigns a class to each individual object and draws a bounding box around it. It is different from classification and localization, as seen in the image below. <\/p>\n\n\n\n

    \"\"\/
    Source: Nirmala Murali<\/a><\/figcaption><\/figure><\/div>\n\n\n\n

    Semantic Segmentation<\/h3>\n\n\n\n

    Semantic segmentation is the next level to object detection. However, instead of drawing a bounding box around the object, it identifies the specific pixels in the image and segments them. Different classes would be assigned different colors, for example, grass = green and sheep = brown. It is commonly used for autonomous driving.<\/p>\n\n\n\n

    Instance Segmentation <\/h3>\n\n\n\n

    Instance segmentation is a level-up version of semantic segmentation. Instead of assigning the same pixel values to all the objects in the same class, it segments and shows different instances of the same class. If there is more than one of the same object detected in the image, it would be labeled accordingly, as seen below, Sheep 1, Sheep 2, and Sheep 3. It is commonly used for crowd count.<\/p>\n\n\n\n

    \"\"\/
    Source: Nirmala Murali<\/a><\/figcaption><\/figure><\/div>\n\n\n\n
    <\/div>\n\n\n\n

    Pose Estimation <\/h3>\n\n\n\n

    Pose estimation is a way of estimating the position and orientation of the joints of a human body. It predicts and tracks the movement of the object by finding the location of key points. Based on this information, it would be able to compare various movements and postures and draw insights. It is commonly used in AR\/VR gaming and sports.<\/p>\n\n\n\n

    \"\"
    Source: Using Pose Estimation on the Jetson Nano With alwaysAI<\/a>
    <\/figcaption><\/figure><\/div>\n\n\n\n
    <\/div>\n\n\n\n

    Popular Computer Vision Libraries and frameworks<\/h2>\n\n\n\n

    PyTorch<\/a><\/h3>\n\n\n\n
    \"\"<\/figure><\/div>\n\n\n\n

    PyTorch is an open-source ML library that uses dynamic computation, which allows for greater flexibility in building complex architectures. From the name, you can know that it supports Python very well. Although its underlying optimization is still in C, basically all its frameworks are written in Python, which makes you look at its source code more concise.  It supports both CPU and GPU computations.<\/p>\n\n\n\n

    The PyTorch NGC Container<\/a> is optimized for GPU acceleration and contains a validated set of libraries that enable and optimize GPU performance. This container also contains software for accelerating ETL (DALI<\/a>, RAPIDS<\/a>), Training (cuDNN<\/a>, NCCL<\/a>), and Inference (TensorRT<\/a>) workloads.<\/p>\n\n\n\n

    NVIDIA Jetson is one of the best platforms to work with PyTorch models mainly due to the inference support, allowing it to run most common computer vision models that can be transfer-learned with PyTorch. Coupled with TensorRT and PyTorch API technology, you are able to seamlessly run PyTorch models on NVIDIA Jetson and also Raspberry Pi. Learn more on how to do so from this blog post<\/a> by PyTorch. <\/p>\n\n\n\n

    If you just get started, don’t miss out Torchvision object detection fine-tuning tutorial<\/a>. The torchvision<\/a> package consists of popular datasets, model architectures, and common image transformations for computer vision.<\/p>\n\n\n\n

    torchvision<\/a>  includes the following packages:<\/p>\n\n\n\n