![\"\"](\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2024\/04\/WhatsApp-Image-2024-04-09-at-09.23.00_7f8c5c81-1030x860.jpg\")
<\/figure><\/div>\n\n\nBy integrating AI’s processing capabilities with IoT’s vast network of connected devices, video analytics enables real-time monitoring and analysis for applications ranging from urban planning and traffic management to retail analytics, healthcare, and industrial safety. Video analytics in AIoT lies in its ability to leverage the synergy between AI’s analytical power and IoT’s data-gathering prowess, heading forward to the next step of intelligent monitoring and decision-making solutions.<\/p>\n\n\n\n
Combining Generative AI: from Detection to Interpretation<\/h2>\n\n\n\n
Well nowadays, we are not satisfied with only detecting actions that can be triggered. There are more and more ideas and applications coming up using Large Language models (LLM)<\/a>, which improve video analytics by making systems more accurate and user-friendly.<\/p>\n\n\n\n Generative AI introduces a new type of method that helps us understand the world in ways we couldn’t before. These models are great at working with natural language inputs and can help us get a better grasp of content, including images and videos. They’re especially good at learning quickly with zero-shot or few-shot data input training, which changes how we interact with digital content, making it more accessible and comprehensible.<\/p>\n\n\n\n LLM plays a pivotal role in this change. It’s a powerful AI model in the context of natural language processing and machine learning. Trained extensively on diverse textual data, it excels in comprehending and generating human-like language. You can easily facilitate these models to:<\/p>\n\n\n\n The integration of LLMs with video analytics platforms enriches user interactions and enhances the automation of content summarization, making video data more searchable, accessible, and actionable for diverse applications ranging from security to retail customer behavior analysis and beyond.<\/p>\n\n\n\n As we showcased recently at NVIDIA GTC 2024, we united the capability of TinyML with LLM deployment, presenting an advanced lightweight AI system with deep, context-aware analysis that can be extended from the edge to the cloud: To handle the simple vision AI detection tasks, tinyML with embedded AI components could save us a lot! Check out our SSCMA<\/a> – the open-source project that gathers optimized AI algorithms designed to run effectively on low-end devices while maintaining impressive speed and accuracy. They cover a wide range of applications from abnormal detection to pose estimation and even customized identification tailored to specific needs. For those interested in exploring these AI solutions, SSCMA is available on GitHub<\/a>, along with a comprehensive model zoo<\/a> for direct access.<\/p>\n\n\n The Jetson Orin edge device takes these capabilities further by processing and inferring keyframes identified with abnormalities or significant events through images or live video streams. By leveraging Live Llava<\/a> on the Jetson AGX Orin<\/a>, developers can achieve a deeper understanding of various situations through a multimodal pipeline and Vision and Language Models (VLMs). This setup enables a more nuanced and comprehensive analysis of video content, extracting pivotal information for advanced content search and interpretation.<\/p>\n\n\nHow Generative AI Empowers Video Analytics<\/h3>\n\n\n\n
\n
\n
![\"\"](\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2024\/04\/Computer-vision-vs-GenAI-1030x417.png\")
<\/figure><\/div>\n\n\nDepth-Interpretation by Leveraging TinyML and LLMs Together<\/h4>\n\n\n\n
\u25cfTinyML comes to life through ultra-low power sensors, including cameras and devices that capture environmental and authority data. These sensors feed streamlined, processed data directly into our LLMs.
\u25cfRunning on edge devices powered by Jetson Orin NX<\/a>, the LLMs provide sophisticated, context-aware insights and real-time responses, enhancing scene perception with layers of complexity.
\u25cfAWS underpins this system, providing the necessary infrastructure for deployment, management, and scalability. In parallel, Node-RED provides a cohesive interface that weaves together disparate components and services, forging a cohesive network of devices.<\/p>\n\n\n![\"\"](\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2024\/04\/image-1030x864.png\")
<\/figure><\/div>\n\n\nGet Hands-on Jetson Orin to Experience TinyML & LLM<\/h3>\n\n\n\n
TinyML<\/h4>\n\n\n\n
![\"\"](\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2024\/04\/sscma-object-detection.png\")
<\/a><\/figure><\/div>\n\n\nLive Llava<\/h4>\n\n\n\n
![\"\"](\"https:\/\/www.seeedstudio.com\/blog\/wp-content\/uploads\/2024\/04\/WeChatWorkScreenshot_af5e2045-322f-45bc-bb98-43b058658a29-1030x443.png\")
<\/a><\/figure><\/div>\n\n\n