{"id":91000,"date":"2024-01-12T12:13:13","date_gmt":"2024-01-12T12:13:13","guid":{"rendered":"https:\/\/www.seeedstudio.com\/blog\/?p=91000"},"modified":"2024-02-02T05:51:15","modified_gmt":"2024-02-02T05:51:15","slug":"generative-ai-at-the-edge-fundamentals-what-is-it-and-how-it-works","status":"publish","type":"post","link":"https:\/\/www.seeedstudio.com\/blog\/2024\/01\/12\/generative-ai-at-the-edge-fundamentals-what-is-it-and-how-it-works\/","title":{"rendered":"Generative\u00a0AI\u00a0at\u00a0the\u00a0Edge\u00a0Fundamentals:\u00a0What\u00a0is\u00a0it and\u00a0how\u00a0it\u00a0works?\u00a0"},"content":{"rendered":"\n

The next significant breakthrough in modern AI came with the advent of GPT(Generative Pre-trained Transformer). Leveraging vast amounts of data from the internet, the core technology of GPTs has enabled AI models to be more generalized. Without the limitation of conventional AI approaches like heavily relying on predefined rules and patterns, or even letting machines make predictions based on the relationship between data and label it learned. In the Generative wave, we can make the results flexible and infinite by only tapping or verbally providing a prompt to the machine, making life more convenient and smarter. Now, let’s dive deep to figure out how to understand generative AI at the edge, and what we can do with generative AI!<\/p>\n\n\n

\n
\"Explain<\/figure><\/div>\n\n\n

What is Generative AI\uff1f<\/h2>\n\n\n\n

Generative AI is designed to learn from patterns in unstructured data based on deep-learning neural networks. Employing a blend of unsupervised and semi-supervised methods in its initial training, it also seamlessly integrates supervised techniques during fine-tuning. It enables the model to autonomously craft new, original content with enhanced efficiency and precision across various domains\u2014be it text, images, audio, or synthetic data. Discover our Generative AI website<\/a> for more insights.<\/p>\n\n\n\n

Generative AI Architecture Underneath<\/h3>\n\n\n\n

The concept of Generative AI, which refers to a pre-trained model that serves as the starting point for various downstream tasks, utilizes the Transformer<\/a> as the underlying architecture, where the foundation model is built and pre-trained with a huge amount of data. Once pre-trained, the model can be fine-tuned for specific tasks or used as is for various creative applications, such as language generation, text completion, or even image synthesis. <\/p>\n\n\n\n

Transformer<\/strong><\/h4>\n\n\n\n

The Transformer framework is a groundbreaking architecture in natural language processing, characterized by its encoder-decoder structure and self-attention mechanism with the whole input rather than processing vectors one by one in order.<\/p>\n\n\n

\n
\"Transformer
The transformer model architecture. Image from paper “Attention Is All You Need”<\/a><\/figcaption><\/figure><\/div>\n\n\n

In the encoder, the input sequence undergoes multi-head self-attention, allowing the model to focus on different parts of the sequence simultaneously. The decoder, responsible for generating the output sequence, utilizes a similar mechanism while incorporating additional components like positional encoding to maintain the order of words. Feedforward neural networks within each layer process the information derived from attention mechanisms, enabling the model to capture complex relationships in the data. Additionally, layer normalization and residual connections contribute to stable and efficient training, allowing the model to effectively learn intricate patterns in sequential data.<\/p>\n\n\n\n

Foundation Model<\/strong><\/h4>\n\n\n\n

Understanding the framework of a foundation model involves recognizing its role as a pre-trained model that serves as a starting point for various downstream tasks.<\/p>\n\n\n\n

Take an example of language models like GPT, the foundation model is trained on an extensive and diverse dataset using unsupervised learning. During this pre-training phase, the model learns the nuances of language, capturing contextual relationships and patterns. The resulting foundation model is then fine-tuned for specific tasks or applications, adapting its pre-learned knowledge to address more specialized challenges. <\/p>\n\n\n\n

This pre-training followed by fine-tuning approach empowers the foundation model to excel in a wide range of language-related tasks, from text completion to sentiment analysis. Let’s take a look at some classic foundation models that can be deployed at the Edge:<\/p>\n\n\n\n