Figure 1:AiFigure 1: three-tier structure and core technology module for the technology warehouse
I. The essence of ai: what is artificial intelligence to learn from machines
The core of artificial intelligence, ai, is not to write complex rules, but to allow machines to learn their own rules from data。
Traditional programming is "tell the computer what to do," and programmers need to write detailed instructions for every step. And machine learning is "let the computer learn how to do it" and, with a lot of data training, machines can automatically discover patterns and make decisions。
Figure ii: a diagram of the rationale of the nervous network
Figure 2: transmitting information from multi-layer neural networks
Three main learning paradigms
Ai's learning methods fall into three main categories:
IiIn-depth learningThe working principles of the technology engine neural network in ai
In-depth learning is currently at the core of ai. It simulates the learning process of the human brain through a multilayered neural network, enabling the automatic extraction of the data's deep characteristics。
Four steps in the training process:
Step 1: forward dissemination - data flow from the input layer to the output layer, and each layer process the data and eventually obtain a projection。
Step 2: calculate loss - compare the gap between the forecast results and the real answer, quantifying this gap with the loss function。
Step 3: reverse diffusion - calculates the contribution of each parameter to the loss (the gradient) and determines which direction the parameter should be adjusted。
Step 4: parameter update - adjust the network parameters according to the gradient, using optimized algorithms such as gradient decline, and repeat the process until the model is condensed。
Figure iii: three core ai competencies and application scenarios
IiiLarge modelTimes: transformer andGptThe rise of transformer: the revolutionary breakthrough of ai
In 2017, the transformer structure proposed by google radically changed the ai area. Its core innovation is the self-focused mechanism (self-attention), which allows models to focus on important components of input, achieves parallel calculations and significantly improves training efficiency。
Three-stage pre-training (pre-training) for large models: training on mass unmarked text to give models a common knowledge of language. Gpt-3 uses 45tb text data. Fine-twining: trains on task-specific labeling data to adapt models to specific application scenarios, such as customer dialogue, code generation. Alignment: making model outputs more responsive to human preferences and safer through human feedback and enhanced learning (rlhf). From theory to practice: key element 1. Data is the basis
High-quality training data are the basis for ai performance. Data quality includes four dimensions:
Algorithms are core
The choice of algorithms needs to take into account factors such as the type of task, the size of the data, the real-time requirement, and interpretability. Small data use traditional methods and large data use in-depth learning。
3. Validity as security
The training of large models requires strong computing support. Gpt-3 spent $10,000 on v100 gpu for months of training at a cost of millions of dollars. Solutions include cloud computing, model compression, knowledge distillation, etc。
Concluding remarks
The core logic of artificial intelligence can be summarized as a data-driven approach that allows machines to learn the mapping relationship from input to output。
From the most basic linear regression to complex large-language models, the essence is to optimize this mapping function. The difference is:
Understanding the principles of ai not only helps us to better use the tools of ai, but also keeps us awake in the age of ai. Technology is advancing, but human creativity, judgement and values are never irreplaceable。
Let's hug ai, but not get lost in ai。
