In the realm of machine learning, the few-shot problem is a persistent challenge, particularly in fields like experimental science and medical research where data is often scarce. Traditional data-driven models struggle with overfitting and poor generalization when data is limited. However, a groundbreaking study led by Chuanjiang Qi from the School of Automation and Electrical Engineering at the University of Science and Technology Beijing might just change the game. Published in the prestigious journal ‘工程科学学报’ (Journal of Engineering Sciences), Qi’s research introduces a novel approach that combines domain knowledge with data, potentially revolutionizing industries like energy that rely heavily on predictive modeling.
The few-shot problem arises when there isn’t enough data to train a model effectively. “Pure data-driven learning relies heavily on the quality and quantity of data,” explains Qi. “When data is scarce, the model is prone to overfitting and its generalization ability will decrease.” To tackle this, Qi and his team developed the Knowledge and Data Cross-Modal Fusion Model (KDFM), a hybrid approach that integrates domain knowledge with numerical data to enhance model performance.
So, how does it work? The KDFM model first categorizes numerical data into different feature types and models them using graphs. These graphs are constructed based on K-means clustering, and multichannel graph convolution is used to process the different types of numerical features. This converts numerical data into graph-level features, making them more expressive.
But the real magic happens when domain knowledge comes into play. The team represents domain knowledge features from semantic modalities using a knowledge graph. Key entities and relationships are extracted from professional books and expert experiences, forming triples that convert unstructured text features into graph-level features. “Textual domain knowledge and experience are organized and converted into the neural network model,” says Qi.
The model then employs a graph convolutional neural network and attention mechanisms to achieve cross-modal feature fusion between knowledge and data. The input includes different graphs constructed from numerical data, feature vectors obtained from the knowledge graph, and numerical vectors from the data. Multichannel graph convolution is applied to achieve deep feature fusion, resulting in a fused multichannel feature vector that serves as the input for downstream tasks.
The KDFM model was validated using two small sample datasets: one for a regression task in the materials field and another for a classification task in the medical field. The results were impressive. In the regression task, the model achieved the best results in terms of mean squared error, mean absolute error, and R², with R² exceeding the suboptimal multilayer perceptron model by over 7%. In the classification task, the model was optimal in five out of seven indicators.
The implications for the energy sector are significant. Energy companies often deal with complex datasets and rely on predictive models for everything from resource exploration to maintenance scheduling. The KDFM model could enhance the accuracy of these models, leading to more efficient operations and significant cost savings.
Qi’s research also highlights the importance of integrating domain knowledge with data. “Most fields have accumulated extensive experience and knowledge,” he notes. “A hybrid approach that combines domain knowledge with data can effectively improve model performance.”
The KDFM model addresses the challenges of weak generalization ability and the integration of knowledge and data modalities in few-shot problems. As Qi puts it, “The proposed model addresses, to some extent, the challenges of weak generalization ability and the integration of knowledge and data modalities in few-shot problems.”
This research could pave the way for future developments in machine learning, particularly in fields where data is scarce but domain knowledge is abundant. It’s a testament to the power of combining traditional knowledge with cutting-edge technology, and it’s a development that energy sector professionals will be watching closely.