|
|
|
题名
|
作者
|
年代
|
出处
|
被引量
|
| 1 | Pre-trained models for natural language processing: A survey显示文摘Recently, the emergence of pre-trained models(PTMs) has brought natural language processing(NLP) to a new era. In this survey, we provide a comprehensive review of PTMs for NLP. We first briefly introduce language representation learning and its research progress. Then we systematically categorize existing PTMs based on a taxonomy from four different perspectives. Next,we describe how to adapt the knowledge of PTMs to downstream tasks. Finally, we outline some potential directions of PTMs for future research. This survey is purposed to be a hands-on guide for understanding, using, and developing PTMs for various NLP tasks. | QIU XiPeng SUN TianXiang XU YiGe SHAO YunFan DAI Ning HUANG XuanJing | 2020 | Science China(Technological Sciences)2020，63，10： | 109 |
| 2 | Dual-axial self-attention network for text classification显示文摘Text classification is an important task in natural language processing and numerous studies aim to improve the accuracy and efficiency of text classification models. In this study, we propose an effective and efficient text classification model which is based on self-attention solely. The recently proposed multidimensional self-attention significantly improved the performance of self-attention. However, existing models suffer from two major limitations:(1) the previous multi-dimensional self-attention models are quite timeconsuming;(2) the dependencies of elements along the feature axis are not taken into account. To overcome these problems, in this paper, a much more computational efficient multi-dimensional self-attention model is proposed, and two parallel self-attention modules, called dual-axial self-attention, are applied to capture rich dependencies along the feature axis as well as the text axis. A text classification model is then derived.The experimental results on eight representative datasets show that the proposed text classification model can obtain state-of-the-art results and the proposed self-attention outperforms conventional self-attention models. | Xiaochuan ZHANG Xipeng QIU Jianmin PANG Fudong LIU Xingwei LI | 2021 | Science China(Information Sciences)2021，64，12： | 2 |
| 3 | Recongnizing Inference in Iexts with Markov Logic Networks显示文摘 | Xipeng Qiu Ling Cao Zhao Liu Xuan jing Huang | 2012 | ACM Language Information Processing2012，11，4： | 1 |
| 4 | Info-margin maximization for feature ex- traction显示文摘 | Qiu Xipeng Wu Lide | 2009 | Pattern Recognition Letters2009，30，16： | 1 |
| 5 | Thermal behavior of the YAG precursor prepered by sol-gel combustion process显示文摘 | QIU Fagui PU Xipeng LI Jiang | 2005 | Ceram Int2005，31，： | 1 |
| 6 | Implicit discourse relation detection using concatenated word embeddings and a gated relevance network显示文摘Dear editor,Discourse relation detection involves recognizing the relationships between pairs of discourse fragments(e.g.,clauses or sentences).As compared with explicit detection,implicit discourse relation detection is much more challenging,owing to connective words,such as'so'or'because',being absent.In such cases,the relationships between the fragments pairs cannot be via simple frequencybased mapping;instead,the relationships must beinferred from potential semantic and logical connections between the two arguments. | Jinlan FU Qi ZHANG Jifan CHEN Minlong PENG Tao GUI Xipeng QIU Xuanjing HUANG | 2019 | Science China(Information Sciences)2019，62，11： | 0 |
| 7 | ChatGPT: potential, prospects, and limitations显示文摘Recently,OpenAI released Chat Generative Pre-trained Transformer(ChatGPT)(Schulman et al.,2022)(http://gffzzdefde08283c749e2skf55vcfwfbok6cpp.ffgz.tsg.suse.edu.cn),which has attracted considerable attention from the industry and academia because of its impressive abilities.This is the first time that such a variety of open tasks can be well solved within one large language model.To better understand ChatGPT,we briefly introduce its history,discuss its advantages and disadvantages,and point out several potential applications.Finally,we analyze its impact on the development of trustworthy artificial intelligence,conversational search engine,and artificial general intelligence. | Jie ZHOU Pei KE Xipeng QIU Minlie HUANG Junping ZHANG | 2024 | Frontiers of Information Technology & Electronic Engineering2024，25，1： | 0 |
| 8 | Text information aggregation with centrality attention显示文摘A lot of natural language processing problems need to encode the text sequence as a fix-length vector, which usually involves an aggregation process of combining the representations of all the words, such as pooling or self-attention. However, these widely used aggregation approaches do not take higher-order relationships among the words into consideration. Hence we propose a new way of obtaining aggregation weights, called eigen-centrality self-attention. More specifically, we build a fully-connected graph for all the words in a sentence, then compute the eigen-centrality as the attention score of each word. The explicit modeling of relationships as a graph is able to capture some higher-order dependency among words, which helps us achieve better results in 5 text classification tasks and one SNLI task than baseline models such as pooling, self-attention, and dynamic routing. Besides, in order to compute the dominant eigenvector of the graph, we adopt a power method algorithm to get the eigen-centrality measure. Moreover, we also derive an iterative approach to get the gradient for the power method process to reduce both memory consumption and computation requirement. | Jingjing GONG Hang YAN Yining ZHENG Qipeng GUO Xipeng QIU Xuanjing HUANG | 2021 | Science China(Information Sciences)2021，64，12： | 0 |
| 9 | Syntax-guided text generation via graph neural network显示文摘Text generation is a fundamental and important task in natural language processing.Most of the existing models generate text in a sequential manner and have difficulty modeling complex dependency structures.In this paper,we treat the text generation task as a graph generation problem exploiting both syntactic and word-ordering relationships.Leveraging the framework of the graph neural network,we propose the word graph model.During the process,the model builds a sentence incrementally and maintains syntactic integrity via a syntax-driven,top-down,breadth-first generation process.Experimental results on both synthetic and real text generation tasks show the efficacy of our approach. | Qipeng GUO Xipeng QIU Xiangyang XUE Zheng ZHANG | 2021 | Science China(Information Sciences)2021，64，5： | 0 |
您所在的机构暂未开通此部分的功能，请联系管理员