Distributed Representation Computation Using CBOW Model and Skip–gram Model
Takamune Onishi, Hiromitsu Shiina
Abstract
Word2Vec is one of methods for generating a distribution representation. Two methods of calculation, CBOW model and skip-gram model, are proposed. However, the skip-gram model has high accuracy of the distribution representation of words, but it takes a long time to learn it. On the other hand, the CBOW model has been shown to be faster but less accurate than the skip-gram model in the distribution representation of words. In this study, we propose a method to combine the CBOW model and the skip-gram model in a way that shares the weight matrix of the CBOW model with the skip-gram model in order to achieve both learning speed and accuracy of the distribution representation of words.
Topics & Concepts
n-gramWord2vecComputer scienceRepresentation (politics)ComputationBag-of-words modelArtificial intelligenceGramian matrixAlgorithmPattern recognition (psychology)Language modelQuantum mechanicsLawEmbeddingPoliticsPhysicsPolitical scienceEigenvalues and eigenvectorsTopic ModelingNatural Language Processing TechniquesAdvanced Graph Neural Networks