三种分类：
term–document
word–context
pair–pattern
semantics：the meaning of a word, a phrase, a sentence, or any text in human language, and the study of such meaning
特点：
从语料库中自动获取信息，节省工作量
衡量词、短语、文档相似性
The Term–Document Matrix：行向量为词terms,列向量为文档documents
bag:可包含重复元素的集合，表示为矩阵X in which each column x:j corresponds to a bag, each row xi: corresponds to a unique member, and an element xij is the frequency of the i-th member in the j-th bag
word–context
The distributional hypothesis in linguistics is that words that occur in similar contexts
tend to have similar meanings
pair–pattern
mason : stone
carpenter : wood
X cuts Y
“X works with Y
extended distributional hypothesis, that patterns that co-occur with similar pairs tend to have similar meanings、
latent relation hypothesis is that pairs of words that co-occur in similar patterns
tend to have similar semantic relations
attributional similarity： word–context sima(a, b) ∈R
relational similarity：pair–pattern simr(a : b, c : d) ∈R
A token is a single instance of a symbol, whereas a type is a general class of tokens
Statistical semantics hypothesis
If units of text have similar vectors in a text frequency matrix, then they tend to have similar meanings
Bag of words hypothesis
If documents and pseudodocuments (queries) have similar column vectors in a term–document matrix, then they tend to have similar meanings
Distributional hypothesis
If words have similar row vectors in a word–context matrix, then they tend to have similar meanings
Extended distributional hypothesis
If patterns have similar column vectors in a pair–pattern matrix, then they tend to express similar semantic relations
Latent relation hypothesis
If word pairs have similar row vectors in a pair–pattern matrix, then they tend to have similar semantic relations
Linguistic Processing for Vector Space Models
1.tokenize the raw text： decide what constitutes a term and how to extract terms from raw text
punctuation (e.g., don’t, Jane’s, and/or), hyphenation (e.g., state-of-the-art versus state of the art), and recognize multi-word terms (e.g., Barack Obama and ice hockey)
2.normalize the raw text: convert superficially different strings of characters to the same form
Case folding
3.annotate the raw text: mark identical strings of characters as being different
Mathematical Processing for Vector Space Models
1. generate a matrix of frequencies
First, scan sequentially through the corpus, recording events and their frequencies in a hash table, a database, or a search engine index. Second, use the resulting data structure to generate the frequency matrix, with a sparse matrix representation
2.adjust the weights of the elements in the matrix
tf-idf (term frequency × inverse document frequency) family of weighting functions
length normalization
Term weighting
Pointwise Mutual Information (PMI) problem： infrequent events
3.smooth the matrix to reduce the amount of random noise and to fill in some of the zero elements in a sparse matrix
Singular Value Decomposition (SVD)奇异值分解
latent meaning, noise reduction, high-order co-occurrence, and sparsity reduction
Optimizations and parallelization for similarity computing
sparse-matrix multiplication 相关性分解成三个部分，X的非零值，Y的非零值，X,Y中的非零值
分布式处理mapreduce hadoop
randomized algorithm: dimension reduction
machine learning