forked from piskvorky/gensim
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindex.html
More file actions
257 lines (221 loc) · 12.3 KB
/
Copy pathindex.html
File metadata and controls
257 lines (221 loc) · 12.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Gensim – Topic Modelling for Humans — gensim</title>
<link rel="stylesheet" href="_static/default.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
VERSION: '0.8.6',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="author" title="About these documents" href="about.html" />
<link rel="top" title="gensim" href="#" />
<link rel="next" title="Introduction" href="intro.html" />
<!-- twitter search widget
<script type="text/javascript" src="_static/widget.js"></script>
-->
<meta property="og:title" content="#gensim" />
<meta property="og:description" content="Efficient topic modelling in Python" />
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-24066335-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
<script type="text/javascript">
(function() {
var at = document.createElement('script'); at.type = 'text/javascript'; at.async = true;
at.src = ('https:' == document.location.protocol ? 'https://' : 'http://') + 's7.addthis.com/js/250/addthis_widget.js#pubid=ra-4d738b9b1d31ccbd';
var sat = document.getElementsByTagName('script')[0]; sat.parentNode.insertBefore(at, sat);
})();
</script>
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="intro.html" title="Introduction"
accesskey="N">next</a> |</li>
<li><a href="#">Gensim home</a>| </li>
<li><a href="tutorial.html">Tutorials</a>| </li>
<li><a href="http://groups.google.com/group/gensim">Support</a>| </li>
<li><a href="https://github.com/piskvorky/gensim/wiki">Contribute</a>| </li>
<li><a href="apiref.html">API reference</a>»</li>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper"><h3>Download</h3>
<p>Get <cite>gensim</cite> version 0.8.6 from the <a href="http://pypi.python.org/pypi/gensim">Python Package
Index</a> or install directly with:</p>
<pre>easy_install -U gensim</pre>
<h3><a href="#">Table Of Contents</a></h3>
<ul>
<li class="toctree-l1"><a class="reference internal" href="intro.html">Introduction</a></li>
<li class="toctree-l1"><a class="reference internal" href="install.html">Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="tutorial.html">Tutorials</a></li>
<li class="toctree-l1"><a class="reference internal" href="distributed.html">Distributed Computing</a></li>
<li class="toctree-l1"><a class="reference internal" href="wiki.html">Experiments on the English Wikipedia</a></li>
<li class="toctree-l1"><a class="reference internal" href="apiref.html">API Reference</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<input type="text" name="q" size="24" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script><h3>Questions? Suggestions?</h3>
<p>Join the <a href="http://groups.google.com/group/gensim">Google discussion group</a></p>
<form action="http://groups.google.com/group/gensim/boxsubscribe">
<input type="text" name="email" placeholder="your@email.com" size="24" />
<input type="submit" name="sub" value="Subscribe" />
</form>
<!-- AddThis Button BEGIN -->
<div class="addthis_toolbox addthis_default_style addthis_32x32_style"
addthis:title="#gensim"
addthis:description="Efficient Topic Modelling in Python"
style="margin:20px 0 0 0">
<script type="text/javascript">
var addthis_share = {
templates: { twitter: 'Efficient topic modelling in Python #gensim http://goo.gl/2VQBS' }
}
</script>
<a class="addthis_button_preferred_1"></a>
<a class="addthis_button_preferred_2"></a>
<a class="addthis_button_preferred_3"></a>
<a class="addthis_button_preferred_4"></a>
<a class="addthis_button_compact"></a>
<a class="addthis_counter addthis_bubble_style"></a>
</div>
<!-- AddThis Button END -->
<p>
Check the source code at <a href="https://github.com/piskvorky/gensim">Github</a>. Report bugs at the <a href="https://github.com/piskvorky/gensim/issues">issue tracker</a>.
</p>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="gensim-topic-modelling-for-humans">
<h1>Gensim – Topic Modelling for Humans<a class="headerlink" href="#gensim-topic-modelling-for-humans" title="Permalink to this headline">¶</a></h1>
<div id="tagcloud">
<span id="0" class="wrd tagcloud2">algorithms</span>
<span id="1" class="wrd tagcloud2">analysis</span>
<span id="2" class="wrd tagcloud5">answer</span>
<span id="3" class="wrd tagcloud2">api</span>
<span id="7" class="wrd tagcloud2">collection</span>
<span id="8" class="wrd tagcloud2">concepts</span>
<span id="10" class="wrd tagcloud8">corpus</span>
<span id="11" class="wrd tagcloud2">design</span>
<span id="12" class="wrd tagcloud9">documents</span>
<span id="14" class="wrd tagcloud3">features</span>
<span id="15" class="wrd tagcloud0">framework</span>
<span id="17" class="wrd tagcloud1">human</span>
<span id="18" class="wrd tagcloud2">index</span>
<span id="19" class="wrd tagcloud0">infer</span>
<span id="20" class="wrd tagcloud0">install</span>
<span id="21" class="wrd tagcloud0">introduction</span>
<span id="5" class="wrd tagcloud7">latent</span>
<span id="4" class="wrd tagcloud3">dirichlet allocation</span>
<span id="24" class="wrd tagcloud4">model</span>
<span id="26" class="wrd tagcloud6">open-source</span>
<span id="27" class="wrd tagcloud0">paragraphs</span>
<span id="29" class="wrd tagcloud3">python</span>
<span id="29" class="wrd tagcloud6">query</span>
<span id="30" class="wrd tagcloud4">questions</span>
<span id="31" class="wrd tagcloud0">random</span>
<span id="32" class="wrd tagcloud2">reference</span>
<span id="33" class="wrd tagcloud6">representation</span>
<span id="37" class="wrd tagcloud10">semantic</span>
<span id="37" class="wrd tagcloud7">similar</span>
<span id="38" class="wrd tagcloud5">space</span>
<span id="39" class="wrd tagcloud2">sparse</span>
<span id="40" class="wrd tagcloud2">structure</span>
<span id="13" class="wrd tagcloud3">SVD</span>
<span id="41" class="wrd tagcloud2">text</span>
<span id="42" class="wrd tagcloud0">thought</span>
<span id="43" class="wrd tagcloud5">topic</span>
<span id="44" class="wrd tagcloud3">training</span>
<span id="45" class="wrd tagcloud2">tutorials</span>
<span id="46" class="wrd tagcloud1">unsupervised</span>
<span id="48" class="wrd tagcloud2">vector</span>
<span id="49" class="wrd tagcloud5">words</span>
</div>
<table class="links" style="font-size: 200%;">
<tr>
<td><a href='intro.html'>What is this?</a></td>
<td><a href='install.html'>Installation</a></td>
</tr>
<tr>
<td><a href='tutorial.html'>Get started!</a></td>
<td><a href='about.html'>About</a></td>
</tr>
</table><div class="section" id="quick-reference-example">
<h2>Quick Reference Example<a class="headerlink" href="#quick-reference-example" title="Permalink to this headline">¶</a></h2>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">gensim</span> <span class="kn">import</span> <span class="n">corpora</span><span class="p">,</span> <span class="n">models</span><span class="p">,</span> <span class="n">similarities</span>
<span class="go">>>></span>
<span class="gp">>>> </span><span class="c"># Load corpus iterator from a Matrix Market file on disk.</span>
<span class="gp">>>> </span><span class="n">corpus</span> <span class="o">=</span> <span class="n">corpora</span><span class="o">.</span><span class="n">MmCorpus</span><span class="p">(</span><span class="s">'/path/to/corpus.mm'</span><span class="p">)</span>
<span class="go">>>></span>
<span class="gp">>>> </span><span class="c"># Initialize a transformation (Latent Semantic Indexing with 200 latent dimensions).</span>
<span class="gp">>>> </span><span class="n">lsi</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">LsiModel</span><span class="p">(</span><span class="n">corpus</span><span class="p">,</span> <span class="n">num_topics</span><span class="o">=</span><span class="mi">200</span><span class="p">)</span>
<span class="go">>>></span>
<span class="gp">>>> </span><span class="c"># Convert another corpus to the latent space and index it.</span>
<span class="gp">>>> </span><span class="n">index</span> <span class="o">=</span> <span class="n">similarities</span><span class="o">.</span><span class="n">MatrixSimilarity</span><span class="p">(</span><span class="n">lsi</span><span class="p">[</span><span class="n">another_corpus</span><span class="p">])</span>
<span class="go">>>></span>
<span class="gp">>>> </span><span class="c"># determine similarity of a query document against each document in the index</span>
<span class="gp">>>> </span><span class="n">sims</span> <span class="o">=</span> <span class="n">index</span><span class="p">[</span><span class="n">query</span><span class="p">]</span>
</pre></div>
</div>
<div class="admonition-what-s-new admonition">
<p class="first admonition-title">What’s new?</p>
<ul class="last simple">
<li>15 Sep 2012: release 0.8.6 : added the <a class="reference external" href="http://en.wikipedia.org/wiki/Hashing-Trick">hashing trick</a> to allow online changes to the vocabulary; fixed parallel lemmatization + <a class="reference external" href="https://github.com/piskvorky/gensim/blob/develop/CHANGELOG.txt">other minor improvements</a></li>
<li>22 Jul 2012: release 0.8.5 : better Wikipedia parsing, faster similarity queries, maintenance fixes</li>
<li>30 Apr 2012: William Bert’s <a class="reference external" href="http://williamjohnbert.com/2012/04/interview-with-radim-rehurek-creator-of-gensim/">interview with me</a></li>
<li>9 Mar 2012: release 0.8.4: new model <a class="reference external" href="http://radimrehurek.com/gensim/models/hdpmodel.html">Hierarchical Dirichlet Process</a> (full <a class="reference external" href="https://github.com/piskvorky/gensim/blob/develop/CHANGELOG.txt">CHANGELOG</a>)</li>
</ul>
</div>
<div class="toctree-wrapper compound">
</div>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
© Copyright 2009-2012, Radim Řehůřek <radimrehurek(at)seznam.cz>.
Last updated on Sep 16, 2012.
</div>
</body>
</html>