Loading...
Loading...

Go to the content (press return)

Fast calculation of entropy with Zhang's estimator

Author
Lozano, A.; Casas, B.; Bentz, C.; Ferrer-i-Cancho, R.
Type of activity
Book chapter
Book
Issues in quantitative linguistics 4
First page
273
Last page
285
Publisher
RAM - Verlag
Date of publication
2016-12-01
ISBN
978-3-942303-44-6 Open in new window
Repository
http://hdl.handle.net/2117/100157 Open in new window
Abstract
Entropy is a fundamental property of a repertoire. Here, we present an efficient algorithm to estimate the entropy of types with the help of Zhang’s estimator. The algorithm takes advantage of the fact that the number of different frequencies in a text is in general much smaller than the number of types. We justify the convenience of the algorithm by means of an analysis of the statistical properties of texts from more than 1000 languages. Our work opens up various possibilities for future res...
Citation
Lozano, A., Casas, B., Bentz, C., Ferrer-i-Cancho, R. Fast calculation of entropy with Zhang's estimator. A: "Issues in quantitative linguistics 4". Lüdenscheid: RAM - Verlag, 2016, p. 273-285.
Keywords
Entropy estimation, Lexical diversity, Parallel corpora
Group of research
COMBGRAPH - Combinatorics, Graph Theory and Applications
LARCA - Laboratory of Relational Algorithmics, Complexity and Learnability

Participants