Change search
ReferencesLink to record
Permanent link

Direct link
Randomness versus specifics for word-frequency distributions
Umeå University, Faculty of Science and Technology, Department of Physics. (IceLab)
2016 (English)In: Physica A: Statistical Mechanics and its Applications, ISSN 0378-4371, E-ISSN 1873-2119, Vol. 444, 828-837 p.Article in journal (Refereed) PublishedText
Abstract [en]

The text-length-dependence of real word-frequency distributions can be connected to the general properties of a random book. It is pointed out that this finding has strong implications, when deciding between two conceptually different views on word-frequency distributions, i.e. the specific 'Zipf's-view' and the non-specific 'Randomness-view', as is discussed. It is also noticed that the text-length transformation of a random book does have an exact scaling property precisely for the power-law index gamma = 1, as opposed to the Zipf's exponent gamma = 2 and the implication of this exact scaling property is discussed. However a real text has gamma > 1 and as a consequence gamma increases when shortening a real text. The connections to the predictions from the RGF (Random Group Formation) and to the infinite length-limit of a meta-book are also discussed. The difference between 'curve-fitting' and 'predicting' word-frequency distributions is stressed. It is pointed out that the question of randomness versus specifics for the distribution of outcomes in case of sufficiently complex systems has a much wider relevance than just the word-frequency example analyzed in the present work.

Place, publisher, year, edition, pages
Elsevier, 2016. Vol. 444, 828-837 p.
Keyword [en]
Word-frequency distributions, Zipf's law, Random Group Formation, Maximum entropy
National Category
Probability Theory and Statistics Physical Sciences
URN: urn:nbn:se:umu:diva-114601DOI: 10.1016/j.physa.2015.10.082ISI: 000366785900075OAI: diva2:902601
Available from: 2016-02-11 Created: 2016-01-25 Last updated: 2016-02-11Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Minnhagen, Petter
By organisation
Department of Physics
In the same journal
Physica A: Statistical Mechanics and its Applications
Probability Theory and StatisticsPhysical Sciences

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 21 hits
ReferencesLink to record
Permanent link

Direct link