umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Randomness versus specifics for word-frequency distributions
Umeå University, Faculty of Science and Technology, Department of Physics. (IceLab)
2016 (English)In: Physica A: Statistical Mechanics and its Applications, ISSN 0378-4371, E-ISSN 1873-2119, Vol. 444, 828-837 p.Article in journal (Refereed) Published
Resource type
Text
Abstract [en]

The text-length-dependence of real word-frequency distributions can be connected to the general properties of a random book. It is pointed out that this finding has strong implications, when deciding between two conceptually different views on word-frequency distributions, i.e. the specific 'Zipf's-view' and the non-specific 'Randomness-view', as is discussed. It is also noticed that the text-length transformation of a random book does have an exact scaling property precisely for the power-law index gamma = 1, as opposed to the Zipf's exponent gamma = 2 and the implication of this exact scaling property is discussed. However a real text has gamma > 1 and as a consequence gamma increases when shortening a real text. The connections to the predictions from the RGF (Random Group Formation) and to the infinite length-limit of a meta-book are also discussed. The difference between 'curve-fitting' and 'predicting' word-frequency distributions is stressed. It is pointed out that the question of randomness versus specifics for the distribution of outcomes in case of sufficiently complex systems has a much wider relevance than just the word-frequency example analyzed in the present work.

Place, publisher, year, edition, pages
Elsevier, 2016. Vol. 444, 828-837 p.
Keyword [en]
Word-frequency distributions, Zipf's law, Random Group Formation, Maximum entropy
National Category
Probability Theory and Statistics Physical Sciences
Identifiers
URN: urn:nbn:se:umu:diva-114601DOI: 10.1016/j.physa.2015.10.082ISI: 000366785900075OAI: oai:DiVA.org:umu-114601DiVA: diva2:902601
Available from: 2016-02-11 Created: 2016-01-25 Last updated: 2016-02-11Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Minnhagen, Petter
By organisation
Department of Physics
In the same journal
Physica A: Statistical Mechanics and its Applications
Probability Theory and StatisticsPhysical Sciences

Search outside of DiVA

GoogleGoogle Scholar

Altmetric score

Total: 43 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf