Natural Language Processing for Historical TextsMore and more historical texts are becoming available in digital form. Digitization of paper documents is motivated by the aim of preserving cultural heritage and making it more accessible, both to laypeople and scholars. As digital images cannot be searched for text, digitization projects increasingly strive to create digital text, which can be searched and otherwise automatically processed, in addition to facsimiles. Indeed, the emerging field of digital humanities heavily relies on the availability of digital text for its studies. Together with the increasing availability of historical texts in digital form, there is a growing interest in applying natural language processing (NLP) methods and tools to historical texts. However, the specific linguistic properties of historical texts -- the lack of standardized orthography, in particular -- pose special challenges for NLP. This book aims to give an introduction to NLP for historical texts and an overview of the state of the art in this field. The book starts with an overview of methods for the acquisition of historical texts (scanning and OCR), discusses text encoding and annotation schemes, and presents examples of corpora of historical texts in a variety of languages. The book then discusses specific methods, such as creating part-of-speech taggers for historical languages or handling spelling variation. A final chapter analyzes the relationship between NLP and the digital humanities. Certain recently emerging textual genres, such as SMS, social media, and chat messages, or newsgroup and forum postings share a number of properties with historical texts, for example, nonstandard orthography and grammar, and profuse use of abbreviations. The methods and techniques required for the effective processing of historical texts are thus also of interest for research in other domains. Table of Contents: Introduction / NLP and Digital Humanities / Spelling in Historical Texts / Acquiring Historical Texts / Text Encoding and Annotation Schemes / Handling Spelling Variation / NLP Tools for Historical Languages / Historical Corpora / Conclusion / Bibliography |
Contents
Introduction | 1 |
NLP and Digital Humanities | 5 |
Spelling in Historical Texts | 11 |
Acquiring Historical Texts | 25 |
Text Encoding and Annotation Schemes | 53 |
Handling Spelling Variation | 69 |
NLP Tools for Historical Languages | 85 |
Historical Corpora | 101 |
Conclusion | 117 |
Bibliography | 119 |
Authors Biography | 145 |
Other editions - View all
Common terms and phrases
accessed algorithm alignment applied approach automatic blackletter Boschetti canonicalization century Chapter character encoding cited Computational Linguistics contains correct diachronic dictionary digital humanities digitization projects Early Modern English edit distance evaluation example Figure FineReader freely available French Gotscharek High German historical corpora historical documents historical languages historical spelling historical texts images Language Resources Language Technology Latin lemma Levenshtein edit distance lexicon manuscript markup Medieval methods Middle High German million running word modern German modern languages modern texts modern word form morphological n-grams natural language processing NLP for historical NLP tools OCR engines OCR output OCRopus Old Czech overview part-of-speech part-of-speech tagging POS tagger POS tagging problem Proceedings query query expansion running word forms scanning script spell-checker spelling variation standardized orthography string tagset Technology for Cultural tokens transcribed transcription Treebank TreeTagger typefaces types Unicode variants Volk