AI-törsten på icke-digitala språk skapar nya jobb

Korpuslingvistik kallas metoden att använda en korpus, en stor samling språkliga data, för språkstudier. Termen myntades först på 1980-talet, men metoden började användas flera årtionden dessförinnan. En svensk pionjär på 1960-talet var Sture Allén med Nusvensk frekvensordbok.

A language model is a probabilistic model of a natural language that can generate probabilities of a series of words, based on text corpora in one or multiple languages it was trained on. In 1980, the first significant statistical language model was proposed, and during the decade IBM performed ‘Shannon-style’ experiments, in which potential sources for language modeling improvement were identified by observing and analyzing the performance of human subjects in predicting or correcting text.Language models are useful for a variety of tasks, including speech recognition (helping prevent predictions of low-probability (e.g. nonsense) sequences), machine translation, natural language generation (generating more human-like text), optical character recognition, handwriting recognition, grammar induction, and information retrieval.Large language models, currently their most advanced form, are a combination of larger datasets (frequently using scraped words from the public internet), feedforward neural networks, and transformers. They have superseded recurrent neural network-based models, which had previously superseded the pure statistical models, such as word n-gram language model.

Kannada eller kanaresiska är ett dravidiskt språk med 35 327 600 talare, varav majoriteten (35,3 miljoner, 1997) i Indien. Det talas i delstaterna Karnataka, Andhra Pradesh, Tamil Nadu och Maharashtra. Det finns också talare i Kanada och USA. I USA finns det ungefär 48 000 talare. Kannada har ordföljden subjekt–objekt–verb. Kannada anses vara ett livskraftigt språk och det delas i tre huvuddialekter.