学习c语言未来的就业方向_语言学习的未来

学习c语言未来的就业方向

Making quality language learning content is hard. Duolingo, which is perhaps the most prolific creator of language learning content, says that just creating a quality curriculum with 2000 sentences can take a team up to nine months. And that’s just to cover half of what they consider to be a Common European Framework of Reference for Languages A1 level course, which essentially means “rank beginner”. (1) At Verbal Earth, when creating our flagship Mandarin Travel product, we found the development of content is considerably more challenging than the development of the software systems to deliver that content.

制作高质量的语言学习内容非常困难。 Duolingo可能是语言学习内容的最多产的创造者，他说，仅创建包含2000个句子的高质量课程，团队最多需要9个月的时间。 这仅仅是覆盖他们认为是欧洲通用语言参考框架A1级课程的一半，这实际上意味着“高级初学者”。 (1)在Verbal Earth中，当我们创建旗舰产品Mandarin Travel产品时，我们发现内容的开发要比提供该内容的软件系统更具挑战性。

Why is creating language learning content so hard? A good course will need to teach the vocabulary and sentence structures a learner is most likely to need first. It will need to introduce vocabulary gradually, both to reuse previously introduced vocabulary to reinforce the learning of that vocabulary, and avoid “cognitive overload” in learners by introducing too much unfamiliar vocabulary at once. Course developers need to imagine scenarios in which a learner will be using the language, and invent sentences that give a learner the language micro-skills to use in those scenarios, all the while hoping they are not missing some key concepts.

为什么创建语言学习内容如此困难？ 一门好的课程将需要教一个学习者最可能首先需要的词汇和句子结构。 它将需要逐步引入词汇，以重用先前引入的词汇以加强该词汇的学习，并通过一次引入过多的陌生词汇来避免学习者的“认知超负荷”。 课程开发人员需要想象学习者将使用该语言的场景，并发明一些句子，为学习者提供在这些场景中要使用的语言微技能，同时希望他们不会错过某些关键概念。

“Good language classes will give the beginner comprehensible input that the outside world will supply only very reluctantly.” — Stephen Krashen

“ 语言课程将为初学者提供令人难以理解的输入，外界只会非常不情愿地提供这些输入。” —斯蒂芬·克拉申

Software can assist with this challenging process. It can give feedback on how quickly a curriculum introduces new vocabulary, and how well it does at reinforcing previously introduced vocabulary under a spaced repetition inspired metric, or other criteria. At Verbal Earth, all of our lesson sequences were continually evaluated on many metrics during development, allowing them to be refined before being put into production. Duolingo uses tools to make sure not too much vocabulary is being introduced in a lesson, and that its used in a sufficient number of sentences. These tools can make for a better product, but it is still a very laborious process for content creators.

软件可以协助这一具有挑战性的过程。 它可以就课程引入新词汇的速度以及在间隔重复启发度量或其他条件下增强先前引入的词汇的效果如何提供反馈。 在“言语地球”中，我们在开发过程中不断对我们的所有课程序列进行许多评估，从而使它们在投入生产之前可以得到完善。 Duolingo使用工具来确保在课程中没有引入太多的词汇，并且确保使用了足够多的句子。 这些工具可以使产品更好，但是对于内容创建者而言，这仍然是一个非常费力的过程。

As the leader of the curriculum development effort for Mandarn Travel, a nagging question kept popping into my head: There is already so much language content being organically generated every day, so why don’t we use it for language learning? If a curriculum developer is creating content to help a learner develop language skills useful in a restaurant, why should she need to think up scenarios, enumerate key vocabulary and language patterns, and shape thousands of invented sentences into a coherent course, when there are real customers interacting with restaurant staff millions of times a day, “producing” content that is by definition ideal for a restaurant setting? And there are terabytes, if not petabytes of existing written, audio and video content on the web, covering every imaginable subject. Can this be harnessed in a way that makes it useful to language learners?

作为Mandarn Travel课程开发工作的负责人，一个烦人的问题不断浮现在脑海：每天已经有机地产生了太多的语言内容，为什么我们不将其用于语言学习呢？ 如果课程开发人员正在创建内容以帮助学习者发展在餐厅中有用的语言技能，那么为什么她需要考虑情景，列举关键的词汇和语言模式，并在实际情况中将成千上万个发明的句子塑造成连贯的课程客户每天与餐厅工作人员互动数百万次，“生产”的内容在定义上最适合餐厅环境吗？ 网络上存在数TB甚至数PB的现有书面，音频和视频内容，涵盖了所有可以想象的主题。 能否以对语言学习者有用的方式加以利用？

机器策展和内容的聚宝盆 (Machine curation and the cornucopia of content)

In principle, there is sufficient content already existing on the internet today to provide all the content needed for a language learner of most major languages. The central challenge is that almost none of it is comprehensible input for language learners. If a machine is going to curate these terabytes of data into a curriculum that is useful to a language student, it will need to introduce the most important vocabulary and grammar first, and build gradually over time with sufficient repetition of previous vocabulary, much as human curriculum creators are currently doing.

原则上，当今互联网上已经有足够的内容来提供大多数主要语言的语言学习者所需的所有内容。 中心挑战是，对于语言学习者来说，几乎所有这些都不是可理解的输入 。 如果机器打算将这些TB级数据整理成对语言学生有用的课程，则它需要首先引入最重要的词汇和语法，并随着时间的推移逐步构建，并充分重复以前的词汇，就像人类一样课程创建者当前正在做。

One way of approaching machine curation of language content for use in language learning is as an optimization problem. In principle a machine could generate an “optimal” curriculum for a general audience, based on criteria such as a prioritized list of vocabulary to be covered, and a desired degree of spaced repetition and pace of introduction of new vocabulary. For a particular domain of language, such as train travel or hotel accommodations, a prioritized list of vocabulary is in principle readily attainable simply by processing the transcripts of real conversations in those contexts, and noting the frequencies of word usage.

解决用于语言学习的语言内容的机器管理的一种方法是作为优化问题。 原则上，一台机器可以根据诸如要覆盖的词汇表的优先顺序列表，所需的间隔重复程度和引入新词汇表的速度等标准为普通读者生成“最佳”课程。 对于特定的语言领域，例如火车旅行或旅馆住宿，原则上只需在这些上下文中处理真实对话的记录，并注意单词使用的频率，就可以容易地获得优先排序的词汇表。

Instead of software providing feedback to human content creators and the creators inventing new sentences, as is done currently, the software can instead choose an appropriate sequence of sentences from its vast library of content. A machine curator might look at the billions of sentences available to it, and choose “I am” as the first sentence in the curriculum because it uses the highest priority vocabulary. It might choose “I am fine” as a follow on, since it introduces a single new word, while giving practice with previous vocabulary. Candidate sentences are initially very few, but as a hundred or more core vocabulary are introduced, the options for the curator open up in an exponential fashion.

该软件可以像目前所做的那样，代替向人类内容创建者提供反馈和发明新句子的创建者，而可以从其庞大的内容库中选择合适的句子序列。 机器管理员可能会查看数十亿个可用的句子，并选择“我是”作为课程表中的第一句，因为它使用了最高优先级的词汇。 它可能会选择“我很好”作为后续操作，因为它引入了一个新单词，同时使用以前的词汇进行了练习。 候选句子最初很少，但是随着引入一百或更多的核心词汇，策展人的选择以指数方式打开。

Building a curriculum then becomes a task of introducing one new word at a time ideally, while optimally reusing vocabulary that has already been introduced in the curriculum. Instead of a human content creator coming up with new sentences, which relies on the limits of their creativity and is biased by their personal memories and blind spots, the spontaneous creative language of thousands of humans is harnessed to provide optimal content.

因此，建立课程表成为一项任务，即一次理想地引入一个新单词，同时最佳地重用课程表中已引入的词汇。 代替人类内容的创建者提出新的句子，后者依赖于他们的创造力的局限性，并因其个人记忆和盲点而受偏见，而是利用成千上万的人类自发的创造力语言来提供最佳的内容。

Having billions of sentences available to choose from, the machine curator will also be able to have the student work with new sentences that don’t introduce any new words at all, but instead serve to practice existing vocabulary in new contexts. This opens up the possibility of review lessons that are entirely new content, but are limited to vocabulary the learner already has familiarity with, giving learners the positive reward of demonstrating they can understand the words in new contexts.

拥有数十亿个句子供您选择，机器管理员还可以让学生使用新句子，而这些新句子根本不会引入任何新单词，而是可以在新环境中练习现有词汇。 这为复习课提供了可能，这些复习课是全新的内容，但仅限于学习者已经熟悉的词汇，从而给学习者带来积极的奖励，表明他们可以在新的上下文中理解单词。

The work of this machine curator would no doubt be an extraordinary computational challenge, but with modern computing resources is not insurmountable. And with the steady march of machine learning techniques and other data science, the possibilities expand continuously.

毫无疑问，这个机器策展人的工作将是一个巨大的计算挑战，但是使用现代计算资源并不是无法克服的。 随着机器学习技术和其他数据科学的不断发展，可能性不断扩大。

掌握实用语言 (Capturing Practical Language)

One big problem with most of the content on the web is that it’s not very suitable for beginning language learners. It’s both very “advanced”, using a lot of specialized vocabulary, and it tends to be not very useful in daily life. A beginning language learner typically needs to learn aspects of the language that enable them to function well in common situations such as getting around, getting fed, taking care of household needs, and basic social interactions. There is not much content published on the web dealing with these subjects, since they are not of much interest to native speakers. Conversations with taxi drivers, ticket agents, and clothing vendors may be very useful material to a beginning language learner seeking to build practical language skills, but there are not many such conversations published on the web, except perhaps in fictional content that briefly touches on these situations.

网络上大多数内容的一个大问题是，它不太适合入门语言学习者。 两者都非常“先进”，使用了大量专业词汇，在日常生活中往往不是很有用。 初级语言学习者通常需要学习语言的各个方面，以使其能够在常见的情况下正常运行，例如四处走动，进食，照顾家庭需求以及基本的社交互动。 网络上没有太多内容涉及这些主题，因为母语人士对它们没有太大兴趣。 与出租车司机，售票员和服装供应商的对话对于寻求建立实用语言技能的初学语言的学习者可能是非常有用的材料，但是在网络上发布的此类对话并不多，除了虚构的内容可以简短地涉及这些内容情况。

However there is another cornucopia waiting to be captured: The millions of conversations that go on between ordinary people in ordinary situations every day. In principle microphones can be placed at grocery checkouts, hotel counters, train stations, and the like to capture these “mundane” conversations. Speech to text has become very accurate, and machine learning techniques are quite good at separating different voices when multiple microphones are available (2). By capturing these every day conversations and adding them to our machine curator’s library of language learning material, a huge amount of material highly useful to beginning learners can be made available.

但是，还有另一个聚宝盆在等待捕获：每天在普通情况下的普通人之间进行的数百万次对话。 原则上，可以在杂货店结帐处，酒店柜台，火车站等处放置麦克风，以捕获这些“平凡的”对话。 文字语音变得非常准确，并且当有多个麦克风可用时，机器学习技术非常擅长分离不同的声音(2)。 通过捕获这些日常对话并将其添加到我们的机器管理员的语言学习资料库中，可以提供大量对初学者非常有用的资料。

Clearly privacy and even legal concerns need to be considered. Laws regarding recording conversations vary around the world, though it is generally allowable where there is no expectation of privacy, which is the case in most practical everyday situations mentioned above. Privacy concerns can be essentially eradicated by capturing only the transcripts of the conversations, and rendered by a text-to-speech system when audio learning content is required. Identifying names and other personal information in transcripts can also be automatically obfuscated (“de-identification”) by trained machine learning models (3).

显然，需要考虑隐私甚至法律问题。 关于录制对话的法律在世界各地有所不同，尽管在不希望有隐私的情况下通常是可以允许的，在上述大多数实际日常情况下就是这种情况。 通过仅捕获对话的记录即可消除隐私问题，并在需要音频学习内容时由文本语音转换系统呈现。 还可通过训练有素的机器学习模型自动混淆身份证明中的姓名和其他个人信息(“取消身份识别”)(3)。

Another challenge is that some spontaneous speech may not be suitable for language learning. Written prose usually consists of well formed sentences, whereas speakers tend to stop sentences mid way through, then express their idea in a better way, or make glaring mistakes that make a sentence grammatically incorrect, even if its still comprehensible to the listener. These conversations are useful for listening practice in their original form, but are not suitable in many language learning situations. Humans can easily decide what is appropriate, and curate this material ‘by hand’, but it may be possible for machine learning models to be created to do this work automatically on a massive scale.

另一个挑战是某些自发的语音可能不适合语言学习。 书面散文通常由结构良好的句子组成，而说话者往往会在句子中途停止句子，然后以更好的方式表达自己的想法，或者犯下明显的错误，使句子在语法上不正确，即使听者仍然可以理解。 这些对话对于原始形式的听力练习很有用，但不适用于许多语言学习情况。 人们可以轻松地决定什么是合适的，并“手工”整理该材料，但是可能会创建机器学习模型来自动大规模地完成这项工作。

Verbal Earth has been experimenting with using recordings of real conversations for use in its curriculum. We think that this is a very promising domain of exploration. With some effort, vast amounts of organically generated real world conversations can potentially be collected for language learning purposes.

“言语地球”一直在尝试将真实对话的录音用于其课程。 我们认为这是一个非常有希望的探索领域。 通过一些努力，可以潜在地收集大量有机生成的现实世界对话以用于语言学习。

Overview of the architecture for an individualized machine curation system

个性化机器管理系统的体系结构概述

寻找“恰到好处的句子” (Finding ‘just the right sentence’)

One challenge for our machine curator is finding just the right sentence or phrase. When introducing a new word, it’s possible there may be no sentence available that does not also use other new words, even with billions of sentences to choose from. Or there are no good choices for reinforcing previous vocabulary. This is less of a problem in natural spontaneous speech, which is frequently short, incomplete sentences. However written prose tends to be composed of long, complex sentences with multiple clauses, which complicates its usefulness in a language learning curriculum.

对于我们的机器策展人来说，挑战之一就是找到正确的句子或短语。 引入一个新单词时，即使有数十亿个句子可供选择，也可能没有不使用其他新单词的句子。 或者没有很好的选择来增强以前的词汇。 在自然自发的讲话中，这通常是简短的，不完整的句子，这不是问题。 但是，书面散文往往由带有多个子句的长而复杂的句子组成，这使其在语言学习课程中的实用性变得复杂。

Take the last sentence of the last paragraph as an example. There are quite a few words that are important in the domain of discussing language learning: “sentences”, “clauses”, “learning”, “curriculum”, etc. For somebody learning English in this domain, the sentence may be useful as practice to somebody already familiar with all this vocabulary, but for a learner just getting started it may overwhelm them with unfamiliar vocabulary.

以最后一段的最后一句话为例。 在讨论语言学习的领域中，有很多重要的单词： “句子”，“从句”，“学习”，“课程”等。对于在该领域学习英语的人，该句子在实践中可能很有用对于已经熟悉所有这些词汇的人来说，但是对于刚入门的学习者来说，它可能会让他们不熟悉的词汇不知所措。

However, this does not make the sentence useless to a machine curator, whose objective is to slowly introduce vocabulary. Long sentences are always composed of comprehensible chunks, which are not fully formed “complete” sentences, but nonetheless convey meaningful units of language. These are typically found in clauses, in noun phrases, or other grammatical structures. For example, “language learning curriculum”. This is a noun phrase that conveys a coherent unit of meaning. This phrase is useful to a machine curator when a learner is familiar with the words “language” and “learning”, but not “curriculum”, as it allows a new word to be introduced in an appropriate context, cognitively tying its meaning to words the learner already knows. Other examples of comprehensible chunks from this sentence are “complex sentences with multiple clauses,” and “written prose tends to be composed of long, complex sentences.”

但是，这对于机器管理员来说并不是没有用处，机器管理员的目的是慢慢引入词汇。 长句子总是由可理解的块组成，这些块不是完全构成的“完整”句子，但是却传达了有意义的语言单元。 这些通常出现在从句，名词短语或其他语法结构中。 例如， “语言学习课程” 。 这是一个名词短语，传达了一个连贯的含义单元。 当学习者熟悉单词“语言”和“学习”而不是“课程”时 ，此短语对机器策展人很有用，因为它允许在适当的上下文中引入新单词，并将其含义与单词相联系学习者已经知道。 该句子中可理解的块的其他示例是“带有多个子句的复杂句子”和“书面散文往往由冗长而复杂的句子组成”。

By identifying the comprehensible chunks of all the billions of sentences in its library, the machine curator then has much more flexibility in its choices for composing a curriculum. 10 billions options becomes 50 billion options, making it easier for a machine curator to find ‘just the right sentence’ that introduces a new word while optimally reinforcing familiar vocabulary, or new sentences that use existing vocabulary in new ways.

通过识别库中所有数十亿个句子中可理解的部分 ，机器策展人可以在选择课程时拥有更大的灵活性。 100亿个选项变成500亿个选项，使机器管理员可以更轻松地找到引入一个新单词的“恰如其分的句子”，同时以最佳方式增强熟悉的词汇表或以新方式使用现有词汇表的新句子。

Comprehensible chunk is a somewhat vague concept, but it turns out that machine learning techniques are good at building models capable of accurately working with vague concepts, so this task of finding the comprehensible chunks can be done by machine. Sentiment analysis, which attempts to classify the sentiment behind a written product or other review (e.g. “good”, “neutral”, “bad”) is one of the oldest use cases of Natural Language Processing. It’s an inherently vague concept, but it turns out with modern deep learning models, machines can achieve results similar to humans.

可理解的块是一个有点模糊的概念，但是事实证明，机器学习技术擅长构建能够准确地处理模糊概念的模型，因此查找可理解的块的任务可以由机器来完成。 情感分析试图对书面产品或其他评论(例如，“好”，“中立”，“差”)背后的情感进行分类，这是自然语言处理最古老的用例之一。 这是一个固有的模糊概念，但事实证明，使用现代深度学习模型，机器可以达到与人类相似的结果。

At Verbal Earth, we are researching the usefulness of machine learning based on Google’s BERT natural language processing model to perform this task. It takes as input an English sentence, and outputs a set of useful comprehensible chunks. Currently the model has a 92% precision rate, meaning 92% of the outputs are considered comprehensible by a human, and not incomprehensible jibberish such as “a time, passed the place where”. We anticipate even better results in the future as our research, and Natural Language Processing models in general progress.

在Verbal Earth，我们正在研究基于Google的BERT自然语言处理模型的机器学习对执行此任务的有用性。 它以英语句子为输入，并输出一组有用的可理解块。 当前，该模型具有92％的准确率，这意味着人类认为92％的输出是可理解的，而不是诸如“时间，经过的地方”之类的不可理解的杂物。 随着我们的研究以及自然语言处理模型的全面进展，我们预计将来会有更好的结果。

机器作为私人导师 (Machine as personal tutor)

Having a generalized curriculum created by a machine curator would be a fantastic achievement, but what if we could go even further, and enable the machine to curate language learning content to meet specific individualized needs? There are two main differences between learners that distinguish them as individuals: learning objectives, and current abilities. By taking into account the current interests of a learner, and their current language skills, a much more personalized curriculum can be created on the fly, turning the machine curator into a personal tutor.

由机器策展人创建通用课程将是一个了不起的成就，但是如果我们走得更远，并使机器能够策划语言学习内容以满足特定的个性化需求，该怎么办？ 在学习者之间，有两个主要区别将他们区分为个人：学习目标和当前能力。 通过考虑学习者的当前兴趣及其当前的语言技能，可以即时创建更多个性化的课程，从而将机器策展人转变为私人导师。

One learner may be interested in building language skills to help them travel in a foreign country, while another may be interested in discussing cooking with a friend in their native tongue. The machine curator, when presenting new content to the first learner can prioritize vocabulary useful in transportation, or shopping. For the second, vocabulary about various ingredients and kitchen utensils can be introduced. Most language learners find that there are units in curriculums made for a general audience that do not interest them, while other units that do interest fall short of meeting their needs.

一个学习者可能对培养语言技能以帮助他们到国外旅行感兴趣，而另一个学习者可能对与朋友用母语交谈来讨论烹饪感兴趣。 机器策展人在向第一学习者介绍新内容时，可以优先考虑可用于运输或购物的词汇。 第二，可以介绍有关各种食材和厨房用具的词汇。 大多数语言学习者发现，课程表中有一些针对普通读者的单元不感兴趣，而其他感兴趣的单元则无法满足他们的需求。

If a machine curator also knows what vocabulary a learner is already comfortable with, and to what degree, it can provide a highly useful customized curriculum that reinforces words already known that need practice, and know which new words need to be introduced to help a learner achieve their goals. Rather than choosing content from it’s library based on a general curriculum for all learners, it can choose specifically to meet a learners unique needs and interests.

如果机器策展人也知道学习者已经熟悉了哪些词汇，以及在什么程度上学习，它可以提供一种非常有用的定制课程，以强化需要练习的已知单词，并知道需要引入哪些新单词来帮助学习者实现他们的目标。 它可以根据特定学习者的独特需求和兴趣而选择特定的内容，而不是根据其针对所有学习者的通用课程表从图书馆中选择内容。

If a machine curator also knows what vocabulary a learner is already comfortable with, and to what degree, it can provide a highly useful customized curriculum that reinforces words already known that need practice, and know which new words need to be introduced to help a learner achieve their goals. Rather than choosing content from it’s library based on a general curriculum for all learners, it can choose specifically to meet a learners unique needs and interests.

如果机器策展人也知道学习者已经熟悉了哪些词汇，以及在什么程度上学习，它可以提供一种非常有用的定制课程，以强化需要练习的已知单词，并知道需要引入哪些新单词来帮助学习者实现他们的目标。 它可以根据特定学习者的独特需求和兴趣而选择特定的内容，而不是根据其针对所有学习者的通用课程表从图书馆中选择内容。

Duolingo is currently providing some individualization in it’s lessons and practice sessions, by evaluating how a learning is doing with the current material . More complete sentences and new vocabulary can be introduced more quickly if the learning seems to be finding the material easy, and material that the learner is having difficulty can be repeated, or similar examples provided. (4) LingQ, which does not have a set curriculum, recommends new content by evaluating how much a learner already knows.

Duolingo目前正在通过评估学习对当前材料的学习方式，在其课程和练习课中提供一些个性化设置。 如果学习似乎很容易找到素材，并且可以重复学习者遇到困难的素材，或者提供类似的示例，则可以更快地引入更完整的句子和新词汇。 (4)LingQ没有固定的课程设置，它通过评估学习者已经知道的知识来推荐新内容。

Probably the easiest way for a machine curator to keep track of a learners language skills is to keep track of all vocabulary used in the generated curriculum thus far, and provide a way for learners to provide direct feedback to the system about familiarity with vocabulary. Familiar words can get a lower priority when the machine curator selects new sentences, but still prioritized from time to time to make sure it stays “fresh” in a learners memory. The machine curator can make an effort to find more usage examples of a word a learner is having difficulty with, by finding sentences in the cornucopia that use the word, and also use words the learner is comfortable with.

机器策展人跟踪学习者语言技能的最简单方法可能是跟踪到目前为止生成的课程表中使用的所有词汇，并为学习者提供一种向系统提供有关词汇熟悉度的直接反馈的方式。 当机器管理员选择新句子时，熟悉的单词可能会获得较低的优先级，但仍会不时地进行优先排序，以确保它在学习者的记忆中保持“新鲜”。 机器策展人可以通过在聚宝盆中找到使用该单词的句子，并使用该学习者喜欢的单词，来努力找到学习者所遇到的单词的更多用法示例。

结论 (Conclusion)

Making quality content for language learning is hard. Fortunately, quality language content is constantly being generated in an organic cornucopia by native speakers and writers of language every day. Using machines to curate this vast amount of content into material suitable for language learners, not just for a general audience, but to meet the specific needs of individual learners, is quickly becoming possible. The exponential growth of computing resources, and the capabilities of machine learning systems to perform the mundane tasks that previously only humans were capable of is what will enable it. Rather than relying on the creative efforts of a few curriculum developers, the vast humanity of millions of native speakers will be made accessible to language learners.

为语言学习制作高质量的内容很难。 幸运的是，每天由母语为母语的人和作家在有机聚宝盆中不断产生高质量的语言内容。 使用机器将大量内容整理成适合语言学习者的材料，不仅适合于普通读者，而且还可以满足单个学习者的特定需求，这已成为可能。 计算资源的指数级增长，以及机器学习系统执行以前只有人类才能完成的日常任务的能力，才能够实现这一目标。 语言学习者将可以访问数百万以母语为母语的人，而不是依靠少数课程开发人员的创造性努力。

翻译自: https://medium.com/@marktsanford/the-future-of-language-learning-e2a710453ecf

学习c语言未来的就业方向