freecodecamp_freeCodeCamp的服务器到底发生了什么？

freecodecamp

Update at 17:00 California time: We have now fixed most of the problems. We're still working on a few known issues, but /learn is now fully operational.

加利福尼亚时间17:00更新 ：我们现在解决了大多数问题。 我们仍在处理一些已知问题，但是/ learn现在可以完全使用。

Here was the culprit - a regular expression-based query that was running against millions of database records every time a someone tried to authenticate.

罪魁祸首是一个基于正则表达式的查询，每次有人尝试进行身份验证时，该查询都会针对数百万个数据库记录运行。

And here's our cluster's CPU usage before and after we fixed the problem:

这是修复问题前后我们群集的CPU使用率：

This may seem obvious, but it took Mrugesh 3 days of detective work to identify the bottleneck.

这看起来似乎很明显，但是Mrugesh花费了 3天的侦探工作来确定瓶颈。

We've concluded that we could have spotted this earlier with a $30 per month tool, so we broke down and bought it for future usage.

我们得出的结论是，我们可以使用每月30美元的工具早些时候发现它，因此我们抛售了它，以备将来使用。

What follows is a more in-depth explanation of what happened.

接下来是对发生的事情的更深入的解释。

简而言之： (In short:)

• On Tuesday we rolled out a ton of new code. Including code that allow us to continuously deliver new features and bug fixes.
  在星期二，我们推出了许多新代码。 包括允许我们不断提供新功能和错误修复的代码。
• We thought we'd load-tested our new code enough. But it wasn't high-performance enough for the sudden weight of 2,000 concurrent users.
  我们认为我们已经对新代码进行了足够的负载测试。 但是对于2000个并发用户的突然增加，它的性能还不够。
• /forum and /news worked fine, but the sign-in functionality on /learn was unreliable for 3 days.
  / forum和/ news可以正常工作，但是/ learn上的登录功能在3天内不可靠。

好。 现在更多细节。 (OK. Now some more detail.)

For the past 10 months or so, we've been accumulating new features, bug fixes, and curriculum improvements.

在过去的10个月左右，我们一直在积累新功能，错误修复和课程改进。

We've continued to merge improvements into our Master branch and deploy them to our beta server at www.freecodecamp.dev.

我们继续将改进合并到我们的Master分支中，并将其部署到我们的beta服务器，网址为www.freecodecamp.dev。

For the past 2 months, lots of contributors have people using this beta version of freeCodeCamp.

在过去的两个月中，许多贡献者都在使用此版本的freeCodeCamp的人们。

And in celebration of freeCodeCamp's 5th anniversary this month, we wanted to go ahead and push all of these improvements into production at www.freecodecamp.org.

为了庆祝本月freeCodeCamp成立5周年，我们希望继续前进，并在www.freecodecamp.org上将所有这些改进投入生产。

On Tuesday, we went down for what we thought would be 3 minutes of planned maintenance. We did a final testing run, took a database backup, sent out a "we'll be right back" tweet, and pushed 10 months worth of code live all at once.

在星期二，我们进行了3分钟的计划维护。 我们进行了最后的测试，备份了数据库，发出了一条“我们马上回来”的推文，并一次发布了10个月的代码。

But Murphy's Law was waiting behind the corner to club us in the knees. And as more and more traffic piled on, our servers buckled.

但是墨菲定律在拐角处等着我们。 随着越来越多的流量堆积，我们的服务器崩溃了。

We were able to get /forum and /news back up almost immediately. But /learn required authentication, and hit some additional API endpoints and servers. So for 3 days we scrambled to get it to work.

我们几乎可以立即获得/ forum和/ news的备份。 但是/ learn需要身份验证，并需要使用其他一些API端点和服务器。 因此，我们花了3天时间努力使它正常工作。

It turned out our new code wasn't as high-performance as we'd thought, and we were hitting our API servers a lot more than necessary.

事实证明，我们的新代码不如我们想像的那样具有高性能，并且我们对API服务器的攻击远远超出了必要。

So we identified parts of the codebase that were making unnecessary API calls and refactored them, while also juggling DevOps challenges.

因此，我们确定了代码库中进行不必要的API调用的部分并对其进行了重构，同时还应对了DevOps的挑战。

为什么会这样呢？ 真？ (Why did this happen, though? Really?)

At the end of the day, this outage was my fault.

归根结底，这次故障是我的错。

Here's why.

这就是为什么。

Our total budget for 2019 is only about $300,000. And yet we're helping millions of people learn to code every month.

我们2019年的总预算仅为300,000美元。 但是，我们每个月都在帮助数百万人学习编码。

We now get more traffic than other learn-to-code sites like Udacity and Codecademy. We get even more traffic than mainstream news websites like TechCrunch.

现在，与Udacity和Codecademy等其他学习型代码站点相比，我们获得的流量更多。 我们获得的流量甚至超过TechCrunch等主流新闻网站。

When you operate at such extreme scale with such a paltry budget, you end up clipping coupons.

当您以如此微不足道的预算在如此极端的规模上运作时，您最终会得到优惠券。

Those giant servers that could give you comfortable overhead for spikes? Too expensive.

那些巨大的服务器可以为您带来峰值的开销？ 太贵了。

Those fancy DevOps services that identify choke points? Too expensive.

那些可以识别阻塞点的高级DevOps服务？ 太贵了

Our team of 5 engineers ends up doing the work of 10.

我们由5名工程师组成的团队最终完成了10名工作。

My point is - it's my fault freeCodeCamp only has $300,000 to work with this year. To put that number perspective, I know individual developers in San Francisco whose salary is larger than $300,000.

我的意思是-这是我的错freeCodeCamp今年只有30万美元可以使用。 从数字角度来看，我知道旧金山的个人开发人员的薪水超过300,000美元。

There's nothing wrong with having a big salary. San Francisco is an expensive city.

高薪没有错。 旧金山是一个昂贵的城市。

But there is a problem when freeCodeCamp - one of the largest education sites on the internet - is trying to operate on such a comically tiny budget.

但是当freeCodeCamp(互联网上最大的教育网站之一)试图以如此可笑的小预算运作时，就会出现问题。

Again, it's my fault.

再次，这是我的错。

I'm inexperienced at grassroots fundraising.

我对基层筹款没有经验。

I'm still learning how to raise awareness of all the work we're doing for the community.

我仍在学习如何提高对我们为社区所做的所有工作的认识。

I am shy when it comes to asking you all to donate money to fund that work.

当我要求大家捐款以资助这项工作时，我很害羞。

So I am going to make a concerted effort to get better, and to increase our budget.

因此，我将共同努力以求更好，并增加我们的预算。

I don't want to run ads.

我不想投放广告。

I don't want to say "freeCodeCamp, brought to you by the Acme Corporation."

我不想说“由Acme Corporation带给您的freeCodeCamp”。

And of course, I don't ever want to charge learners for our learning resources.

当然，我绝对不想向学习者收取我们的学习资源。

So far, we haven't had to do any of these things.

到目前为止，我们还没有做任何这些事情。

But this really only leaves us with one source for funding. We, the people.

但这实际上只给我们提供了一种资金来源。 我们人类。

freeCodeCamp is a grass-roots donor-supported nonprofit. We just need to get better at asking people for money.

freeCodeCamp是由基层捐助者支持的非营利组织。 我们只需要变得更好就可以向人们索要钱。

We are heading into the holiday season. This is when about 80% of the year's charitable gifts are made here in the US.

我们即将进入假期。 这是当年大约80％的慈善礼物是在美国制造的。

So I'm going to stay focused on this. I will document what I learn as I experiment. And I will eventually create a fundraising handbook for other grass-roots donor-supported nonprofits based on what I learn.

所以我将继续专注于此。 我将记录我在实验中学到的东西。 最后，我将根据我所学的知识，为其他基层捐助者支持的非营利组织编写筹款手册。

This is a bit embarrassing, but our current donate page is down today because our authentication is still wonky.

这有点令人尴尬，但是由于我们的身份验证仍然很糟糕，我们当前的捐赠页面今天已关闭。

So I've set up a PayPal page where you can make tax-deductible one-time donations to freeCodeCamp.

因此，我建立了一个PayPal页面，您可以在其中一次性向freeCodeCamp捐款进行免税捐款 。

We still welcome your monthly support of freeCodeCamp. Your $5 donations each month are what makes freeCodeCamp possible, and what gives us the stable budget to plan ahead.

我们仍然欢迎您每月对freeCodeCamp的支持。 您每个月的$ 5捐款使freeCodeCamp成为可能，并使我们有稳定的预算来提前进行计划。

But if you do have a some extra cash on hand for a one-time donation, it would be a huge help.

但是，如果您确实有一笔额外的现金可用于一次性捐赠， 那将是巨大的帮助 。

这是我对您的承诺： (Here is my commitment to you:)

freeCodeCamp will stay free.

freeCodeCamp将保持免费。

freeCodeCamp will not run ads.

freeCodeCamp将不会投放广告。

And when we ask you to donate, we will do so tastefully and honestly. We won't use pathos, or make those "we're going bankrupt unless you donate right now" type claims that some other nonprofits resort to.

当我们要求您捐赠时，我们将做到高雅而诚实。 我们不会使用悲哀，也不会让那些“除非您现在就捐赠，否则我们将破产”的类型声称其他一些非营利组织会采取这种行动。

Because the reality is this: Even if freeCodeCamp completely ran out of money, we would still keep going.

因为现实是这样的：即使freeCodeCamp完全没钱了，我们仍然会继续前进。

Yes, we would have to lay everyone off, including myself. But I would go get some other job and pay for the servers myself.

是的，我们必须解雇所有人，包括我自己。 但是我会去找其他工作，自己支付服务器费用。

Because freeCodeCamp is clearly something that the world needs.

因为freeCodeCamp显然是世界需要的东西。

I've poured 5 years of my life into this community. I've poured $150,000 of my personal savings from my teaching career into freeCodeCamp.

我已经为这个社区投入了5年的生命。 我从我的教学生涯中将我的个人储蓄中的$ 150,000投入了freeCodeCamp。

freeCodeCamp will never die.

freeCodeCamp将永生。

It's just a question of how vibrantly freeCodeCamp can live.

这只是freeCodeCamp可以活得多么生动的问题。

Again, if you have some cash to spare, we are a highly efficient nonprofit, and we will put it to effective use. Donate here.

同样，如果您有剩余的现金，我们将是一个高效的非营利组织，我们将充分利用它。 在这里捐款 。

And thank you again for your patience with the outage on /learn.

再次感谢您对/ learn中断的耐心等待。

Once we've fixed all this, I will let you all know, and I'll publish my 5th anniversary article that details all the big improvements we have for our 5th anniversary.

解决所有问题后，我将通知大家，我将发表5周年纪念文章，其中详细介绍了5周年纪念日我们取得的所有重大改进。

Happy coding.

快乐的编码。

翻译自: https://www.freecodecamp.org/news/freecodecamp-servers-update-october-2019/

freecodecamp