展开查看详情
1. Mailchimp Scale:
A MySQL Perspective
John Scott
Mailchimp
2.What is
Mailchimp’s secret
sauce?
Hint: It’s not much of a secret.
2
3.Focus on the small business
“Empowering the Underdog”
3
4. “We give marketers
production-ready software
designed to help them
grow…”
Mailchimp Engineering Mission Statement
https://mailchimp.com/culture/how-our-engineering-team-found-its-mission-statement/
5.5
6.Another way to say it
“We SCALE through
togetherness,
momentum,
and pragmatism.”
6
7.Old Mentality: The 3 Disciplines of Data
Administration
● OPS / KTLO
● Support
● Performance
8.Old Mentality: The 3 Disciplines of Data
Administration
● OPS / KTLO
● Support
● Performance
“I’m a DevOps DBA”
9.Old Mentality: The 3 Disciplines of Data
Administration
● OPS / KTLO
● Support
● Performance
10.Old Mentality: The 3 Disciplines of Data
Administration
● OPS / KTLO
● Support
● Performance
“I help other
departments work
with databases”
11.Old Mentality: The 3 Disciplines of Data
Administration
● OPS / KTLO
● Support
● Performance
12.Old Mentality: The 3 Disciplines of Data
Administration
● OPS / KTLO
● Support
● Performance
“over the fence”
13.New Mentality:
“Ops is product”
14.Ops is Product
“If you improve database
performance resulting in 10%
reduction in churn, you would
create an additional
number>.”
15.Ops is Product
“Developer
Enablement”
New paradigm “looking at ops through the lens of product” --Tyler Treat
● https://bravenewgeek.com/operations-in-the-world-of-developer-enablement/
● https://www.youtube.com/watch?v=JUy3GYkPfto
OR in the case of Mailchimp, ops actually developing software, too.
16.Developer Enablement
Product Enablement
In most organizations “Product enablement” is
sales term with the “four Ps”
● Positioning
● Pitch
● Play
● Program
17.Developer Enablement
Product Enablement
1000 employees
350+ engineers
0 salespeople
18.Mailchimp “Board Room”
19.
20. Sounds great.
But what does that mean for a
database engineer?
21.#togetherness in action
MySQL log analysis based on
pt-query-digest and
Elasticsearch / Kibana
resulted in a Top 20 table
activity graph
22.End of story?
“Toss it over the wall.”
“Not my problem.”
“I don’t have commit rights.”
23.This is Mailchimp Engineering
“We succeed through togetherness,
Momentum, and Pragmatism”
24.We identified an N+1
pattern and fixed it,
together.
25.But wait....
26.What was the impact to the user
experience?
27.265 billion queries per week
247 thousand unique
query fingerprints
2200 Instances of mysql
28.Old Mentality: Effective Slow Query Log
Analysis Across The Infrastructure FTW!
“Query Macroeconomics”
https://johnscott.net/2018/08/03/query-macroeconomics/
● Prioritize query fixes by how much DB capacity you get back
○ MySQL not stressed with contention equals what?
■ A pretty innodb status?
■ Nice looking graphs?
29.Old Mentality: Effective Slow Query Log
Analysis Across The Infrastructure FTW!
“Query Macroeconomics”
https://johnscott.net/2018/08/03/query-macroeconomics/
● Prioritize query fixes by how much DB capacity you get back
○ MySQL not stressed with contention equals what?
■ A pretty innodb status?
■ Nice looking graphs?