firebase a/b_使用Firebase在Android应用中运行A / B测试

firebase a/b

An A/B test is a powerful tool for a team working on a mobile application. It allows you to split your users into various groups and give a different version of your app to each one simultaneously, without external factors impacting one version more than another. You can use it to compare the performances of the different versions based solely on how good they are at satisfying your users' needs.

A / B 测试是团队工作在移动应用程序上的强大工具。 它使您可以将用户分为不同的组，并同时为每个用户提供不同版本的应用程序，而不会受到外部因素的影响。 您可以使用它仅根据它们在满足用户需求方面的表现来比较不同版本的性能。

In this article, I will take you through how to set up an A/B test in your Android app, using Firebase, and hopefully convince you to start an experiment if you’ve never done so before!

在本文中，我将带您逐步了解如何使用Firebase在Android应用中设置A / B测试，并希望说服您开始实验(如果您从未尝试过的话)！

为什么要运行A / B测试？ (Why should you run an A/B Test?)

Running an A/B test is a great way to understand the impact of each change you introduce in your app, and make sure they are improving your conversion rates and other key metrics. I believe you should try to run a test every time you’re introducing a change in the key features of your app.

进行A / B测试是了解您在应用程序中引入的每个更改的影响并确保它们正在改善您的转化率和其他关键指标的好方法。 我相信，每当您对应用程序的主要功能进行更改时，都应该尝试进行测试。

For example, I’ve recently run two experiments in the app I’m working on, on two crucial parts of the app. The first was on the registration flow. We thought that modifying the order in which we asked information about our user could improve the number of users going through the whole funnel, but were hesitating between two different versions. We ran a test between those versions versus the previous, and we saw that both versions had a significant improvement on the previous version and that one of the two was performing a bit better than the other one, which allowed us to choose one.

例如，我最近在正在开发的应用中的两个关键部分上运行了两个实验。 首先是注册流程。 我们认为，修改询问用户信息的顺序可以提高遍历整个渠道的用户数量，但在两个不同版本之间犹豫不决。 我们在这些版本与以前的版本之间进行了测试，我们发现这两个版本都比先前的版本有了显着改进，并且两个版本中的一个的性能要优于另一个版本，因此我们可以选择其中一个。

The other experiment was on the most important part of our app, the key feature people use our app for. We thought we could improve the experience of users on that screen, and came up with an alternative solution with our UX team, about which we were quite excited. We ran the A/B test for a few weeks and discovered that the solution we thought of didn’t actually improve the performance at all. Thanks to this experiment, we avoided releasing a change that would not have had a positive impact on our users, and we decided to spend more time on the subject and come up with another solution, one that would actually improve the experience of our users.

另一个实验是在我们应用程序的最重要部分上，人们使用我们的应用程序的主要功能。 我们认为我们可以改善用户在该屏幕上的体验，因此我们的UX团队提出了一个替代解决方案，对此我们感到非常兴奋。 我们进行了A / B测试几周，发现我们认为的解决方案实际上并没有真正改善性能。 通过此实验，我们避免了发布不会对用户产生积极影响的更改，因此我们决定花更多时间在该主题上，并提出另一种解决方案，该解决方案实际上可以改善用户体验。

Firebase工具：远程配置和A / B测试 (Firebase tools: Remote Config and A/B tests)

The Firebase platform lets you run A/B tests of different types. You can for example try different types of Notifications, which could help you find out which texts or icon makes users want to click on the push more. You can also test In-App Messages, and discover which message would drive more engagement.

Firebase平台可让您运行不同类型的A / B测试。 例如，您可以尝试使用不同类型的Notifications ，这可以帮助您确定哪些文本或图标使用户想要更多地单击推送。 您还可以测试“ 应用内消息” ，并发现哪条消息可以吸引更多用户。

The 3rd type of experiment is the one that interests us here: Remote Config. This tool enables us to control parts of the app directly from the Firebase console. It lets you define a series of parameters, and set a value for each in the Console. From the app, you then query the Firebase Remote Config SDK and retrieve the value of each parameter. Based on the value, you can adjust the corresponding feature in the app. For example, you can choose to show or hide a feature, show different versions of the same screen, modify the text of a message in the app, the color of a button, etc…

实验的第三种类型是我们感兴趣的一种： Remote Config 。 该工具使我们能够直接从Firebase控制台控制应用程序的各个部分。 它使您可以定义一系列参数，并在控制台中为每个参数设置一个值。 然后，从应用程序中查询Firebase远程配置SDK，并检索每个参数的值。 根据该值，您可以在应用程序中调整相应的功能。 例如，您可以选择显示或隐藏功能，显示同一屏幕的不同版本，修改应用程序中的消息文本，按钮的颜色等等。

The way the A/B test tool from Firebase works with Remote Config is that it will split your users into different groups, and give a different value for the chosen parameter depending on which group the user belongs to. You’re then able to track the performance of each value in the Firebase Console and decide which one you want to keep. Once you have chosen, you can put a stop to the test, and roll out the chosen variant to all your users by setting the winning value in the parameter in Remote Config. This means that all your users will immediately get the best performing variant, without needing to update their app.

Firebase的A / B测试工具与Remote Config配合使用的方式是，它将用户划分为不同的组，并根据用户所属的组为所选参数提供不同的值。 然后，您可以在Firebase控制台中跟踪每个值的性能，并确定要保留哪个值。 选择之后，您可以停止测试，并通过在Remote Config中的参数中设置获胜值，向所有用户推出所选变体。 这意味着您的所有用户将立即获得性能最佳的版本，而无需更新其应用程序。

Before we take a closer look at the implementation, it’s important to note that there are many other tools that let you perform A/B tests on Android applications. Firebase is a good choice as it’s free, easy to implement, and you most likely will already be using some other of its tools (like the Analytics, Crashlytics, etc…). I did however use another analytics tool to perform deeper analysis on the performance of the A/B test, but I’ll get back to this later.

在我们仔细研究实现之前，需要注意的是，还有许多其他工具可让您在Android应用程序上执行A / B测试。 Firebase是一个不错的选择，因为它是免费的，易于实现的，而且您很可能已经在使用其他工具 (例如Analytics(分析)，Crashlytics等)。 但是，我确实使用了另一个分析工具来对A / B测试的性能进行更深入的分析，但稍后我会再讲到这一点。

Show me the code!

给我看看代码！

在应用程序中设置远程配置。 (Setting up Remote Config in the app.)

In case Firebase isn’t set up in your project at all, you’ll want to follow this guide, which will take through the steps of creating your Firebase project, registering your app, and adding your Firebase Configuration file.

如果您的项目中根本没有设置Firebase，则需要遵循 本指南将逐步完成创建Firebase项目，注册应用程序以及添加Firebase配置文件的步骤。

Once Firebase is set up, you’re ready to use Remote Config.

设置Firebase后，就可以使用远程配置了。

First, you’ll want to add the dependency:

首先，您需要添加依赖项：

implementation ‘com.google.firebase:firebase-config-ktx:19.2.0’ implementation ‘com.google.firebase:firebase-analytics-ktx:17.4.4’

The analytics dependency is needed in order to be able to use it when analyzing the results of your test. In addition, you may want to use some analytics event as a trigger for your A/B test. Using the ktx library will allow you to use the Kotlin extensions of the Firebase library.

需要分析依赖性，以便能够在分析测试结果时使用它。 此外，您可能希望使用一些分析事件作为A / B测试的触发器。 使用ktx库将使您可以使用Firebase库的Kotlin扩展。

You will need to set up in-app default values. Those values will be used in case your app can’t connect to the Firebase backend, or if the parameter you’re querying is not set in the backend. For this, you’ll need to create an XML file. Go to your res folder, and create an xml folder if you do not have one already. Create a new XML file in this folder (named remote_config_values for example). Here’s what the content of the file should look like:

您将需要设置应用内默认值 。 这些值将用于您的应用无法连接到Firebase后端或后端未设置您要查询的参数的情况。 为此，您需要创建一个XML文件。 转到您的res文件夹，如果尚未创建一个xml文件夹。 在此文件夹中创建一个新的XML文件(例如，名为remote_config_values )。 文件内容如下所示：

You will pass this file to your RemoteConfig object with the method setDefaultAsync. You’ll use Firebase.remoteConfig to retrieve your Remote Config instance. This instance will be used for configuring Remote Config in your app and fetch the values set in the Console.

您将使用setDefaultAsync方法将此文件传递给RemoteConfig对象。 您将使用Firebase.remoteConfig检索远程配置实例。 该实例将用于在您的应用中配置远程配置，并获取在控制台中设置的值。

The first thing you'll want to do is to set the minimal fetch interval, which is the minimum amount of time you want to wait before fetching new values from the backend. The default value is 12 hours, which means that the next time you'll try to fetch new values, the same values will be returned if the previous time was less than 12 hours ago. Otherwise, the latest values in the console will be retrieved.

您要做的第一件事是设置最小获取间隔 ，这是从后端获取新值之前要等待的最短时间。 默认值为12 hours ，这意味着下次您尝试获取新值时，如果前一个时间少于12小时，则将返回相同的值。 否则，将检索控制台中的最新值。

There is a server-side quota on how many queries your app can do per hour. The Firebase team recommends against choosing a small interval value, as this could potentially make your app hit the quota. However, when developing your app, you probably won't want to wait for hours before being able to change your values and test them. Therefore, when working on a development build, you can choose a much lower value, 10 seconds for example. In my app, I have a dev flavor and a prod flavor, depending on who I'm building the app for. I'm using this flavor to decide which minimal fetch interval to set.

服务器端配额是您的应用每小时可以执行的查询数量。 Firebase团队建议不要选择较小的时间间隔值，因为这可能会使您的应用达到配额。 但是，在开发应用程序时，您可能不想等待几个小时才能更改您的值并对其进行测试。 因此，在进行开发构建时，您可以选择一个低得多的值，例如10秒 。 在我的应用程序中，我有一个开发者风格和一个产品风格，这取决于我为谁开发应用程序。 我正在使用这种方式来确定要设置的最小获取间隔。

Here’s what we discussed so far will look like in your app:

到目前为止，我们在您的应用程序中讨论的内容如下：

Here, I’ve created an interface called RemoteConfigManager, which will contain the method used to retrieve the value from the backend. I’m implementing this interface in RemoteConfigManagerImpl (with no method so far), which is class I’ll be accessing in my app whenever I want to get a value from the backend.

在这里，我创建了一个名为RemoteConfigManager的接口，该接口将包含用于从后端检索值的方法。 我正在RemoteConfigManagerImpl中实现此接口(到目前为止尚无方法)，这是我想从后端获取值时将在我的应用程序中访问的类。

With our RemoteConfig instance configured, we can now get a parameter from the backend! If you do not yet have parameters in your Firebase Console, head over to the Console and to Remote Config. You’ll be asked to add a new parameter, with a key and a value. In my example above, the key was my_parameter_key. In the default config file, I’ve put the value default_value, but now I want to put a real one to check we properly get the value from the backend. Let’s put remote_config_value. You can also add a description, which is a good way to keep track of the different parameters you’ll add.

配置了RemoteConfig实例后，我们现在可以从后端获取参数了！ 如果您的Firebase控制台中还没有参数，请转到控制台和“ 远程配置”。 系统会要求您添加一个新参数，包括一个键和一个值。 在上面的示例中，键是my_parameter_key 。 在默认的配置文件中，我已经输入了default_value值，但是现在我想输入一个真实的值来检查我们是否从后端正确获取了该值。 让我们放置remote_config_value。 您还可以添加描述，这是跟踪要添加的不同参数的好方法。

When you’re done adding values here, you need to click on Publish Changes in order to validate everything and make them accessible in your app.

在此处完成添加值后，您需要单击“ 发布更改”以验证所有内容并使它们在您的应用中可访问。

Let’s go back to the code now, and try to get the value in the app.

现在让我们回到代码，并尝试在应用程序中获取值。

You’ll need to fetch and activate the values with the Remote Config object. You can use the method fetchAndActivate() which will asynchronously perform those operations and returns a Task with the result, to which you can add listeners, and finally retrieve your value.

您需要使用Remote Config对象获取并激活值。 您可以使用方法fetchAndActivate()来异步执行这些操作，并返回带有结果的Task，您可以在其中添加侦听器，最后检索您的值。

Because this is done asynchronously, you may want to take a moment and think of the best user experience to give your users. For example, you could show a loading screen while the value is fetched if the value drastically has an impact on the way your UI looks.

由于此操作是异步完成的，因此您可能需要花一点时间来考虑为用户提供最佳的用户体验。 例如，如果该值严重影响您的UI外观，则可以在获取该值时显示加载屏幕。

在Firebase控制台中设置A / B测试 (Setting up the A/B test in the Firebase Console)

Now that you have Remote Config enabled in your app, and you know how to retrieve values, you can start an experiment on Firebase. Head over to A/B Testing in the Console, and click on Create an experiment. Select a Remote Config experiment.

现在，您已在应用程序中启用了远程配置，并且知道如何检索值，您可以在Firebase上开始实验。 转到控制台中的A / B测试 ，然后单击创建实验。 选择一个远程配置实验。

Enter the details of your test (name, etc…), and move to the Targeting section, where you have the choice to add an Activation event. If you set up one, it means that only the users that have performed this event will enter your experiment. All the other users will not be taken into account in your results.

输入测试的详细信息(名称等)，然后移至“ 目标”部分，您可以在其中选择添加激活事件 。 如果您设置了一个，则意味着只有执行此事件的用户才能进入实验 。 您的结果中不会考虑所有其他用户。

Here’s an example of why this can be important: Imagine you’re running a test on a social media app that has a registration and then features like post a photo. You want to improve the registration with an experiment. The main metric you will look at could be the increase in the percentage of new users completing the registration, but you also want to have a “big picture” metric like the percentage of users posting a photo.

这是为什么这很重要的示例：假设您正在具有注册功能的社交媒体应用上运行测试，然后发布照片等功能。 您想通过实验来改善注册。 您将要查看的主要指标可能是完成注册的新用户所占百分比的增加，但是您还希望拥有一个“大图片”指标，例如发布照片的用户所占的百分比。

If you do not use an activation event such as “start registration”, all of your users, including those that have registered in the past, will be in your test, and their photo posts will be in your results. But, what you really want, is only the posts from the new users completing the registration now, as they are the ones that will be affected by your experiment. I encourage you to take a moment and think about the experiment you have in mind, and whether you should add an activation event.

如果您不使用“开始注册”之类的激活事件，则所有用户(包括过去注册的用户)都将参加测试，而其照片帖子将出现在您的结果中。 但是，您真正想要的只是新用户现在完成注册的帖子，因为它们将受到您的实验的影响。 我鼓励您花一点时间考虑一下您要进行的实验，以及是否应该添加激活事件。

Next, in the Goals section, you’ll be asked for a success metric, such as performing another event, or a retention improvement… As I was mentioning at the beginning of the article, I personally think that the analysis part of an A/B test on Firebase has limitations, and therefore I’m performing deeper analysis with another Analytics tool, which is Amplitude. It’s a very powerful tool, that offers a free plan up to a certain amount of monthly events, so I encourage you to check it out. In my example of the registration flow experiment, one thing I would like to know for example (and that Firebase can’t tell me) is the completion of each step during the experiment.

接下来，在“ 目标”部分中，系统会要求您提供成功指标，例如执行其他事件或保留率的提高……正如我在本文开头提到的那样，我个人认为A / Firebase上的B测试具有局限性，因此我正在使用另一个分析工具Amplitude进行更深入的分析。 这是一个非常强大的工具，它提供了免费的计划，每月最多可以进行一定数量的活动，因此，我建议您检查一下 。 在我的注册流程实验示例中，我想知道的一件事(并且Firebase不能告诉我)是实验过程中每个步骤的完成。

However, if you have simple needs in terms of analytics (ex: clicking on a button or not, buying something or not, an amount of money spent, a retention rate…) then you could only be using Firebase.

但是，如果您在分析方面有简单的需求(例如：是否单击按钮，是否购买东西，花费的金额，保留率...)，则只能使用Firebase。

Finally, in the Variants section, you get to link your experiment to the Remote Config that you set up earlier. You should be seeing the parameter you have created before, but you can also create a new one then. You can now define several variants of your parameter and split your users into a group for each.

最后，在“ 变体”部分中，您可以将实验链接到之前设置的“远程配置”。 您应该会看到之前创建的参数，但是您也可以随后创建一个新参数。 现在，您可以定义参数的多个变体，并将每个用户分成一组。

The Control group should be the same version of the app you currently have. You could name the value control for example. Then you can create a new Variant for each version you want to test.

控制组应与您当前拥有的应用程序版本相同。 例如，您可以命名值控件 。 然后，您可以为要测试的每个版本创建一个新的变体。

You can now hit Review and create your Draft of an experiment. Once it is in draft, you can check that your experiment is behaving as expected using the test device feature, which can force a specific variant to your device. You can read more about this here:

现在，您可以点击“ 查看”并创建实验草案。 完成草稿后，您可以使用测试设备功能检查实验是否按预期进行，这可以强制您的设备使用特定的版本。 您可以在此处了解更多信息：

Back in your app, you can now implement all your different variants, and display them to your users based on the value returned by Remote Config.

回到您的应用程序中，您现在可以实现所有不同的变体，并根据Remote Config返回的值将其显示给用户。

结论…… (To conclude…)

I hope that this article will have helped you get started running A/B Test in your Android app. Regardless of the size of your team, it is worth spending time and resources on an A/B Test culture as it is a very valuable tool to use in order to understand better how your users behave in your app, and how to improve their experience to the best it can be.

希望本文能帮助您开始在Android应用中运行A / B测试。 无论您的团队规模如何，都值得在A / B测试文化上花费时间和资源，因为它是一种非常有价值的工具，可以用来更好地了解您的用户在您的应用程序中的行为方式以及如何改善他们的体验尽其所能。

I’d love to hear your thoughts, comments, or suggestions in the comment section! What type of experiment have you been running?

我希望在评论部分听听您的想法，评论或建议！ 您正在进行哪种类型的实验？

Thanks for reading, happy coding!

感谢您阅读，编码愉快！

翻译自: https://medium.com/swlh/running-an-a-b-test-in-your-android-app-with-firebase-2e830a4e40ad

firebase a/b