lighthouse使用
Lighthouse is a service to scan webpages and see how good they score on terms of SEO, performance and best practices. You can use the scanner here: https://lighthouse.admantium.com/.
Lighthouse是一项服务,可扫描网页并查看其在SEO,性能和最佳实践方面的得分。 您可以在这里使用扫描仪: https : //lighthouse.admantium.com/ 。
Currently lighthouse consists of three microservices: A frontend that delivers static HTML, an API to request new scans and query job status, and several workers that perform the actual web page scan. During refactoring the application into true microservices, I recognized that scaling the app is about scaling the workers. And as for now, I just provide a fixed number of workers, and each can only process up to 3 scans at the same time.
当前,灯塔由三个微服务组成:一个提供静态HTML的前端,一个用于请求新扫描和查询作业状态的API,以及一些执行实际网页扫描的工作程序。 在将应用程序重构为真正的微服务期间,我认识到扩展应用程序就是扩展工作人员。 就目前而言,我只提供固定数量的工作人员,每个工作人员最多只能同时处理3个扫描。
In this article, I explore and implement the concept of a serverless function. In a nutshell, serverless means that you implement and deploy a function with a standardized HTTP interface somewhere in the cloud, and the framework takes care of availability and scaling of this particular function. There are many serverless platform or providers available. The platform of my choice is OpenFaas. I will use OpenFaaS because it runs natively on Kubernetes, can be deployed easily, does not restrict the used programming language and uses (customizable) Docker images for function execution.
在本文中,我探索并实现了无服务器功能的概念。 简而言之,无服务器意味着您可以在云中的某个地方使用标准化的HTTP接口来实现和部署功能,并且该框架负责此特定功能的可用性和扩展。 有许多无服务器平台或提供程序可用。 我选择的平台是OpenFaas 。 我将使用OpenFaaS,因为它在Kubernetes上本地运行,可以轻松部署,不限制所使用的编程语言,并且使用(可自定义的)Docker映像执行功能。
In this article, I will start with a short introduction to OpenFaaS. I will only cover core concepts, read my OpenFaas Deep Dive to get even more familiar. Then I will explain how the current worker node in my lighthouse functions. And then we will modify this microservice step-by-step to a serverless function.
在本文中,我将简要介绍OpenFaaS。 我将只介绍核心概念,请阅读我的OpenFaas Deep Dive以更加熟悉。 然后,我将解释灯塔中当前的工作节点如何工作。 然后,我们将逐步将此微服务修改为无服务器功能。
This article originally appeared at my blog.
本文最初出现在 我的博客中 。
OpenFaaS核心概念 (OpenFaaS Core Concepts)
OpenFaaS provides a powerful command line utility to build, push, and deploy serverless applications. Users start by choosing a template for their functions. Templates consist of a special Docker image with a built-in health check functions as well as programming language specific software packages. When you choose a template, you can either get a base image, for example the NodeJS template, or a specialized template for an HTTP framework in this language, for example Express. The difference: The base image will just expose one function, and will just return HTTP status codes 200 and 400. The specialized framework allows you full HTTP methods, different endpoints and status codes.
OpenFaaS提供了一个功能强大的命令行实用程序,用于构建,推送和部署无服务器应用程序。 用户首先选择功能模板。 模板包括一个特殊的Docker映像,该映像具有内置的运行状况检查功能以及特定于编程语言的软件包。 选择模板时,既可以获取基本映像(例如NodeJS模板),也可以获取使用该语言的HTTP框架专用模板(例如Express)。 区别:基本映像仅公开一个功能,并且仅返回HTTP状态代码200和400。专用框架允许您使用完整的HTTP方法,不同的端点和状态代码。
When you choose a template, three artifacts will be generated for you. First, a project folder with a handler file in which you put your function code. Second, for the template you a choose, template folder with the Dockerfile and language-specific file. And third, a configuration file that links your function to the template.
选择模板时,将为您生成三个工件。 首先,一个带有处理程序文件的项目文件夹,您在其中放置了功能代码。 其次,为模板选择一个包含Dockerfile和特定于语言的文件的模板文件夹。 第三,一个配置文件,将您的功能链接到模板。
Once the template is initialized, you simply add function code to the handler file and then use the OpenFaaS cli to build, push and deploy your function. Your function can be viewed in an UI. In the UI, you can invoke your function, and see usage statics etc.
模板初始化后,您只需将功能代码添加到处理程序文件中,然后使用OpenFaaS cli来构建,推送和部署功能。 可以在UI中查看您的功能。 在用户界面中,您可以调用函数,并查看使用情况静态信息等。
![Image for post](https://i-blog.csdnimg.cn/blog_migrate/a6b2dbbbb12cacfa0c944dbb13b000fb.png)
工人微服务 (Worker Microservice)
The worker microservice is developed with NodeJS. It is based on Hapi, a framework that provides similar abstractions than Express. Hapi comes with a rich set of easy to apply plugins for query parameter parsing, validity checks, logging and even automatic health check endpoints. You can read more about Hapi in my earlier article.
工作者微服务是使用NodeJS开发的。 它基于Hapi ,该框架提供与Express类似的抽象。 Hapi带有丰富的易于应用的插件集,用于查询参数解析,有效性检查,日志记录甚至自动运行状况检查端点。 您可以在我之前的文章中了解有关Hapi的更多信息。
The microservice provides just one endpoint: GET /api/scan
. Requests are passed to scanner.run
. The scan function returns a HTTP status code and a message that is returned to the caller. Here is a short excerpt of the HTTP endpoint:
微服务仅提供一个端点: GET /api/scan
。 请求被传递到scanner.run
。 扫描功能返回HTTP状态代码和返回给调用方的消息。 这是HTTP端点的简短摘录:
const hapi = require('@hapi/hapi');async function init() {
const server = hapi.server({
port: 8080,
host: "0.0.0.0",
}); server.route({
method: 'GET',
path: '/api/scan',
},
handler: async (request, h) => {
request.log('REQUEST /scan'); const scan = await scanner.run(request.query.url, server); return h.response(scan.msg).header('Access-Control-Allow-Origin', '*').code(scan.code);
}
}); // 40 lines of HAPI configuration code await server.start();
server.log('info', { msg: 'BOOTING lighthouse-scanner v0.2.1' });
}init();
The scanner.run
function provide the core logic. First, it keeps track of the number of concurrent scans. At any time, a maximum number of 3 scans is allowed. If more are requested, an error will be thrown. Second, it converts the URL to a domain name, and executes a ping
check to see if this domain exits. Third, if all is well, a lighthouse scan is executed and the result returned. Here is again an excerpt of the function.
scanner.run
函数提供了核心逻辑。 首先,它跟踪并发扫描的数量。 在任何时候,最多允许3次扫描。 如果请求更多,将引发错误。 其次,它将URL转换为域名,并执行ping
检查以查看该域是否退出。 第三,如果一切顺利,将执行灯塔扫描并返回结果。 这又是该功能的摘录。
const max_jobs = 3;
let counter = 0;function response(msg, code, uuid) {
return { msg, code, uuid };
}exports.scanner = {
run: async (url) => {
if (!(counter < max_jobs)) {
return response('Scanner request denied, all workers exhausted, please try again', 429, '');
} const domain = url.replace(/http(s?):\/\//, ''); try {
ping(domain);
} catch (e) {
if (e.failed && e.stderr !== '') {
const msg = e.stderr ? e.stderr : 'could not resolve address';
return response(msg, 404, '');
}
} try {
const result = await scan(`http://${domain}`));
return response(result, 200);
} catch (e) {
const errDetails = { stderr: e.stderr, reason: e.shortMessage };
return response(errDetails, 400);
}
}
};
工作者无服务器功能 (Worker Serverless Function)
Now we will rewrite this microservice as a serverless function.
现在,我们将该微服务重写为无服务器功能。
The first step is to select a suitable template. I like to work with complete HTTP methods and status codes, so we will use a template based on the express framework. Download the template with faas-cli template store pull node10-express-service
.
第一步是选择合适的模板。 我喜欢使用完整的HTTP方法和状态代码,因此我们将使用基于express框架的模板。 使用faas-cli template store pull node10-express-service
下载模板。
Second, we initialize a new project. When we execute faas-cli new --lang node10-express-service lighthouse
, the resulting files are:
其次,我们初始化一个新项目。 当我们执行faas-cli new --lang node10-express-service lighthouse
,结果文件为:
lighthouse.yml
lighthouse
├── handler.js
└── package.json
To get our serverless function running, we need to do four steps.
要使无服务器功能运行,我们需要执行四个步骤。
步骤1:添加功能代码和库 (Step 1: Add Function Code and Libraries)
This is a rather obvious step. You need to take your core function and provide a small HTTP wrapper around it. The complete microservice core logic in lighthouse could be reduced to an almost 1:1 copy of the scanner.run
function and 9 lines of additional code for the endpoint. The final result looks like this.
这是一个相当明显的步骤。 您需要使用核心功能,并在其周围提供一个小型HTTP包装器。 灯塔中完整的微服务核心逻辑可以简化为scanner.run
函数的几乎1:1副本以及该端点的9行附加代码。 最终结果如下所示。
"use strict"
async function scan(req) {
//Same as scanner.run
}module.exports = async (config) => {
const app = config.app; app.get('/scan', async (req, res) => {
const r = await scan(req);
res.status(r.code);
res.send(r.msg);
})
}
This was surprising. All the HAPI configuration code is not needed, and the simplicity of defining HTTP endpoints based on Express speaks for itself.
这真是令人惊讶。 不需要所有的HAPI配置代码,并且基于Express定义HTTP端点的简单性就说明了这一点。
If your function needs additional libraries, you can add them to the provided package manager. For NodeJS, that is the package.json
file, and you add them as usual with npm install
.
如果您的函数需要其他库,则可以将它们添加到提供的包管理器中。 对于NodeJS,这是package.json
文件,您可以使用npm install
照常添加它们。
步骤2:自定义Dockerfile(可选) (Step 2: Customize Dockerfile (optional))
Your templates Dockerfile resides at templates/TEMPLATE_NAME/Dockerfile
. Take a look at it, and add any additional software package you might need. In my case, that is the iputils
package to execute ping
system calls.
您的模板Dockerfile驻留在templates/TEMPLATE_NAME/Dockerfile
。 看一看,然后添加您可能需要的任何其他软件包。 就我而言,这是执行ping
系统调用的iputils
包。
FROM node:12.13.0-alpine as ship# Additional tools
RUN apk update
RUN apk --no-cache add ca-certificates iputils# ...
步骤3:建立并推送您的图片 (Step 3: Build and Push your Image)
In this step, you use the OpenFaaS command line utility again. An important prerequisite: You need to have access to a Docker registry. I prefer to use a private registry, but you can also use Dockerhub. Read the OpenFaaS Documentation about setup a private Docker registry.
在此步骤中,您将再次使用OpenFaaS命令行实用程序。 一个重要的前提条件:您需要访问Docker注册表。 我更喜欢使用私有注册表,但是您也可以使用Dockerhub。 阅读有关设置私有Docker注册表的OpenFaaS文档。
First, configure registry and image name in the OpenFaaS config file. Here is the content of lighthouse.yml
.
首先,在OpenFaaS配置文件中配置注册表和映像名称。 这是lighthouse.yml
的内容。
version: 1.0
provider:
name: openfaas
gateway: https://functions.admantium.com
functions:
lighthouse:
lang: node10-express-service
handler: ./lighthouse
image: docker.admantium.com/lighthouse-openfaas:latest
Then, execute faas-cli build -f lighthouse.yml
and fs push -f lighthouse.yml
.
然后,执行faas-cli build -f lighthouse.yml
和fs push -f lighthouse.yml
。
步骤4:部署 (Step 4: Deploy)
The final step is a simple command: faas-cli deploy -f lighthouse.yml
. When you use a private registry, you need to add the name of the registry secret as follows: faas-cli deploy -f lighthouse.yml --secret dockerfaas
.
最后一步是一个简单的命令: faas-cli deploy -f lighthouse.yml
。 使用私有注册表时,需要按如下所示添加注册表密钥的名称: faas-cli deploy -f lighthouse.yml --secret dockerfaas
。
Head over to the UI to see your function.
转到UI以查看您的功能。
![Image for post](https://i-blog.csdnimg.cn/blog_migrate/1c999f24dcb7415336a0aa9377942446.png)
All done — your microservice is now serverless, and OpenFaaS takes care about your functions availability.
全部完成-您的微服务现在无服务器,并且OpenFaaS会关注您的功能可用性。
结论 (Conclusion)
With OpenFaaS, you can create lean serverless functions that are accessible with standardized HTTP endpoints. Supported by a comprehensive and complete command line utility, you select a template, initialize a project, build, push and deploy your function. OpenFaaS is about choice: A huge number of programming languages and frameworks are supported, and the Dockerfile that runs your function can be customized.
使用OpenFaaS,您可以创建精简的无服务器功能,这些功能可以通过标准化的HTTP端点进行访问。 在全面而完整的命令行实用程序的支持下,您可以选择模板,初始化项目,构建,推送和部署功能。 OpenFaaS可供选择:支持大量的编程语言和框架,并且可以自定义运行您的函数的Dockerfile。
In this article, I showed the essential steps to rewrite a NodeJS microservice to a serverless function. In essence, you can copy your existing function, and add a lean HTTP handler around it. Then you optionally customize the Dockerfile. And finally, you are using the OpenFaaS command line utilities for building, pushing and deploying the application.
在本文中,我展示了将NodeJS微服务重写为无服务器功能的基本步骤。 本质上,您可以复制现有函数,并在其周围添加精益HTTP处理程序。 然后,您可以选择自定义Dockerfile。 最后,您将使用OpenFaaS命令行实用程序来构建,推送和部署应用程序。
This was my first practical application of a serverless function. As a developer, I like the simplicity of creating and deploying a function. However, I spend a lot of time learning about the templates and learning how to redesign my function with OpenFaaS. Part of this difficulty is actually an OpenFaaS strength: The many options you have, including customizing the Docker image. There are many concepts of OpenFaaS that I did not use yet: Event driven communication between functions, and particular auto scaling. I’m looking forward to explore these concepts as well.
这是我第一次无服务器功能的实际应用。 作为开发人员,我喜欢创建和部署功能的简单性。 但是,我花了很多时间来学习模板并学习如何使用OpenFaaS重新设计我的功能。 这个难题的一部分实际上是OpenFaaS的优势:您拥有许多选择,包括自定义Docker映像。 我尚未使用过许多OpenFaaS概念:函数之间的事件驱动通信,以及特定的自动缩放。 我也期待探索这些概念。
lighthouse使用