背景介绍

MOSN 在 Service Mesh 领域作为东西向 RPC 服务治理网络转发组件已经在社区很多公司都得到了一定实践，为了进一步节省研发运维等相关成本，目前业界也有很多公司在考虑把东西向和南北向数据转发面统一。MOSN 在作为东西 Sidecar 场景当前性能足以，但在作为南北向网关会有一定性能上限（比如单机百万级长连接、高并发等场景）。

解决方案（草案）

为 MOSN 在 Network 层做一个高性能的网络扩展层，使其可以和高性能网络库结合使用（如Nginx、Envoy、OpenResty 等），使其底层网络具备高性能处理的同时，又能复用 MOSN Stream Filter 等能力。

如上图所示，MOSN 通过扩展一个标准的 GOHPNF(Golang On High Performance Network Framework) 适配层和底层的高性能网络组件（如 Nginx、Envoy、Openresty 等）生态打通。使得其底层网络具备 C 同等的处理能力的同时，又能使上层业务可高效复用 MOSN 的处理能力及 GoLang 的高开发效率。

注：MOSN 和 高性能网络库是运行在一个进程空间的，高性能网络库（如 Envoy、Nginx）是通过使用 CGO 的方式来驱动 MOSN 的代码运行的，不存在跨进程通信和数据 COPY。

相关问题

• Q0 MOSN 为什么要支持底层网络库的扩展能力？

• A0 一方面是使得 MOSN 具备作为南北向数据面时需要的极致性能，另一方面可以让 MOSN 和其他高性能网络库生态（如 Nginx、Envoy、OpenResty 等）有更多的结合，在实现高性能的同时又具备了 GoLang 的开发效率。

• Q1 高性能底层网络组件是 C/C++ 实现的，后续排查问是否会困难？

• A1 底层网络扩展模块会做成一个轻量级的（成熟后基本上不用修改），另外也会完善其周边相关 Debug 调试工具。

• Q2 MOSN 在支持高性能底层网络扩展后，其 MOSN 的开发流程是否有变化？

• A2 没有变化，和之前 MOSN Stream Filter 开发流程一样。另外 MOSN 底层高性能网络扩展是可选择，用户可以根据自身业务场景选择是否开启。

• Q3 目前计划是准备使用那些高性能网络库？

• A3 还没明确，会先抽象一个 GOHPNF （GoLang On High Performance Network Framework） 适配层，然后只需在相关高性能网络库调用 GOHPNF 提供的 SDK 就可以和 MOSN 结合了。

现在，我们已经支持把 Envoy 作为一个 MOSN 的七层网络扩展，欢迎大家试用。这是相关使用文档。

动态路由事例

如下事例是通过把 MOSN 的网络扩展层设置为 Envoy 后，通过在 MOSN 侧使用 GoLang 语言开发一个 stream filter 就可以实现 Envoy 的动态路由能力：
根据用户请求 header 的 uid 字段做路由，比如将 uid 范围在 [1, 100] 的请求路由到应用的 s1 站点，将 uid 范围在 [101, 200] 的请求路由到 s2 站点，将其它范围 uid 请求路由到 s3 站点, 如果没有 uid 则将轮训转发到 s1、s2、s3。

步骤1（安装 mosn network on envoy 镜像）

docker pull mosnio/mosn-network-on-envoy:v0.20.0

步骤2（启动 mosn network on envoy 容器）

docker run -it --rm --net=host mosnio/mosn-network-on-envoy:v0.20.0

当容器正常启动后，就可以使用 curl 命令发起对应的请求，然后就可以看到上述事例路由规则的工作效果：

$curl localhost:2990 -H "uid:2"
site s1 from 3450

$curl localhost:2990 -H "uid:2"
site s1 from 3451

$curl localhost:2990 -H "uid:200"
site s2 from 3452

$curl localhost:2990 -H "uid:201"
site s3 from 3453

欢迎大家讨论和共建 ！

Background introduction

MOSN has been adopted by many companies in the community as their East-West RPC service networks data plane. To further save R&D, as well as reducing operation and maintenance overheads, many companies are considering converging their East-West and North-South data plane into one production. Currently, MOSN's performance is sufficient for East-West Sidecar scenario, yet insufficient for the North-South gateway scenario (e.g., C1000K, high concurrency, etc.).

Solution(draft)

We propose a high-performance network extension machism for MOSN, so that it can be used to combine with other high-performance network libraries (e.g. Nginx, Envoy, OpenResty, etc.). So that the network has high-performance processing and can reuse MOSN Stream Filter and other features.

As shown in the above figure, MOSN uses a standard GOHPNF (Golang On High Performance Network Framework) shim layer to connect with the underlaying high-performance network components (such as Nginx, Envoy, Openresty, etc) ecosystem. This proposal allows MOSN to increase network processing performance, while reserving Golang's high development efficiency.

Note: MOSN and the High Performance Network library run in the same process. The High Performance Network library (e.g. Envoy, Nginx) runs by calling MOSN code using CGO, and there is no cross-process communication or data COPY.

Q&A

• Q0: Why does MOSN need to support the extension of its network library?
• A0: On the one hand, it enables MOSN to obtain network performance required by a North-South data plane. On the other hand, it allows MOSN to have the ability to integrate with high-performance network library ecosystems (such as Nginx, Envoy, OpenResty, etc.) while reserving the development efficiency of GoLang.
• Q1: Is it difficult to troubleshoot since the high-performance network plugin are implemented in C/C++?
• A1: The network extension module will be made into a lightweight one (basically there is no need to modify after maturity), and the debugging tools will be improved as well.
• Q2: After MOSN supportting the plugin of high-performance network, is there any change in its MOSN development process?
• A2: No, just like the previous MOSN stream filter development process. In addition, MOSN's high-performance network extension is alternative. Users can choose their prefered network extension according to their own business scenarios.
• Q3: Which network libraries are you planning to use?
• A3: TBD, it will first abstract a GOHPNF (GoLang On High Performance Network Framework) shim layer, and then only need to call the SDK provided by GOHPNF in the high-performance network library to with MOSN.

Now, We have been supported Envoy as a Layer 7 network extension for MOSN, welcome to try it. Here is the documentation about it.

Quic start dynamic route sample

In this sample, after set Envoy as a network extension for MOSN , then Envoy's dynamic routing feature can be implemented by developing a stream filter on the MOSN side using the GoLang language.

For example, requests with uid range [1, 100] will be routed to test app site s1, requests with uid range [101, 200] will be routed to site s2, requests with other uid ranges will be routed to site s3, and if there is no uid header, the request will be round robin to s1, s2, and s3.

Step 1(Install mosn network on envoy image)

docker pull mosnio/mosn-network-on-envoy:v0.20.0

Step 2(Start mosn network on envoy container)

docker run -it --rm --net=host mosnio/mosn-network-on-envoy:v0.20.0

Then we could see the route rules at work by making a request via curl in another terminal:

$curl localhost:2990 -H "uid:2"
site s1 from 3450

$curl localhost:2990 -H "uid:2"
site s1 from 3451

$curl localhost:2990 -H "uid:200"
site s2 from 3452

$curl localhost:2990 -H "uid:201"
site s3 from 3453

Welcome to discuss and build together!