Community tasks

As a programming enthusiast , have you ever felt that you want to participate in the development of an open source project, but don't know where to start?
In order to help everyone better participate in open source projects, the MOSN community will regularly publish community tasks to help everyone learn by doing!

Task List

Tasks of different difficulty are released:

Easy

Layotto runtime

• Reduce the risk of panic. Add recover for all codes that create new goroutines. See Reduce the risk of panic #197
• Fail fast,make Layotto kill itself when error accurs during startup. See fail fast and kill the process when error occurs during startup #275

Bug fix

• fix bug of actuator: apollo health status "INIT",even if no apollo component configurated. see bug(actuator): apollo health status "INIT",even if no apollo component configurated #462
• 修复 rpc 相关代码的单元测试报错问题
  🤔 UT of mosn.io/layotto/components/rpc/invoker/mosn/channel failed #550

Write a Feature

• Use in-memory component to implement the Actuator API. See Use in-memory component to implement the Actuator API #463

Comment related

• Add comment to exported function/method/interface/variable (such as RPC/distributed lock modules), see add comment to exported function/method/interface/variable #112
  Considering that the workload of adding comments to all the modules is relatively large, we split it into multiple tasks:
• add comments to RPC related code
• add comments to pubsub related code
• add comments to lock related code . assigned
• add comments to state API related code
• add comments to actuator related code
• add comments to tcpdump related code
• add comments to WASM related code
• add comments to flow-control related code
• Add comments for file API related code in the proto file.
  

SDK related

• add metadata field for go,js and dotnet sdk. See add metadata field for js sdk #320
• add distributed lock API for java sdk
• add distributed sequncer API for java sdk
• add nodejs sdk. assigned ,see Add nodejs SDK #258
• add python sdk
• add c/c++ sdk
• add .net sdk
  assigned.see Add .net sdk #130
• add more sdk api into java sdk(currently there are only protobuf files,but some users ask us to add more feature in sdk)
  assigned. see simple java client #153
• compile proto files into java code as java sdk. See gengrate other language pb files #79 Already done by @MentosL

Document related

• add go-sdk usage documents.See http://mosn.io/layotto/#/en/sdk_reference/go/start

• add document for State API.See https://mosn.io/layotto/#/en/api_reference/state/reference

• add document for Pub/Sub API.See https://mosn.io/layotto/#/en/api_reference/pubsub/reference

• add document for Distributed Lock API.See https://mosn.io/layotto/#/en/api_reference/lock/reference

• add document for RPC API.See https://mosn.io/layotto/#/en/api_reference/rpc/reference already assigned Add document for RPC API. #185

• Change MinIO's demo to use docker to deploy MinIO by default, and Layotto to invoke local MinIO.
  Currently, the demo uses the MinIO official demo service by default, but the service is unstable, so we want to use docker to deploy MinIO and make the demo more stable.
  See doc(quickstart): use local minIO instead of remote minIO service #450

• Delete redundent demos. See https://github.com/mosn/layotto/issues

Translation

• Translate Skywalking quickstart documentation into English

• Translate How to deploy and upgrade Layotto into English

• Translate "Explain Layotto's github workflow" into Chinese

• 将“介绍 layotto CI” 的文档翻译成英文
  translate: docs/en/development/github-workflows.md to English #545

• 将“介绍如何自动测试 quickstart” 的文档翻译成英文
  https://mosn.io/layotto/#/zh/development/test-quickstart

Tests related

• Add unit tests for actuator module. The code is under pkg/filter/stream/actuator/http
  

• Understand the implementation of wasm module & add unit tests: see Add unit tests for WASM related code; 为wasm模块补充单测 #105

• add more unit test whereever you like to make layotto's unit test coverage higher(currently it's only 46%)

• Add unit tests for runtime/runtime.go and grpc/api.go. These two files are the core engine code of layotto, responsible for component lifecycle management and grpc request processing respectively
  Already assigned to @tianjipeng See Add unit tests for runtime/runtime.go and grpc/api.go #138

• Add integration test cases for a certain type of API. See Add integration test cases #107 assigned to @seeflood

• Understand the implementation of the rpc module & add unit tests: see Add unit tests for RPC related code #106 Already done by @tianjipeng

• Add integrate test. see [GLCC] Add more components and api features in integrate test #415

Components

• Change the apollo sdk to https://github.com/apolloconfig/agollo .Currently we are using github.com/zouyx/agollo

in-memory components

See #67 (comment)
We want to let users and sdk developers run our demo without go , docker and back-end storage (e.g. redis) pre-installed.
To achieve this goal, we need to add in-memory components,including:

• pubsub
• lock
• sequencer
• configuration API

You can take the in-memory state store component as an example,see #327

Medium

Add support for Dapr API

We want Layotto to support both Layotto API and Dapr API. In this way, if users are worried about vendor lock-in, they can use Dapr SDK to switch between Layotto and Dapr freely.
You can refer to:
#361
#362

• InvokeService
• InvokeBinding
• State
• pubsub. See feature： add Dapr pubsub API #406
• Configuration
• Secret
• GetMetadata/SetMetadata
• Shutdown

SDK

• Improve .net sdk. Make it have the same capabilities and api as go sdk
  assigned.see Add .net sdk #130

• Develop python sdk
  Because we want to reduce the cost of maintaining multi-language SDKs, we want to reuse Dapr SDKs as much as possible. Therefore, this task suggests to fork the Dapr python SDKs for modification (because the package path of the proto interface is different, so the code compiled by proto is different, so we need to do some modification).

• Develop the spring-boot-layotto package. See feat: layotto springboot layotto/java-sdk#8
  Let layotto integrate spring boot so that users can use annotations to register pubsub subscription callback

Actuator

• Add a actuator metrics API. See Add a actuator metrics API #201

WASM

• Let Layotto monitor whether the .wasm file has changed, and reload the .wasm file if there is a change.
  Achieve the effect of dynamically replacing .wasm at runtime. see .wasm module hot reload #165
• WASM Function access cache through state API. see New Feature: WASM Function access cache through state API #192
• Upgrade the wasm demo developed by rust. see Upgrade the wasm demo developed by rust #255
• Upgrade the wasm demo developed by AssemblyScript. see Upgrade the wasm demo developed by AssemblyScript #256

File api implementation

• Add components for file api. File api have been supported in layotto, need add other components for file api. Add components for file interface #236 , [Proposal] OSS api design #98
  • awsOss
  • local file system(file on local disk)
  • HDFS
  • minIo
  • Implementing File API components with ceph

Sequencer API related

• Sequencer API Component:Choose an open source component or cloud service you like (such as zookeeper, leaf, etc.) to implement distributed auto-increment id generation service.

  • Etcd

  • Stand-alone redis
    assigned

  • Zookeeper
    assigned

  • Leaf

  • Mongo

  • Consul

  • snowflake algorithm (need to avoid clock rollback problems) assigned.see [GLCC] Implement Sequencer API with snowflake algorithm #193

  • Mysql

  • PostgreSQL

  • Any other storage

• Implement the segment caching feature of Sequencer API. assigned to @ZLBer ,see Implement the segment caching feature of Sequencer API #158
  You can refer to Leaf to do double buffer optimization
  The function to be implemented is at https://github.com/mosn/layotto/blob/main/pkg/runtime/sequencer/cache.go

Distributed Lock API

• Choose an open source component or cloud service (such as zookeeper) you like to implement distributed lock API. See Add more components for distributed lock API #104
  • redis standalone
    done
  • redis cluster (with redlock or some other algorithm to make it safer)
  • zookeeper
    assigned.see Zookeeper lock #111
  • etcd
    assigned.see Add etcd lock #128
  • Consul
    assigned.see Use Consul to implement Distributed Lock API #129
  • Mongo
    assigned.see add mongo distributed lock #348
  • Cassandra
    refer to https://github.com/dekses/cassandra-lock
  • anything else. e.g. some build block provided by AWS or aliyun

Engineering

• Make Layotto CI more powerful. See ci: stabilize new pipelines #532 (comment)
• Add more linters. See ci: add more linters to make our CI stricter; 在CI中添加更多 linter #599
• Provide Layotto Dockerfile,so that users can deploy Layotto with Docker. See Layotto Dockerfile #178
• Automatically generate API documents: use some tools to automatically generate API documents based on proto files. Refer to the document automatically generated by Etcd
• Automatically check the pull request title and ensure it to meet the format of type(scope): subject in order to make the commit history more readable.
  See feat: add commit message lint to workflow #243
• Automatically check that new PR must have related issues
• Automatically check code style with go lint in ci/cd pipeline. For example:
  • no Chinese in the code;
  • Every code that starts a new goroutine must have a recover
    It doesn't matter if these examples can't be realized. As long as we can do valuable automated inspections

Currently we use github actions as ci/cd pipeline, and the configuration file is here
You can refer to some github actions high-quality tutorials:

https://www.ruanyifeng.com/blog/2019/09/getting-started-with-github-actions.html

http://www.ruanyifeng.com/blog/2019/12/github_actions.html

• Automated build: Automatically compile multi-platform binary files (binary files under linux/mac), compress (compress binary files), and build docker images
  You need to investigate the solution to see whether you write a build script yourself, using the cross-compilation feature of go, or just use a exist platform to help you build

Hard

Observability

• Integrate with Skywalking, Jaeger and other systems
  Layotto currently supports tracing and we hope to integrate with observability-related platforms such as Skywalking and Jaeger.
  • Skywalking. assigned Integrate with Skywalking #310
  • Jaeger. assigned feature: layotto integration jaeger #547
  • Zipkin
  • Some other tracing platform...

Runtime API Lab

Hard

#530

• Redis API.

• Kafka API.
  Alicloud SLS component can implement Kafka-like API too.

• Design transaction message API (like RocketMQ's) for pubsub

• Delay message API for pubsub

• Let Layotto support Dapr's Config API (alpha version)
  We have been discussing and working with Alibaba and Dapr community to contribute a Config API for Dapr since March of this year,and recently it was finally merged into Dapr .
  Now it's time to can make Layotto support Dapr's Config API.
  Dapr Config API is still in the alpha version, which is similar to Layotto's existing Config API but lacks some fields. You can refer to Layotto's existing implementation during development.
  See Dapr's API definition

• Let Layotto support secret API.
  Layotto's goal is to build a Runtime API standard with Dapr and other communities (promote Dapr API as an industry standard, and Layotto as an implementation of this API), so it needs to support Dapr's secret API. Therefore, this task needs to port Dapr's secret API into Layotto. For Dapr's secret management related documents, see https://docs.dapr.io/developing-applications/building-blocks/secrets/
  assigned. see Let Layotto support secret API #212

• Layotto support Binding API. Same as above, porting Dapr's binding API into Layotto

WASM Lab

• Support loading multiple wasm files,so that Layotto can serve as FaaS container. See New feature: load multi wasm instance #176
• Support dynamic load wasm file. See New Feature: dynamic load wasm file #191

If you are interested, you can reply and we will assign the task to you

Kubernetes Lab

• Deploy Layotto on Kubernetes. see Deploy Layotto on Kubernetes #189

Istio

• Integrates with istio 1.10, allowing layotto's invokeService API to reuse istio's traffic management capabilities. See [OSPP 2022] run layotto along with istio 1.10.0 #311