External nats support and 2 node cluster setup. #70

oz123 · 2023-07-26T07:33:15Z

oz123
Jul 26, 2023

Thank you for the reply in #68.

I found out that I can still use marmot with external nats-server, which is great.
This brings two questions:

Is this always going to be the case, or are you planning to drop this?
Will there be a possibitly to build marmot without the embedded nats-server?

Answered by maxpert

Jul 26, 2023

You can run Marmot single node with ./marmot -config node-1-config.toml don't specify cluster address. The moment you specify cluster address it tries to build a cluster. This will launch marmot with embedded nats if the nats.urls in config file is empty, otherwise just embedded NATS with jetstream. I would suggest for your setup that you are describing bring up a single NATS server with JS and just put localhost:4222 like so in config for both.

View full answer

maxpert · 2023-07-26T07:49:47Z

maxpert
Jul 26, 2023
Maintainer

Is this always going to be the case, or are you planning to drop this?

No, this is main intention of Marmot.

Will there be a possibitly to build marmot without the embedded nats-server?

No, I've resisted this idea for long time, my ultimate conclusion was that not everyone wants to deploy 2 different binaries/sidecars. I intended to make getting up and running really quick and easy. People can obviously add their own NATS instances as part of cluster. At the end of day it makes life easier than difficult for people

0 replies

oz123 · 2023-07-26T08:14:29Z

oz123
Jul 26, 2023
Author

OK. I understand you want to make life easier. That is awesome.|
The current demo bash script only shows how to run with 3 nodes only.

Can I start marmot with 1 binary runing jetstream and the other one connected to it without requiring RAFT consensus?

Something like:

./marmot -config node-1-config.toml -cluster-addr localhost:4222
./marmot -config node-2-config.toml

In this configuration for node-2-config.toml I would specify the nats address localhost:4222.
However, before this can be even done, I need to figure out how to run the embedded nats as a single node.

It crashes with :

vagrant@ubuntu2204:~$  ./marmot -config node-1-config.toml -cluster-addr localhost:4222
2023/07/26 08:24:13.545288 [INF] Starting nats-server
2023/07/26 08:24:13.545332 [INF]   Version:  2.9.19
2023/07/26 08:24:13.545344 [INF]   Git:      [not set]
2023/07/26 08:24:13.545365 [INF]   Cluster:  e-marmot
2023/07/26 08:24:13.545374 [INF]   Name:     marmot-node-1
2023/07/26 08:24:13.545385 [INF]   Node:     OWL7P9aV
2023/07/26 08:24:13.545393 [INF]   ID:       ND6Z7SAPCXQ3FFAUGRMJSQFDFNHKZSTS7PZEP56RS5CMMTCTPICOSGEC
2023/07/26 08:24:13.546508 [INF] Starting JetStream
2023/07/26 08:24:13.546845 [INF]     _ ___ _____ ___ _____ ___ ___   _   __  __
2023/07/26 08:24:13.546877 [INF]  _ | | __|_   _/ __|_   _| _ \ __| /_\ |  \/  |
2023/07/26 08:24:13.546892 [INF] | || | _|  | | \__ \ | | |   / _| / _ \| |\/| |
2023/07/26 08:24:13.546905 [INF]  \__/|___| |_| |___/ |_| |_|_\___/_/ \_\_|  |_|
2023/07/26 08:24:13.546919 [INF] 
2023/07/26 08:24:13.546932 [INF]          https://docs.nats.io/jetstream
2023/07/26 08:24:13.546945 [INF] 
2023/07/26 08:24:13.546957 [INF] ---------------- JETSTREAM ----------------
2023/07/26 08:24:13.546981 [INF]   Max Memory:      32.00 MB
2023/07/26 08:24:13.547312 [INF]   Max Storage:     1.00 GB
2023/07/26 08:24:13.547385 [INF]   Store Directory: "/home/vagrant/nats/marmot-node-1/jetstream"
2023/07/26 08:24:13.547395 [INF] -------------------------------------------
2023/07/26 08:24:13.548206 [INF] Starting JetStream cluster
2023/07/26 08:24:13.548244 [FTL] Can't start JetStream: JetStream cluster requires configured routes or solicited leafnode for the system account
vagrant@ubuntu2204:~$

So there is some configuration to pass here.
My understanding of nats-server is still limited ... I apologize if this is trivial.

1 reply

maxpert Jul 26, 2023
Maintainer

You can run Marmot single node with ./marmot -config node-1-config.toml don't specify cluster address. The moment you specify cluster address it tries to build a cluster. This will launch marmot with embedded nats if the nats.urls in config file is empty, otherwise just embedded NATS with jetstream. I would suggest for your setup that you are describing bring up a single NATS server with JS and just put localhost:4222 like so in config for both.

Answer selected by oz123

oz123 · 2023-07-26T09:02:20Z

oz123
Jul 26, 2023
Author

Nice! That works.
The only caveat is that the embded nats server starts on a random port.
I see that there is an option server_config="" which allows me to specify the port.

First node:

$ cat node-1-config.toml 
seq_map_path="./marmot-1-sm.cbor"
db_path="./marmot-1.db"
node_id=1

[nats]
server_config="./nats.config"

embeded nats configuration:

vagrant@ubuntu2204:~$ cat nats.config 
listen: 127.0.0.1:4222

Second node configuration:

vagrant@ubuntu2204:~$ cat node-2-config.toml 
seq_map_path="./marmot-1-sm.cbor"
db_path="./marmot-2.db"
node_id=2

[nats]
urls=[
    "nats://localhost:4222"
]

Running 1st node like this:

./marmot -config node-1-config.toml

And the 2nd node:

./marmot -config node-2-config.toml

And this works like a charm. Thank you for your patience and continued work on this project!

1 reply

maxpert Jul 26, 2023
Maintainer

I have a PR coming that will let you set bind port of embedded NATs. Feel free to join discord and if you are going more deeper or need help integrating further.

Navigation Menu

External nats support and 2 node cluster setup. #70

Uh oh!

{{title}}

Uh oh!

Replies: 3 comments 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

External nats support and 2 node cluster setup. #70

Uh oh!

oz123 Jul 26, 2023

Replies: 3 comments · 2 replies

Uh oh!

Uh oh!

maxpert Jul 26, 2023 Maintainer

Uh oh!

Uh oh!

oz123 Jul 26, 2023 Author

Uh oh!

maxpert Jul 26, 2023 Maintainer

Uh oh!

Uh oh!

oz123 Jul 26, 2023 Author

Uh oh!

maxpert Jul 26, 2023 Maintainer

oz123
Jul 26, 2023

Replies: 3 comments 2 replies

maxpert
Jul 26, 2023
Maintainer

oz123
Jul 26, 2023
Author

maxpert Jul 26, 2023
Maintainer

oz123
Jul 26, 2023
Author

maxpert Jul 26, 2023
Maintainer