Worker: manual setup#

This page sets up a compute worker file by file, the same layout worker-setup.sh writes. Read Run a compute worker first for the requirements, the owner key and the bond.

Install the binaries#

Use the release binaries (see Get the release), or build them from the release's source on a Linux machine whose glibc is no newer than the worker's. A build on Ubuntu 22.04 or Debian 12 runs on both and on anything newer:

sudo apt install -y build-essential git curl cmake
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh   # Rust 1.80 or newer
git clone <SOURCE_REPO_URL> tog && cd tog
git checkout <RELEASE_TAG>
cargo build --release --locked -p tog-worker -p tog-signer -p tog-executor -p tog-cli

User and directories#

Everything lives under /opt/tog, and the processes run as a system user tog:

sudo useradd --system --home-dir /opt/tog --shell /usr/sbin/nologin tog
sudo install -d -o root -g root -m 755 /opt/tog /opt/tog/bin
sudo install -o root -g root -m 755 target/release/{tog-worker,tog-signer,tog-executor,tog-cli} /opt/tog/bin/
sudo install -d -o tog -g tog -m 750 /opt/tog/worker /opt/tog/worker/togd \
  /opt/tog/worker/executor /opt/tog/worker/executor/work /opt/tog/worker/executor/keys /opt/tog/worker/executor/images
sudo install -d -o tog -g tog -m 700 /opt/tog/worker/keys
id -u tog          # the tog uid: you need it in the signer unit and executor.toml

Create the directories as shown, owned by tog. If root owns them, togd and the executor refuse to start.

Keys#

All are 32-byte random seeds. On the worker:

cd /opt/tog/worker/keys
sudo -u tog sh -c 'umask 077; head -c 32 /dev/urandom > worker.key; head -c 32 /dev/urandom > vrf.key'
sudo /opt/tog/bin/tog-cli address /opt/tog/worker/keys/worker.key   # the worker id

togd creates network.key on its first start. The signer and togd refuse a key file that group or others can read. Back up worker.key and vrf.key encrypted.

Network files#

sudo curl -fsSL -o /opt/tog/genesis.borsh <GENESIS_URL>
sudo curl -fsSL -o /opt/tog/kernels.tda <KERNEL_SET_URL>

Check the genesis against the published hash before you go on. The values for <NETWORK_ID>, the bootnodes and the trusted header are on the Networks page.

togd.toml#

/opt/tog/worker/togd.toml:

data_dir = "/opt/tog/worker/togd"

[hub]
network_id = <NETWORK_ID>
genesis = "/opt/tog/genesis.borsh"
trusted_height = <TRUSTED_HEIGHT>
trusted_block_hash = "<TRUSTED_BLOCK_HASH>"

[p2p]
listen = ["/ip4/0.0.0.0/tcp/30333"]
external = ["/ip4/<PUBLIC_IP>/tcp/30333"]
bootnodes = ["<BOOTNODE_1>", "<BOOTNODE_2>", "<BOOTNODE_3>"]
grid_bootnodes = ["<GRID_BOOTNODE_1>"]
# metrics = "127.0.0.1:9615"     # optional Prometheus endpoint

[executor]
socket = "/opt/tog/worker/executor.sock"

[signer]
socket = "/opt/tog/worker/signer.sock"

[cas]
in_max_bytes = 21474836480        # 20 GiB; the default is 512 GiB

• trusted_height = 0 with trusted_block_hash of 64 zeros starts from genesis. Once the network is older than 14 days, use a published checkpoint instead. The same applies when the worker was off for more than 14 days.
• List every published bootnode. At each era change togd accepts the new validator set only when peers in different networks confirm it; with bootnodes in a single network it stops following the Hub at the first era change.
• external publishes your address right away; without it togd waits until peers tell it its address.
• [cas] in_max_bytes caps togd's input cache. Lower it on a small disk.

sudo -u tog /opt/tog/bin/tog-worker -c /opt/tog/worker/togd.toml check-config

executor.toml#

/opt/tog/worker/executor.toml. Size the limits to your machine:

togd_uid = <TOG_UID>                       # id -u tog
socket = "/opt/tog/worker/executor.sock"
work_root = "/opt/tog/worker/executor/work"
keys_dir = "/opt/tog/worker/executor/keys"
cas_root = "/opt/tog/worker/togd/cas"      # togd's data_dir + /cas
runtime = "process"
allow_unisolated = true                    # required by the process runtime: no sandbox

[policy]
tiers = ["t1"]
accept_t1_risk = true                      # required to offer t1
max_cpu_cores = 2
max_ram_bytes = 2147483648                 # 2 GiB
max_scratch_bytes = 1073741824             # 1 GiB
max_output_bytes = 1073741824
max_image_bytes = 1073741824
max_input_bytes = 1073741824

[images]
root = "/opt/tog/worker/executor/images"
max_bytes = 10737418240                    # 10 GiB
min_idle_secs = 3600

sudo -u tog /opt/tog/bin/tog-executor -c /opt/tog/worker/executor.toml check-config

client.toml (on your own computer)#

tog-cli reads only the [hub] and [p2p] sections of a togd-format file. Keep the genesis file next to it:

[hub]
network_id = <NETWORK_ID>
genesis = "/home/<you>/tog/genesis.borsh"
trusted_height = <TRUSTED_HEIGHT>
trusted_block_hash = "<TRUSTED_BLOCK_HASH>"

[p2p]
bootnodes = ["<BOOTNODE_1>", "<BOOTNODE_2>", "<BOOTNODE_3>"]

Firewall#

sudo ufw allow 30333/tcp

Behind NAT, forward TCP 30333 to the machine and put the public address in external.

systemd units#

The signer's --chain-id is the network id shifted left by 32 bits (echo $(( <NETWORK_ID> << 32 ))), and --uid is the tog uid.

/etc/systemd/system/tog-worker-signer.service:

[Unit]
Description=TOG worker's signer
After=network-online.target
Wants=network-online.target

[Service]
User=tog
WorkingDirectory=/opt/tog/worker
Environment=RUST_LOG=info,libp2p=warn
ExecStart=/opt/tog/bin/tog-signer --socket /opt/tog/worker/signer.sock --keys /opt/tog/worker/keys --journal /opt/tog/worker/signer-journal --chain-id <CHAIN_ID> --uid <TOG_UID>
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

/etc/systemd/system/tog-worker-executor.service:

[Unit]
Description=TOG worker's executor (process runtime)
After=network-online.target
Wants=network-online.target

[Service]
User=tog
WorkingDirectory=/opt/tog/worker
Environment=RUST_LOG=info,libp2p=warn
ExecStart=/opt/tog/bin/tog-executor -c /opt/tog/worker/executor.toml run
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

/etc/systemd/system/tog-worker.service:

[Unit]
Description=TOG worker (togd)
After=network-online.target tog-worker-signer.service tog-worker-executor.service
Wants=network-online.target tog-worker-signer.service tog-worker-executor.service

[Service]
User=tog
WorkingDirectory=/opt/tog/worker
Environment=RUST_LOG=info,libp2p=warn
ExecStart=/opt/tog/bin/tog-worker -c /opt/tog/worker/togd.toml run
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Start and register#

Start the signer and the executor. Register before you start togd:

sudo systemctl daemon-reload
sudo systemctl enable tog-worker-signer tog-worker-executor tog-worker
sudo systemctl start tog-worker-signer tog-worker-executor

1. On your computer, read the owner's next nonce with tog-cli account <OWNER_ADDRESS> -c client.toml.
2. On the worker, build the unsigned registration as the tog user:

   sudo -u tog /opt/tog/bin/tog-worker -c /opt/tog/worker/togd.toml register-worker \
     --owner <OWNER_ADDRESS> --owner-nonce <NONCE> \
     --bond 10000000000000000000000 \
     --region <XX> \
     --kernel-set /opt/tog/kernels.tda \
     --out /opt/tog/worker/register.borsh

   --bond is in atto-TOG (1 TOG = 10¹⁸ atto), so the number above is 10 000 test-TOG. --region is your country as two capital letters.
3. Copy register.borsh to your computer and, within 300 Hub blocks (about 10 minutes), sign and send it: tog-cli send register.borsh --key owner.key -c client.toml.
4. Start togd: sudo systemctl start tog-worker.

After a reboot systemd starts all three; togd waits for its signer and executor. Check it as described in Start and check.