--- title: "Fix an erpc worker failure" description: "The erpc worker error messages you meet most, what each means and what to do, how to recover a server that may have been taken over, and what erpc never does." canonical: "https://erpc.global/en/doc/erpc-cli/worker-troubleshooting/" locale: "en" --- # Fix an erpc worker failure Find the start of the message `erpc worker` printed, and do what its row says. This page describes ERPC CLI `v0.1.0-rc.4`, a pre-release. ## Error messages and what to do Worker failures use erpc's codes and exit statuses: see [Errors and exit codes](https://erpc.global/en/doc/erpc-cli/commands/#errors-and-exit-codes). | Message (start of it) | Why | What to do | | --- | --- | --- | | ` cannot start without ` | `run.requires` names a setting nothing sets | `erpc worker secret put `, set it in `[vars]`, or deploy with the line that sets it (`--grpc`, `--unlimited`) | | `your key was added but does not get in` | The SSH agent did not sign: 1Password locked, or its request not allowed | Unlock 1Password, allow the request, and run the same command again. It carries on | | `1Password could not …: authorization timeout` | 1Password asked for approval and nobody answered | Allow the request in 1Password, and run it again | | ` is empty; nothing was kept` | `secret put` read an empty standard input: the shell is not a terminal | Run it in a terminal, or pipe the value in | | `the keystore is locked, and nobody is at a terminal …` | A command without a terminal needs the keystore, and no unlock is kept | `erpc worker secret unlock` at a terminal, then the command again | | `nothing is kept under keystore:… on this machine` | The value was kept in another machine's keystore | `erpc worker secret put --replace` here; for the key, bring that machine's whole `keystore` directory and its passphrase | | `release failed: …; the server went back to ` | The new release did not pass its health check | See `erpc worker logs`. The previous release still runs | | `… is locked: the trading key has not been handed over` | The server rebooted, the agent failed or was killed and systemd started it again, or `stop --lock` | `erpc worker start` | | `port is taken: …` | Another worker that is kept running has the same health-check port, or another program listens on it | Give the app a port of its own in `erpc.toml`, then deploy again | | `release runs, and its WAF did not come up: …` | The WAF did not start listening. Most often, another program is on `waf.port` | `erpc worker logs --waf`, then free the port or change `waf.port`, and deploy again. The app runs meanwhile | | `waf. …` (from `check`, `deploy`) | A `[waf]` key is outside its bounds, or `waf.port` is below 1024, `run.port`'s, SSH's or the health check's | Fix `erpc.toml`. `erpc worker check` lists every problem at once | | `the WAF's port was not opened to …` (from `firewall allow`, `deny`, `sync`) | The change would let someone new in under WAF rules the server is not running yet | `erpc worker deploy`: it puts the new rules in place, then opens the port | | ` line did not become active in minutes` | The line is still being set up | Deploy again to wait more | | `OUT_OF_STOCK` | No server of that plan is free in that region | Deploy with `--region ` | | `the start may or may not have gone through: …` | The request left and no answer came back, or the answer was 5xx, 408 or 429 | Run the same command again: it sends the same request | | `this purchase, or another of the account's, is still being made …` | The first request is still at work, or another purchase of the account's holds its lock | Run the same command again shortly: it sends the same request | | `… changed in 1Password since this command read it` | Another machine wrote the inventory item meanwhile | Run the command again | | `the shared RPC takes none of the account's API keys from this machine ( tried)` (from `secret put --from-account`, `erpc rpc call`, `erpc webhook run`) | Each of the account's keys was refused from this machine: the keys have ended, the account has no active shared RPC plan, or the keys are limited to other addresses, such as a server's | The [ERPC Web Dashboard](https://dashboard.erpc.global/en) shows a key's end date and the plan, and makes a new key. For a key limited to the server's address: `erpc worker secret put ERPC_API_KEY`, typed or piped in | | `shared: … HTTP 401` (bench) | The shared RPC refused the key: no active shared RPC plan, or a wrong key | Check the plan on the dashboard, or set `ERPC_API_KEY` | | `shared: … HTTP 429` (bench) | The plan's rate limit | Use `--interval 150` or more | ### A purchase that erpc cannot put on record When a server or a line may have been started by an earlier request that erpc could not put on record, look at your account's servers and lines in the dashboard. Take a server up with `erpc worker deploy --vps `, or end what is not wanted. Then remove the entry from `[env..starting]` in `erpc.toml` by hand: erpc has no flag that drops it, and until then the same command sends the same request. ## If the server may have been taken over 1. Run `erpc worker backup off` first, to revoke the server's backup access key. `off` does not undo what the server already read. 2. Move funds off its trading key, and replace its secrets. 3. Rebuild the server from the [ERPC Web Dashboard](https://dashboard.erpc.global/en), and deploy again. If the rebuild changed the server's SSH host key, erpc refuses the server until you update the pinned key. A deploy brings back the code, not the app's state, which only a backup can restore ([Back up the app's state](https://erpc.global/en/doc/erpc-cli/worker-status-and-billing/#back-up-the-apps-state)). ## What erpc never does - Put a key, a secret, or a URL that carries one on a command line. - Write a trading key or an app's secret to a file, with one exception: its keystore on your machine, where each is encrypted under your passphrase. On a server, `erpc worker backup on` leaves the server's own backup keys in a root-only file. - Start anything without showing its price. - Leave a server open: the firewall is in force before the app first starts. - Reach a server whose host key does not match the pinned one. [The worker guide](https://github.com/elsoul/erpc-cli/blob/v0.1.0-rc.4/docs/worker.md) lists every message, including the WAF's.