Test and Troubleshoot
{"content":"{{\u003c tier level=\"sovereign\" \u003e}}\n\nTest your tool's contract and failure behavior before sharing it. Passing these checks gives you confidence in the inputs and host setup you tested. When reusing the tool, verify the workflow's inputs, working directory, dependencies, and credentials.\n\n## Test a tool before sharing it\n\nFor every supported action, check:\n\n| Check | What to verify |\n| --- | --- |\n| Input contract | Required fields, extra fields, wrong types, empty and Unicode values, and boundary values. Invalid input must never trigger effects. |\n| Result contract | One JSON result on stdout, the right data shape, handled errors, logs on stderr, and exit codes. |\n| Sensitive output | Use dummy secrets to check that your tool removes credentials and unnecessary sensitive data from results, errors, and logs. Redaction is best effort; you are responsible for what the tool emits. |\n| Host prerequisites | A missing runtime or CLI, missing environment variables, sign-in and permission failures, and the supported OS and architecture. |\n| Working directory and packaging | Run from a folder outside the tool's source and confirm bundled files still resolve. Validate, package, import, export, and re-import. |\n| Limits and interruption | Input and output limits, timeout, cancellation, and cleanup of child processes. Check external effects separately. |\n| Retries and updates | Safe repetition or explicit reconciliation, new revisions, and older workflows that still pin the previous revision. |\n\nA direct stdin test, like the one in [Build Your First Tool](../quickstart/#4-run-it-directly), checks your program. A workflow test also checks file verification, pinned schemas, the working directory, environment filtering, output mappings, and recovery. Inspect what remains visible in saved errors and logs, and remember that [successful result data is not redacted by Valdr](../runtime/#redaction-and-sensitive-output). For a real service, run a known, permitted read before you add writes; a fake CLI or a `--version` check can't prove account access.\n\n## Diagnose common failures\n\nOpen the step that stopped the run in **Runs**. It shows as blocked, or as failed when the step sets `onFailure.action: fail`, and its output mappings aren't filled. The **Error** field shows what went wrong. If your program crashed, the exit code or signal is part of that message. If your error result [reports a wrapped CLI's `exitCode` and `signal`](../runtime/#rules-for-a-clean-result), Runs also shows them as **Exit code** and **Signal**.\n\nRuns doesn't display your program's stderr. If the action is safe to repeat, run the [direct stdin test](../quickstart/#4-run-it-directly) with the step's input, from the step's working directory, with only the environment variables your manifest passes, to capture stderr from a new execution. For a write with an uncertain outcome, check what already happened before rerunning: a direct test executes the program again outside Valdr's retry controls.\n\n**Import, validation, and the Builder**\n\n| Symptom | Check |\n| --- | --- |\n| Manifest rejected | Unknown or missing fields, duplicate action IDs or paths, unsupported schema keywords, or values YAML read as the wrong type. Quote revision strings that YAML could read as numbers. |\n| Missing listed file | Paths in `files` are relative to the manifest, and the pack's `includes` must cover them. |\n| \"User tool revision conflict for '…'\" | Bump the revision after any change to the manifest or to any file it lists. Never edit an installed copy. |\n| \"User tool owner conflict for '…'\" | Another pack has installed a tool with this ID on this host, even if that tool was later removed. Choose a different ID, or ship the change from the pack that owns it. |\n| The Builder shows an action's unsupported reason, and saving or testing fails with it | The action is listed with `execution: unsupported`, so Valdr won't save or start a workflow that uses it. Choose another action, or implement it and ship a new revision. |\n\n**When a step runs**\n\nIf a tool works in your terminal but can't find a variable through the UI or MCP, check both its pinned manifest's `process.inheritEnv` and the [shared environment setup](../runtime/#environment-variables). Restart Valdr and your MCP hosts after changing `~/.valdr/environment.env` or its sources. If startup reports a reference error, check that each entry uses `${NAME}` or `$(command)` and that the source variable or command is available in your shell startup environment.\n\n| What **Runs** shows | Check |\n| --- | --- |\n| \"User tool executable is not installed: …\" | Valdr couldn't find the manifest's `executable`. Install it on the workflow host and check the `PATH` Valdr runs with, not just your terminal's. |\n| \"User tool input is invalid: …\" | An expression in the step's `inputs` produced a value the pinned input schema rejects. Check what the workflow inputs or earlier outputs resolved to. Valdr doesn't coerce types or insert defaults. Fixed inputs are checked earlier, when you save or test the workflow. |\n| \"User tool request exceeds the input byte limit.\" | Send less input, or raise `maxInputBytes` within the 1 MiB host maximum. |\n| \"Workflow command working directory is unavailable: …\" or \"…is not a directory: …\" | Valdr uses the same message as Command steps. The step's `cwd`, or its default folder, must already exist. Your program never started, so once the folder exists you can choose **Retry step**. |\n| \"User tool exited with code …\" or \"…with SIG…\" | Your program crashed or exited nonzero, so Valdr didn't read its result. A handled error should return an error envelope with exit code zero. |\n| \"User tool stdout must contain exactly one JSON result.\" or \"User tool result envelope is invalid.\" | Make sure your program prints its result before it exits. Remove logs and banners from stdout, return one UTF-8 JSON object, and use the exact success or error envelope. |\n| Your tool's own error, as `code: message` | Your program returned `{\"ok\":false,...}`. Fix the cause it reports. |\n| A schema message with no prefix, such as \"/words must be integer\" | Your program's `data` doesn't match the action's output schema. Fix the program, or change the schema and ship a new revision. |\n| \"User tool exceeded its execution timeout.\" or \"User tool result exceeds the byte limit.\" | Reduce the work or output, or raise `timeoutSeconds` (up to 300) or `maxResultBytes` (up to 1 MiB) in a new revision. Check for external effects before retrying. |\n| The failure is titled **needs attention** and **Retry step** isn't offered | Your program started and then failed, and the action declares `retry: manual` or `sideEffects: unknown`, or Valdr couldn't confirm an attempt's outcome. Check the external system and confirm what happened. Then cancel the run, and start a new one only if repeating the action is safe. |\n\n## Next step\n\nTool works? [Share it](/valdr/docs/workflows/user-tools/package-and-run/#export-and-share). Still stuck? See the [request and result format](/valdr/docs/workflows/user-tools/runtime/) or [workflow troubleshooting](/valdr/docs/workflows/troubleshooting/#a-user-tool-is-unavailable-or-fails).","description":"Test a user tool before you share it, and diagnose manifest, runtime, and output failures fast.","title":"Test and Troubleshoot"}
Sovereign
Test your tool’s contract and failure behavior before sharing it. Passing these checks gives you confidence in the inputs and host setup you tested. When reusing the tool, verify the workflow’s inputs, working directory, dependencies, and credentials.
Test a tool before sharing it
For every supported action, check:
| Check | What to verify |
|---|---|
| Input contract | Required fields, extra fields, wrong types, empty and Unicode values, and boundary values. Invalid input must never trigger effects. |
| Result contract | One JSON result on stdout, the right data shape, handled errors, logs on stderr, and exit codes. |
| Sensitive output | Use dummy secrets to check that your tool removes credentials and unnecessary sensitive data from results, errors, and logs. Redaction is best effort; you are responsible for what the tool emits. |
| Host prerequisites | A missing runtime or CLI, missing environment variables, sign-in and permission failures, and the supported OS and architecture. |
| Working directory and packaging | Run from a folder outside the tool’s source and confirm bundled files still resolve. Validate, package, import, export, and re-import. |
| Limits and interruption | Input and output limits, timeout, cancellation, and cleanup of child processes. Check external effects separately. |
| Retries and updates | Safe repetition or explicit reconciliation, new revisions, and older workflows that still pin the previous revision. |
A direct stdin test, like the one in Build Your First Tool, checks your program. A workflow test also checks file verification, pinned schemas, the working directory, environment filtering, output mappings, and recovery. Inspect what remains visible in saved errors and logs, and remember that successful result data is not redacted by Valdr. For a real service, run a known, permitted read before you add writes; a fake CLI or a --version check can’t prove account access.
Diagnose common failures
Open the step that stopped the run in Runs. It shows as blocked, or as failed when the step sets onFailure.action: fail, and its output mappings aren’t filled. The Error field shows what went wrong. If your program crashed, the exit code or signal is part of that message. If your error result reports a wrapped CLI’s exitCode and signal, Runs also shows them as Exit code and Signal.
Runs doesn’t display your program’s stderr. If the action is safe to repeat, run the direct stdin test with the step’s input, from the step’s working directory, with only the environment variables your manifest passes, to capture stderr from a new execution. For a write with an uncertain outcome, check what already happened before rerunning: a direct test executes the program again outside Valdr’s retry controls.
Import, validation, and the Builder
| Symptom | Check |
|---|---|
| Manifest rejected | Unknown or missing fields, duplicate action IDs or paths, unsupported schema keywords, or values YAML read as the wrong type. Quote revision strings that YAML could read as numbers. |
| Missing listed file | Paths in files are relative to the manifest, and the pack’s includes must cover them. |
| “User tool revision conflict for ‘…’” | Bump the revision after any change to the manifest or to any file it lists. Never edit an installed copy. |
| “User tool owner conflict for ‘…’” | Another pack has installed a tool with this ID on this host, even if that tool was later removed. Choose a different ID, or ship the change from the pack that owns it. |
| The Builder shows an action’s unsupported reason, and saving or testing fails with it | The action is listed with execution: unsupported, so Valdr won’t save or start a workflow that uses it. Choose another action, or implement it and ship a new revision. |
When a step runs
If a tool works in your terminal but can’t find a variable through the UI or MCP, check both its pinned manifest’s process.inheritEnv and the shared environment setup. Restart Valdr and your MCP hosts after changing ~/.valdr/environment.env or its sources. If startup reports a reference error, check that each entry uses ${NAME} or $(command) and that the source variable or command is available in your shell startup environment.
| What Runs shows | Check |
|---|---|
| “User tool executable is not installed: …” | Valdr couldn’t find the manifest’s executable. Install it on the workflow host and check the PATH Valdr runs with, not just your terminal’s. |
| “User tool input is invalid: …” | An expression in the step’s inputs produced a value the pinned input schema rejects. Check what the workflow inputs or earlier outputs resolved to. Valdr doesn’t coerce types or insert defaults. Fixed inputs are checked earlier, when you save or test the workflow. |
| “User tool request exceeds the input byte limit.” | Send less input, or raise maxInputBytes within the 1 MiB host maximum. |
| “Workflow command working directory is unavailable: …” or “…is not a directory: …” | Valdr uses the same message as Command steps. The step’s cwd, or its default folder, must already exist. Your program never started, so once the folder exists you can choose Retry step. |
| “User tool exited with code …” or “…with SIG…” | Your program crashed or exited nonzero, so Valdr didn’t read its result. A handled error should return an error envelope with exit code zero. |
| “User tool stdout must contain exactly one JSON result.” or “User tool result envelope is invalid.” | Make sure your program prints its result before it exits. Remove logs and banners from stdout, return one UTF-8 JSON object, and use the exact success or error envelope. |
Your tool’s own error, as code: message | Your program returned {"ok":false,...}. Fix the cause it reports. |
| A schema message with no prefix, such as “/words must be integer” | Your program’s data doesn’t match the action’s output schema. Fix the program, or change the schema and ship a new revision. |
| “User tool exceeded its execution timeout.” or “User tool result exceeds the byte limit.” | Reduce the work or output, or raise timeoutSeconds (up to 300) or maxResultBytes (up to 1 MiB) in a new revision. Check for external effects before retrying. |
| The failure is titled needs attention and Retry step isn’t offered | Your program started and then failed, and the action declares retry: manual or sideEffects: unknown, or Valdr couldn’t confirm an attempt’s outcome. Check the external system and confirm what happened. Then cancel the run, and start a new one only if repeating the action is safe. |
Next step
Tool works? Share it. Still stuck? See the request and result format or workflow troubleshooting.