Troubleshooting

Where to look first#

• A model's log. On the Models page, select the model and choose View log. The same file is logs/worker-<model>.log in the data folder.
• The server's log. The desktop app writes it to logs/studio.log in its application-data folder. From a source checkout, it is the terminal running ./run.sh.
• A download's log. logs/download-<model>.log in the data folder.
• The environment check. From a source checkout, .venv/bin/python -m basal.doctor checks that PyTorch sees the GPU and that every model library imports. A line reading out of memory right now means other programs are using the GPU, not that PyTorch is broken: close them or eject models, and run it again. (Versions before 0.3.0 reported that case as "PyTorch failed to import".)

Installing#

Starting from source#

Run ./install.sh first.#

run.sh found no .venv in the checkout. Run ./install.sh (or install.ps1 on Windows) once.

Models#

Downloads are slow#

Hugging Face limits anonymous downloads. Run hf auth login once in a terminal and restart the studio. Downloads run one at a time, smallest first, so a large model waits behind smaller ones.

A model does not load#

Most load failures are memory. The studio runs each model in its own process and reports why it stopped.

Answers are slow, and the answer says it ran on the processor#

Running on the processor, about ten times slower: the GPU did not have enough free memory when this model loaded. Eject other models (or close other GPU programs) and load it again.

The model asked for the GPU, but the GPU was full when it loaded, and its library fell back to the processor. The answers are the same, only slower. Free GPU memory, then eject the model and load it again. Until it is reloaded, the note appears with every answer from that model: in the Playground, in the studio API's notes, and in the wire formats' notes when you send X-Basal-Extensions: 1.

A call says no model is loaded#

In the studio API these come back as 409 model_not_loaded, 409 model_not_downloaded, 404 model_not_found, 503 model_load_failed and 504 model_timeout; see Errors.

Run on the processor, or switch back to the GPU#

This computer cannot run models on 'cuda'. Run setup again to install support for it. The engine was installed for another device. Setup installs one PyTorch build at a time; run it again and choose the device (System, then Run setup again, or ./install.sh --device cuda from source). Everything else is kept.

Training#

The Train page says training isn't available#

Training needs a GPU. This computer runs models on the processor. Training is off on computers without a supported GPU; every model still runs there. If the computer has a GPU, the engine was installed for the processor: run setup again and choose the GPU (System, then Run setup again, or ./install.sh from source). On Intel Arc and Core Ultra graphics, AMD on Linux, NVIDIA RTX 20 series and Apple M1, choose Try training on this GPU, or turn on Allow experimental training on the System page.

The file can't be used#

The review screen lists each problem in plain words; one shown in red stops training until the file is fixed.

Training stops or waits#

The result wasn't kept#

The page names the reason; the original model is unchanged. See When the result isn't kept for what helps, or choose Teach another model instead. The saved model did not answer exactly like the trained one is a bug: the job folder (training/jobs/<id> in the studio's data folder) keeps the rejected files and the log; please report it.

Importing a trained model fails#

Calling the API#

Every studio API error has a code, the parameter at fault and often a suggestion, all listed on Errors.

History#

History does not save, and the server's log says why#

When history cannot open, decisions keep working and nothing is saved, and the history endpoints answer 503 history_unavailable. The server prints the reason at startup, in its log.

Before applying an upgrade to your database, the studio copies it to backups/ in the data folder, keeping the last three, so an upgrade can always be undone by copying one back.

The answer is correct, but it could not be saved to history.#

A decision came back with this warning (history_write_failed) when writing to the database failed, usually a full disk. The answer is unaffected. GET /v1/studio/settings lists recent write errors under storage.store_errors. If you would rather get an error than an unsaved answer, set history.on_store_error to fail; the studio then answers 503 store_failed.

Decisions disappear from History#

Decisions are kept for 30 days by default, and history is capped at 20 GB, oldest first. Change both on the History page or with PATCH /v1/studio/settings. Pinned decisions and labelled ones (while keep labelled is on) are never removed; see Data and storage.

A background decision failed with interrupted#

The studio stopped while the decision was queued or running. At the next start it is marked failed with error.code: "interrupted" rather than run again, so no request ever runs twice. Send it again, or rerun it from History.

Search says it is unavailable#

Full-text search needs an SQLite build with FTS5, which this one lacks. The q filter needs SQLite's full-text search, which some builds leave out. Every other filter works.

Still stuck#

Report a problem on GitHub with the message, what you did, your operating system and device, and the relevant log: the model's log for a model problem, the server's log for anything else, or setup's Copy details for an install.