Skip to main content
App Library
Ollama /

Ollama

Polished CLI for local LLM inference, with built-in model management and an OpenAI-compatible API. Simple to manage, and compatible with both GPUs and CPUs.

LLM Inference
Versionlatest
LicenseMIT License
InterfaceAPI
GPUOptional
One-Click Install on Laboratory OS
GitHub
Securely deploy Ollama on Laboratory OS.

Ollama

Laboratory OS Installation Details

Environment

Working Directory/workspace/ollama

Environment Variables

APP_DIRECTORY
/workspace/ollama
EXECUTABLE_PATH
${APP_DIRECTORY}/runtime/bin/ollama
OLLAMA_PORT
11434
OLLAMA_MODELS
${APP_DIRECTORY}/models
OLLAMA_HOST
0.0.0.0:${OLLAMA_PORT}
OLLAMA_MAX_VRAMMaximum VRAM to use in bytes, for example 0 for unlimited.
0
OLLAMA_CONTEXT_LENGTHContext length for the Ollama model.
32768
OLLAMA_KEEP_ALIVEHow long models stay loaded in memory by default. Supports duration strings like 10m, seconds like 3600, 0 to unload immediately, or negative values to keep models loaded.
5m
OLLAMA_NUM_PARALLELMaximum number of parallel requests each loaded model can process. Memory usage grows with OLLAMA_NUM_PARALLEL multiplied by OLLAMA_CONTEXT_LENGTH.
1
OLLAMA_FLASH_ATTENTIONEnable Flash Attention to reduce memory usage as context size grows.
—
OLLAMA_KV_CACHE_TYPEQuantization type for the K/V cache when Flash Attention is enabled. This is a global Ollama setting that applies to all loaded models.
f16

Install & Run

install-and-run.sh
#!/bin/bash
set -e

export PATH="/opt/conda/bin:/root/.local/bin:$PATH"

# Create the directory if it doesn't exist
mkdir -p "/workspace/ollama"
  
# Change to the app directory
cd "/workspace/ollama"


# Environment setup complete
echo "Environment setup complete!"

echo "Preparing durable Ollama runtime..."
mkdir -p "$OLLAMA_MODELS"
rm -rf "$APP_DIRECTORY/runtime"
mkdir -p "$APP_DIRECTORY/runtime"

echo "Downloading Ollama..."
curl -fL --progress-bar https://ollama.com/download/ollama-linux-amd64.tar.zst -o /tmp/ollama-linux-amd64.tar.zst
echo "Extracting Ollama..."
tar --zstd -x -C "$APP_DIRECTORY/runtime" -f /tmp/ollama-linux-amd64.tar.zst
rm -f /tmp/ollama-linux-amd64.tar.zst
"$EXECUTABLE_PATH" --version || true

exec ollama serve $OPENLAB_EXTRA_ARGS

CLI Command

cli.sh
# Ollama CLI
exec ollama

# Ollama Pull
exec ollama pull