Skip to content

Setting Up a Local System Under Test (SUT)

A System Under Test (SUT) is a FHIR server that you run tests against. This guide covers starting a local HAPI FHIR server via Docker and registering it so FHIR Frog can find it.

Quick start

# Start a HAPI FHIR server on localhost:8081
# Note: starts plain HAPI without any IG preloaded
frog_server_start_local(confirm: true)

# The server is ready when the command completes
# Test it: curl http://127.0.0.1:8081/fhir/metadata

What gets created

When you call frog_server_start_local, the MCP:

  1. Creates a Docker network named frog-net (if it does not exist)
  2. Pulls the HAPI FHIR image (if not cached)
  3. Starts a container named frog-sut bound to 127.0.0.1:8081
  4. Waits for the server to respond to GET /fhir/metadata (typically 40–60 seconds)

The server starts plain HAPI without any IG preloaded. To use an IG-aware server, either use a server that already has the IG loaded, or set the FROG_SUT_IMAGE environment variable (for the MCP server) to your own IG-aware image; the image argument only accepts hapiproject/hapi images.

The tool also registers the server as the managed backend frog-sut in ~/.fhir-frog/backends.yaml (managed backends need no confirm for runs). Select it with server: "frog-sut": the MCP uses http://127.0.0.1:8081/fhir for frog_run_local and frog_server_check, and http://frog-sut:8080/fhir (the Docker-network name) for frog_run_remote, so you never have to choose between them. frog_server_stop removes the container and, when nothing else is attached, the frog-net network.

The server is then ready to accept FHIR requests.

Starting and stopping

Start a server

frog_server_start_local(
  port: 8081,
  confirm: true
)

Parameters: - port (optional): host port to bind (default: 8081); container always listens on 8080 - image (optional): override the HAPI image; only hapiproject/hapi images are accepted here. To use your own image (for example one that bundles your IG) set the FROG_SUT_IMAGE environment variable for the MCP server instead - confirm: must be true to actually start the server (shows a plan first if omitted)

After the server starts, register it with frog_server_register to refer to it by name in test runs.

Stop a server

frog_server_stop(
  confirm: true
)

Removes the container but keeps the frog-net network (so you can restart quickly).

Check status

frog_server_list()

Shows all registered servers and which are running:

Name        URL                                Status
my-sut      http://127.0.0.1:8081/fhir         RUNNING
prod-fhir   https://fhir.example.org/fhir      REGISTERED (not running)

Using a local SUT in tests

Register the server first, then refer to it by the registered name:

frog_server_register(
  name: "my-sut",
  url: "http://127.0.0.1:8081/fhir",
  confirm: true
)

Then use it in a local run:

frog_run_local(
  scriptPath: "patient-search.json",
  server: "my-sut"
)

Or use the runner with a test package:

frog_run_remote(
  testPackage: "fhirfrog.au-core-tests#0.1.0",
  server: "my-sut",
  runnerUrl: "http://127.0.0.1:8080"
)

HAPI FHIR version

The MCP uses the latest stable HAPI FHIR image by default (currently v7.0.0). To use a different version, check available tags and set the image:

frog_server_start_local(
  image: "hapiproject/hapi:6.10.0",
  confirm: true
)

HAPI version notes

  • v7.0.0+: Full FHIR R4 support; recommended
  • v6.x: Also works; v6.10.0 is the latest v6 release
  • v5.x and earlier: May not work with recent TestScripts; not recommended

Preloading test data or IGs

Simple approach: populate via TestScript

Load data during the Setup phase of your test:

TestScript: PopulateAndTest
Title: "Populate and Test"
Status: #draft

Fixture: patient
  * resource = "patient.json"
  * autocreate = false
  * autodelete = false

Setup:
  * operation create Patient from "patient" as p1
  * assert responseCode = 201

Test: "Search"
  * operation search Patient as s
  * assert responseCode = 200
  * assert resource = Bundle

Advanced approach: IG preloading (v0.2)

For larger datasets or IG profiles, preload an IG on the server at startup. This is planned for FHIR Frog v0.2+. In v0.1, the HAPI server starts without any IG. To get IG-aware validation today, either:

  • Use a server that already has the IG installed, or
  • Set the FROG_SUT_IMAGE environment variable (for the MCP server) to your own image that bundles the IG; the image argument only accepts hapiproject/hapi images

Manual data loading

You can also run a script to populate the server before testing:

# Fetch FHIR resources from a file or URL
curl -X POST http://127.0.0.1:8081/fhir \
  -H "Content-Type: application/fhir+json" \
  -d @patient.json

Docker notes

Port binding

The SUT is bound to 127.0.0.1:8081 by default, so it is not accessible from the network. If you want remote access, use a firewall rule or reverse proxy:

# NOT recommended: bind to all interfaces
docker run -p 8081:8080 ...

# Recommended: use a firewall or proxy

Container name

The container is always named frog-sut. Only one can run at a time. To run multiple servers, either:

  1. Stop the first and start another (reload data as needed)
  2. Use different host ports with a custom container orchestration tool (beyond the scope of this guide)

Volumes and persistence

By default, the SUT is ephemeral: when stopped, all data is deleted. To persist data, mount a volume:

# Advanced: mount a named volume
# This is not exposed in the simple API; use docker directly:
docker run -d \
  --name frog-sut \
  -v frog-sut-data:/root/data \
  -p 127.0.0.1:8081:8080 \
  hapiproject/hapi:7.0.0

However, most testing workflows do not need persistence (data is created and cleaned up per test run).

Memory

HAPI FHIR needs at least 1 GB of memory; 2–4 GB is recommended for larger test datasets. Check Docker's memory allocation:

# macOS: Docker Desktop → Preferences → Resources
# Linux: docker system info | grep Memory
# Windows: Docker Desktop Settings

Troubleshooting

"Server not ready after 60 seconds"

The HAPI startup takes time. Check the container logs:

docker logs frog-sut

If the logs show errors, the image may not have started. Retry:

frog_server_stop(confirm: true)
frog_server_start_local(confirm: true)

"Connection refused: 127.0.0.1:8081"

The server is not running. Start it:

frog_server_start_local(confirm: true)

Or check that the port is correct:

docker ps | grep frog-sut

"Address already in use"

Another process is using port 8081. Either:

  1. Stop the other process
  2. Use a different port: frog_server_start_local(port: 9000, confirm: true)

Data persists after restart

The container preserves data while stopped. To start fresh, remove the container:

docker rm frog-sut
frog_server_start_local(confirm: true)

Registering a remote server

If you want to test against a server elsewhere (dev, staging, production), register it:

frog_server_register(
  name: "staging",
  url: "https://fhir.staging.example.org/fhir",
  conformsTo: "http://example.org/ImplementationGuide/my-ig#4.2.0",
  confirm: true
)

Then use it in tests:

frog_run_local(
  scriptPath: "patient-search.json",
  server: "staging"
)

Verifying the server

Check that the server is running and responding:

frog_server_check(url: "http://127.0.0.1:8081/fhir")

Output:

URL:    http://127.0.0.1:8081/fhir
Status: ✓ responding (HTTP 200 /metadata)
Conformance: FHIR R4

Or use curl:

curl http://127.0.0.1:8081/fhir/metadata

See also