Skip to content

gtdbprep: classify genomes without the 100 GB download

gtdbprep is the laptop half of a split GTDB-Tk. It runs GTDB-Tk’s own light stages (identify, then align --skip_gtdb_refs) on your genomes, after giving them and their contigs opaque names. It packs the results into a bundle of a few kilobytes per genome. The heavy step, tree placement, can then run elsewhere: on a machine with the full reference package, on a laptop with gPlace and a memory cap, or on a shared server if one is ever built. gtdbtk classify --align_dir <bundle> reads the bundle unchanged.

It wraps GTDB-Tk rather than reimplementing it. Every number in the bundle comes from GTDB-Tk 2.8.0 itself, run as a subprocess.

GTDB-Tk’s reference data for R232 is a 61 GB download that unpacks to about 100 GB, and bacterial placement needs about 140 GB of RAM (see GTDB-Tk and memory). Gene calling, marker search and alignment need none of that. They need 77 MB of marker HMMs, masks and metadata, and take well under a minute per genome on a laptop. Splitting there means:

  • the laptop never downloads the 100 GB package;
  • only a masked protein alignment and marker counts leave the laptop, never the DNA;
  • the result is still GTDB-Tk’s own result, as the round trip below checks.
# Laptop: GTDB-Tk 2.8 in a conda/micromamba env, plus the 77 MB client reference data
gtdbprep prep --genomes mags/ -x fa --work prep-work --out bundle.tar.gz \
--data r232-client/ --env ~/micromamba/envs/gtdbtk
irit-gtdb-prep ... # the same as `gtdbprep prep ...`
# Optional: also add skani 0.3.1 sketches for a remote ANI screen (contain DNA, see Privacy)
gtdbprep prep ... --sketch --skani /path/to/skani-0.3.1
# Where the full reference data is: classify the bundle, results get the original names back
gtdbprep classify --bundle bundle.tar.gz --work prep-work --data release232/
# Testing: prep, classify from the bundle, plain classify_wf, compare genome by genome
gtdbprep verify --genomes mags/ -x fa --work rt-work --data release232/ --env ...
# Laptop: species by ANI, then tree placement with gPlace for the rest (sections below)
gtdbprep ani --genomes mags/ -x fa --work ani-work ...
gtdbprep place --genomes mags/ -x fa --work place-work --skip-assigned ani-work \
--data placement-lite/release232 --env ~/micromamba/envs/gtdbtk

Options shared by all commands: --data (or GTDBTK_DATA_PATH), --env (or GTDBPREP_ENV, a conda prefix whose bin/ goes first on PATH), --cpus (default 4). --full-tree true|false on classify, verify and place defaults to false, or to true when the reference data has an empty split/ folder (GTDB’s mockup database).

prep writes into --work:

Path What Leaves the laptop?
renamed/<id>.fna genomes with opaque names (sequence bytes unchanged) no
mapping.tsv opaque ID → GTDB-Tk genome ID, source path, source SHA-256, contig count, and why an input was left out (if it was) no, never
contigs/<id>.tsv opaque contig name → original header no
gtdbtk/ GTDB-Tk’s full identify/align output (genes, HMM hits) no
logs/ one log per command no
bundle/ the bundle (also copied to --out) yes, this is what you send

On Unix the work folder is made owner-only (mode 0700), and mapping.tsv and contigs/*.tsv are written with mode 0600.

Input files that are empty or not FASTA (no > header, or sequence before the first one) are left out with a warning instead of stopping the run. mapping.tsv lists them with opaque ID - and the reason; prep prints them; the manifest records only how many there were (inputs_excluded), not their names. classify repeats the warning, and verify reports them as excluded (not as differences) and runs classify_wf on the remaining inputs only.

Spaces in paths. GTDB-Tk 2.8.0 refuses genome paths that contain a space (“unsupported by downstream applications”), and it reads the renamed genomes from the work folder, so prep and classify refuse a work folder with a space in its path before doing anything. The input folder may have spaces: only gtdbprep reads it, and verify gives classify_wf a folder of links to the inputs inside the work folder. A --data path with a space is untested.

--out takes a folder, a .tar, or a .tar.gz. It is checked before anything runs: it must not exist yet (an empty folder is fine) and its parent folder must exist.

What is reproducible. Tar files have fixed metadata (mtime 0, mode 644, owner 0, sorted entries), and every gzip file in the bundle is written with a fixed header (no file name, mtime 0; GTDB-Tk’s own .gz files are decompressed and recompressed so its header, which names the source file and the time it ran, does not leave the laptop). So the same bundle folder always gives the same .tar bytes. Two prep runs on the same genomes do not give the same bundle: the opaque IDs are random and the manifest records started and created times. .tar.gz bytes also depend on the gzip compressor (flate2 and its backend version), so a different gtdbprep build can compress the same .tar to different bytes; compare .tar files or the manifest’s files hashes instead.

The bundle uses GTDB-Tk’s own folder layout and file names (prefix gtdbtk), so it is a valid --align_dir:

manifest.json
identify/gtdbtk.bac120.markers_summary.tsv marker counts and lists per genome
identify/gtdbtk.ar53.markers_summary.tsv
identify/gtdbtk.translation_table_summary.tsv genetic code per genome (11 or 4)
identify/gtdbtk.failed_genomes.tsv Prodigal failures (often empty)
align/gtdbtk.bac120.user_msa.fasta.gz masked alignment, 5,010 columns
align/gtdbtk.ar53.user_msa.fasta.gz masked alignment, 4,369 columns (if any archaea)
align/gtdbtk.{bac120,ar53}.filtered.tsv genomes below 10% of columns
align/intermediate_results/gtdbtk.{align,identify}.failed.tsv (when GTDB-Tk writes them)
sketches/<id>.fna.sketch only with --sketch

manifest.json records:

  • format, client (name, version), started and created (RFC 3339, UTC);
  • gtdb_release (from metadata/metadata.txt) and tools: the version line each tool printed (gtdbtk, prodigal, hmmer, and skani with --sketch). A line containing / or \ (a Python traceback naming a path, say) is not kept: reported is then just the parsed version;
  • commands: the exact GTDB-Tk and skani argument lists, with local paths replaced by <work> and <GTDBTK_DATA_PATH>;
  • reference: for markers/, masks/, metadata/ and taxonomy/, GTDB-Tk’s own folder hash (sha1_dir) next to the REF_HASHES value GTDB-Tk 2.8.0 expects for that release, plus a SHA-256 over the folder’s sha256sum listing, and per-file SHA-256 for every folder except markers/ (182 files, covered by the listing hash). prep refuses data whose markers, masks or metadata don’t match REF_HASHES (--allow-ref-mismatch overrides). See Reference data for taxonomy/;
  • genomes: per opaque ID, the SHA-256 of the renamed FASTA, contig count, domain call, single-copy marker counts, genetic code, which MSA holds it, and why GTDB-Tk excluded it if it did;
  • inputs_excluded: how many input files prep left out (empty or not FASTA), without names;
  • sketches: skani version and parameters (c = 125, marker c = 1000, k = 15) and the warning text;
  • files: size and SHA-256 of every other file in the bundle. classify rejects a bundle whose files don’t match, and checks again after GTDB-Tk has run that nothing was changed;
  • privacy: the statements below.

Genomes are listed in the order of their random IDs, so the bundle doesn’t reveal the original file order.

Leaves the laptop (the bundle): for each genome, a masked alignment of 120 bacterial (or 53 archaeal) marker proteins (about 5,000 amino acids); marker counts and the list of which markers were found; the genetic code; the domain call; the contig count; the SHA-256 of the renamed FASTA. Also the tool versions, reference-data hashes, and the number of inputs left out. The contig count is a mild identifying feature: with the genome size class it can help match a bundle to a known assembly. The bundle’s own privacy text says so.

Stays on the laptop: DNA sequences (except the k-mer samples in sketches, if you add them), genes and proteins outside the markers, file names, contig names, folder paths, and the mapping from opaque IDs back to your genomes.

Leak scan. After the manifest is written, prep searches every bundle file, manifest.json included, decompressed, for: the original genome IDs, file names and contig headers (whole header and first word); the work, data and input folders and every input file path, each as given, made absolute, and with symlinks resolved; $HOME in the same three forms; and, as whole words only, $USER, $LOGNAME and the host name. With --out, it scans what it wrote again (for a .tar or .tar.gz, the tar headers and every entry, decompressed). On any hit it deletes the bundle (or the --out copy) and fails, so a bundle that fails the scan cannot be sent by mistake.

Names and paths shorter than 6 characters (user and host names: shorter than 3) are not scanned for, because they would match by chance inside alignments and marker lists. They cannot reach the bundle by construction: GTDB-Tk only ever sees the renamed files (<random id>.fna, contigs <id>_c<n>), so none of its outputs can contain an original name; the bundle files are GTDB-Tk’s outputs plus a manifest that gtdbprep builds from opaque IDs, counts and hashes; and the commands recorded in it have local paths replaced by <work> and <GTDBTK_DATA_PATH>. The scan is a second check of that, not the mechanism.

What the alignment still says. It identifies the organism; that is its purpose. For a novel lineage it contains partial sequences of conserved housekeeping proteins, which may matter before publication. The FASTA hash lets someone who already has the same genome confirm it is in the bundle; it reveals nothing to anyone else.

Sketches are not anonymous. With --sketch, the bundle also carries a skani sketch per genome. It holds about 1 in 125 of the genome’s 15-mers with their positions and 1 in 1,000 of its 21-mers: a sparse but real sample of the DNA. Names inside it are opaque (gtdbprep sketches the renamed files), but the k-mers remain. prep prints this warning and writes it into the manifest. Don’t send sketches for unpublished or sensitive genomes. The design describes a route that keeps the ANI screen local (split GTDB-Tk, section 4).

classify_wf runs the ANI screen first and never runs identify/align on genomes it assigns to a species, so their msa_percent and translation_table are N/A. A plain gtdbtk classify given a bundle that contains every genome fills those two fields in instead. That was the only difference in the first round trip. gtdbprep classify therefore:

  1. runs GTDB-Tk’s own ANI screen (ANIScreener.run_aniscreen, because GTDB-Tk 2.8 has no ani_screen subcommand) on the renamed genomes. It runs under the Python that runs gtdbtk: the environment’s python3 with --env, otherwise the interpreter named in the shebang of the gtdbtk script on PATH (conda’s #!/bin/sh + exec form included). The call refuses any GTDB-Tk version other than 2.8.0, since the API is internal, and refuses genome IDs it was not given; gtdbprep then checks every assigned ID is in the bundle’s manifest;
  2. stages a copy of the bundle without the assigned genomes (rows removed from every TSV, records from every MSA), which is what classify_wf’s align folder looks like;
  3. runs gtdbtk classify --genome_dir <renamed> --align_dir <staged> [--full_tree];
  4. maps opaque IDs back to the original GTDB-Tk genome IDs (replay/summary.original_names.tsv).

A placement server would do the same in steps 1 and 2, with the ANI screen fed by sketches.

All subprocesses run with LC_ALL=C. Under some host locales, Bioconda’s pplacer aborts in glibc’s locale loader (_nl_intern_locale_data assertion), and the C locale changes no output.

prep reads markers/, masks/ and metadata/. Their SHA-1 equals GTDB-Tk’s REF_HASHES for R232 in GTDB’s 25 MB mockup archive (checked by prep on every run). GTDB-Tk also opens taxonomy/gtdb_taxonomy.tsv, but only to refuse input genome IDs equal to a GTDB accession. Opaque IDs (g + 16 hex digits) never are, so the mockup’s 5-line stub works. gtdbprep records its hash and notes the REF_HASHES mismatch without stopping. These four folders total 77 MB. With only them on disk, prep produced byte-identical alignment rows to a run against the whole mockup. dblock add gtdbtk-r232-client && dblock fetch fetches exactly this subset and checks it against REF_HASHES (see dblock); pass the path dblock env prints to --data.

classify and verify need the full package (or the mockup for tests), as GTDB-Tk does.

2026-11-10, GTDB-Tk 2.8.0 (Bioconda: Prodigal 2.6.3, HMMER 3.4, pplacer 1.1.alpha19, skani 0.4.0 for GTDB-Tk’s own ANI screen), skani 0.3.1 built from its tag for sketches, R232 mockup database (md5 59512f60fa48010647977460bb855666, see GTDBTk#726), 4 threads on a 12-core Linux laptop. Genomes: GTDB’s test genome G021783475 (1.91 Mb, 138 contigs), E. coli K-12 MG1655 (4.64 Mb) and P. aeruginosa PAO1 (6.26 Mb), gzipped.

Round trip (gtdbprep verify): 3/3 genomes identical to classify_wf. The merged summary.tsv is identical line for line once rows are sorted. The test genome is assigned by the ANI screen, and the other two are placed with pplacer (on the mockup’s 5-genome tree, so the classifications themselves are not meaningful). The renamed genomes’ alignment rows are byte-identical to classify_wf’s own user MSA. The whole verify (prep, classify from the bundle, classify_wf) took 78 s wall.

Bundle part Bytes (3 genomes) Per genome
bac120.user_msa.fasta.gz 7,499 2.5 KB
marker summaries (bac120 + ar53, plain text) 5,782 1.9 KB
translation table, filtered/failed lists 63 <0.1 KB
manifest.json 5,794–6,343 ~0.3 KB per genome entry + ~5 KB fixed
folder total, no sketches 19,197–19,687 6.4–6.6 KB
.tar.gz, no sketches 11,201 3.7 KB
skani 0.3.1 sketches 205,666 / 491,686 / 699,402 1.9 / 4.6 / 6.3 Mb genome

Tests: 37 unit tests (renaming, GTDB-Tk genome IDs, gzip input, non-FASTA inputs, leak scan of folders and tarballs, user/host/path needles, manifest round trip and tamper detection, SHA-256 vectors, GTDB-Tk’s sha1_dir order including symlinks, tar reproducibility, gzip header rewrite, file modes, --out checks, summary comparison, dropping ANI-assigned genomes, the ANI-screen interpreter, version and ID checks against a stand-in gtdbtk Python package). One of them runs the whole prep against a stand-in gtdbtk shell script, with spaces in the input, data and environment paths, a refused work folder with a space, two bad inputs, and a local name planted where only the manifest would carry it. One integration test, ignored by default, runs verify on GTDB’s test genome folder, copied into a folder whose name has a space and joined by an empty input file. It checks every genome is identical, that the empty input is reported as left out, that no original name is in the .tar.gz, and that classify from the .tar.gz gives classify_wf’s rows:

GTDBPREP_ENV=/path/to/gtdbtk-env GTDBTK_DATA_PATH=/path/to/release232_mockup \
GTDBPREP_TEST_GENOMES=/folder/with/test_genome_G021783475.fna.gz \
GTDBPREP_SKANI=/path/to/skani-0.3.1 \
cargo test --release -- --ignored

The ANI species screen without the 85 GB database (ani)

Section titled “The ANI species screen without the 85 GB database (ani)”

gtdbprep ani runs GTDB-Tk’s ANI screen (the first step of classify_wf, which gives species names) on a laptop. It does not need the 85 GB skani database. How it works and what was measured: The ANI species check on a laptop.

End to end, for GTDB R232:

# 1. Reference data, once: the screen index (2.1 GB download) and GTDB-Tk's taxonomy and
# radii (the placement mirror's lite entry, 175 MB).
dblock add gtdbtk-r232-ani-screen gtdbtk-r232-placement-lite
dblock fetch
dblock env # prints both folders
# 2. GTDB-Tk 2.8.0 with skani pinned to 0.3.1 (the version the R232 database was made with)
micromamba create -n gtdbtk -c conda-forge -c bioconda python=3.12 gtdbtk=2.8.0 skani=0.3.1
# 3. The screen
gtdbprep ani --genomes mags/ -x fna --work ani-work \
--index <store>/gtdbtk-r232-ani-screen/ani-screen-r232 \
--data <store>/gtdbtk-r232-placement-lite/release232 \
--env ~/micromamba/envs/gtdbtk --skani ~/micromamba/envs/gtdbtk/bin/skani \
--source /shared/gtdbtk/release232/skani/database/sketches.db

--source says where sketches come from; there is no default. Without a copy of the skani database, --source gtdb reads them from GTDB’s server (see below).

The results are GTDB-Tk’s own files, in ani-work/out/classify/ani_screen/: gtdbtk.bac120.ani_summary.tsv, gtdbtk.ar53.ani_summary.tsv and gtdbtk.ani_screen_unassigned_hits.tsv.gz.

GTDB-Tk’s usual summaries. ani-work/gtdbtk_data is then a complete GTDB-Tk data folder: links to the placement-lite data plus a small skani database of the fetched references. With it as GTDBTK_DATA_PATH, gtdbtk classify_wf runs its ANI screen on that database and writes the usual classify/gtdbtk.bac120.summary.tsv (isotable reads it), in seconds for genomes the screen assigns. Genomes it doesn’t assign go on to tree placement, which needs the placement setup on a laptop.

From an AI agent. irit-mcp has this as the gtdbprep_ani tool, recorded and replayable like the others, then gtdbtk_classify_wf with data_path: <work>/gtdbtk_data (irit-mcp).

What it downloads per genome. Every reference that passes skani’s marker screen has to be fetched, because GTDB-Tk’s files list all the close references, not only the species call. That was 2 to 1,941 references per genome in the tests (B. subtilis 23, E. coli 420, P. aeruginosa 1,090, a 400 kb piece of E. coli 1,941), about 0.51 MB each. With the default source (GTDB’s archive, --source gtdb, see below) each reference costs about 2.9 MB of download on average, so roughly:

Genome References From GTDB’s archive (--source gtdb) From an unpacked sketches.db
B. subtilis 168 23 about 70 MB 12 MB
E. coli K-12 420 about 1.2 GB 0.2 GB
P. aeruginosa PAO1 1,090 about 3.2 GB 0.56 GB
400 kb fragment 1,941 about 5.7 GB 1.0 GB

These are estimates from the measured average (9.3 GB for 3,166 references). Genomes from the same genus share most references, and all runs share one cache (below): a second run of the same genomes fetched nothing and took 18 s, and needs no --source.

If you only need the species, --species-only fetches far less. It fetches only the references that could give the species call (see below). Measured with --source gtdb and an empty cache: M. tuberculosis H37Rv 1 reference instead of 377 (2.9 MB instead of 1.23 GB), E. coli K-12 18 instead of 420 (61 MB instead of 1.32 GB), B. subtilis 168 14 instead of 21. The species call is the same; what can be missing are rows of the other two files (below). In crowded genera the saving is smaller: five genomes from Streptococcus, Bacillus cereus, Burkholderia, Pseudomonas and Klebsiella needed 542 references instead of 2,694 (1.68 GB instead of 8.07 GB), with the same calls and the same related references.

--source gtdb is opt-in, at GTDB’s request. GTDB publishes the skani database only inside its 60.8 GB .tar.gz. gtdbprep ani reads sketches out of that file on GTDB’s server with HTTP range requests, through the seek index that comes with the screen index (a checkpoint every 4 MiB). Each sketch costs the compressed bytes from the checkpoint before it, so about 5.7 times the sketch itself, and the requests load GTDB’s server. GTDB’s maintainers asked that this be opt-in and that fetched sketches be cached, never fetched twice (GTDBTk#727); they don’t plan to publish skani/database/ unpacked. So don’t run large batches this way: if you have the full package on a shared disk, point --source at its sketches.db.

How it works:

  1. skani 0.3.1 sketches the genomes, with the database’s parameters.
  2. The genomes are screened against the screen index: every reference’s marker count and 32-bit fingerprints of its marker k-mers. This keeps every reference skani search would compare in full (and perhaps a few more).
  3. Only those references’ full sketches are fetched, by HTTP range requests, into the cache (--cache, default $XDG_CACHE_HOME/irit/gtdbprep-ani or ~/.cache/irit/gtdbprep-ani, shared by all work folders). Each is checked against a hash in the index, in the cache too.
  4. They are written as a small skani database, and GTDB-Tk’s own ANIScreener runs against it. So the species rule (closest reference with AF >= 0.5, within its radius) and the output files are GTDB-Tk’s.

Options:

  • --source has no default. It is needed only when some sketch is not in the cache.
  • --source gtdb (opt-in) reads the sketches from GTDB’s own gtdbtk_r232_data.tar.gz, through the seek index in the screen index folder (gz.idx). --gtdb-url points at another copy of the same archive (a URL or a local path).
  • --source <path or URL of sketches.db> reads an unpacked sketches.db, local or hosted.
  • --source '<url>/sketches.db.{part}' --part-size N reads one split into numbered parts.
  • --data needs taxonomy/, radii/ and metadata/ (the gtdbtk-r232-placement-lite mirror has them). --env is a GTDB-Tk 2.8.0 environment.
  • --skani must be the version the database was made with (0.3.1 for R232), or pass --allow-skani-version. Bioconda’s GTDB-Tk 2.8.0 brings skani 0.4.0 unless you pin it.
  • --fetch-jobs N (default 4, at most 16): range requests in flight at once. 1 fetches one at a time, as earlier versions did. Failed requests (connection errors, a cut-off body, 408, 429 and 5xx) are retried up to 5 times with growing waits; a request that gets no answer for 60 s, or whose body arrives slower than 32 KiB/s, is cut off and retried. Each sketch is checked and written to the cache as soon as it arrives, so an interrupted run keeps what it fetched.
  • --species-only fetches only the references that could give the species call. See below.
  • --full-db <skani/database> is for testing: it runs the same GTDB-Tk call on a full database.

A genome skani makes no sketch of is left out of the screen, not fatal. skani skips assemblies too short to sketch, such as a 407 bp assembly from 5× reads. gtdbprep prints gtdbprep: warning: … it is not screened, screens the rest, and lists that genome with - for its markers. If no genome can be sketched, the run fails.

<work>/screen.tsv lists every reference the screen kept for each genome: its shared marker count, both marker counts, the count --species-only requires, and whether it was used.

--species-only. GTDB-Tk assigns a species from the closest reference with AF >= 0.5, and only if that reference’s ANI reaches its own radius. No radius in R232 is below 95. A reference that can’t reach ANI 95 can therefore never give the call. If it isn’t the closest, it doesn’t matter. If it is the closest, every other reference is also below 95, so the genome is unassigned either way. --species-only drops references whose marker count shows they can’t reach that ANI, with a wide margin:

  • it keeps a reference if the genome and the reference share at least the number of marker k-mers expected at ANI 93 (2 points below the smallest radius, read from radii/gtdb_radii.tsv) and AF 0.5, less 4 standard deviations (a binomial count). That is 10.9% of the smaller marker set (356 of 4,000 markers), against 0.9% in skani’s own screen;
  • on six test genomes (882 references screened in), every reference with ANI >= 95 and AF >= 0.5 was kept, the closest one to the limit with 4.7 times the shared markers required, and GTDB-Tk’s ani_summary.tsv came out byte-identical to the full run’s for all five assigned genomes.

What you lose: gtdbtk.ani_screen_unassigned_hits.tsv.gz (every skani hit of genomes the screen does not assign) lists only the references that were fetched. For a test genome with no species within reach, that was 2 rows instead of 14. GTDB-Tk’s classify reads this file for its notes on genomes placed in the tree (closest reference, near misses), so those notes can differ too; the placement and the classification itself don’t use it. The “other related references” column of ani_summary.tsv lists the 50 closest references with AF >= 0.5, so it loses a row only if a dropped reference had AF >= 0.5. No related reference was lost on any of the ten assigned test genomes (for one, two tied references came out in the other order); on the five from crowded genera, every dropped reference had AF below 0.5 (the highest ANI among them was 88.24). Measurements: The ANI species check on a laptop.

It prints, per genome, the number of marker k-mers and of references screened in, and the bytes fetched. Peak memory was 1.9 GB for 9 genomes (3,166 references); GTDB-Tk’s own screen on the full database took 14.7 GB.

Making the screen index (maintainers; needs the full database once):

gtdbprep ani-index build --db release232/skani/database --out ani-screen-r232 --release r232
curl -s "$URL" | gtdbprep ani-index gz-scan --input - --out scan --url "$URL"
gtdbprep ani-index gz-attach --index ani-screen-r232 --scan scan --every 4

gz-scan reads the archive once (it can share one download with tar -x) and keeps a checkpoint every --spacing bytes (default 1 MiB) of sketches.db. gz-attach --every N keeps every N-th checkpoint: a smaller index, but more data downloaded per sketch.

Privacy. Nothing about the genomes leaves the laptop. The requests name byte ranges of the public reference database, so the server can see which references were fetched, and so roughly which taxa the genomes belong to.

Tree placement on the placement-only data (place)

Section titled “Tree placement on the placement-only data (place)”

Genomes the ANI screen doesn’t assign to a species need tree placement. gtdbtk classify_wf can’t do that on the gtdbtk-r232-placement-lite data: it aligns genomes against GTDB’s reference MSA (7.9 GB), which that mirror leaves out. Placement itself doesn’t need the MSA. gtdbprep place runs the route that does work: GTDB-Tk’s identify, align --skip_gtdb_refs, then classify without the ANI screen, with gPlace instead of pplacer, under a memory cap. Every step is GTDB-Tk’s (GTDB-Tk, Chaumeil et al.), and the placements are gPlace’s (by GTDB-Tk’s author; identical to pplacer’s on 59 genomes, gPlace vs pplacer). gtdbprep only arranges the calls and checks them. Cite GTDB, GTDB-Tk and gPlace for results.

gtdbprep place --genomes bins/ -x fa --work place-work \
--data <store>/gtdbtk-r232-placement-lite/release232 \
--env ~/micromamba/envs/gtdbtk --cpus 6 --max-memory 6G \
--skip-assigned ani-work # optional: the --work folder of `gtdbprep ani`

What it does:

  1. Links the input files into <work>/genomes/, since GTDB-Tk refuses genome paths with spaces, and lists them with GTDB-Tk’s own rules, so genome IDs match the ones ani reports. With --skip-assigned, it reads <ani work>/assigned.json and the screen’s ani_summary.tsv files, and unlinks the genomes the screen assigned: they need no placement.
  2. gtdbtk identify, then gtdbtk align --skip_gtdb_refs.
  3. gtdbtk classify --pplacer_cpus N, run through GTDB-Tk’s own main() after setting options.skip_ani_screen = True. That is the flag classify_wf sets after its own ANI step; GTDB-Tk 2.8 has no command-line option for it, and the placement-only data has no skani database. The call refuses any GTDB-Tk version other than 2.8.0.
  4. gPlace as the placer. GTDB-Tk has no option for choosing the placer (GTDBTk#728). It runs pplacer -m WAG -j N -c REFPKG -o OUT.json MSA by name, so place writes a pplacer script into <work>/bin/ and puts it first on PATH. The script runs gplace place --max_memory <cap> with the same arguments and logs each call to <work>/logs/gplace-calls.log. Only pplacer --version goes to the real pplacer (GTDB-Tk logs it). Two checks: before classify, the Python that runs GTDB-Tk confirms that pplacer resolves to that script; afterwards, every pplacer.*.json placement file under <work>/classify must be the output of a logged gPlace call. Otherwise the run fails.

--max-memory (default 6G) goes to gPlace’s --max_memory. gPlace then splits a tree into tiles when the whole placement wouldn’t fit. At 6G every R232 tree peaked at about 5 GiB (gPlace vs pplacer). With the full bacterial tree (--full-tree true, not in placement-lite), don’t go below about 5.2G. --pplacer-cpus (default 1) is gPlace’s -j. --gplace names the program if it isn’t the environment’s or the one on PATH.

Outputs in --work:

Path What
place.tsv one row per input genome (below)
classify/ GTDB-Tk’s own output: gtdbtk.bac120.summary.tsv (and ar53), trees, logs, the placement files in classify/intermediate_results/pplacer/
identify/, align/ GTDB-Tk’s identify and align output
place.json the run record: tool versions (GTDB-Tk, gPlace), options, counts, times, number of gPlace calls
logs/ one log per command, and gplace-calls.log

place.tsv columns: genome, classification, method, gtdbtk_method, note, pplacer_taxonomy, red_value, msa_percent, translation_table, closest_reference, closest_reference_ani, closest_reference_af, warnings. method is:

  • placement: placed in the tree; the rest of the row is GTDB-Tk’s summary row (gtdbtk_method is its classification_method, closest_reference its closest_placement_reference; with no ANI screen these are N/A);
  • ani_screen: skipped because gtdbprep ani assigned it; the classification and the closest reference (with its ANI and AF) are the screen’s;
  • not_placed: GTDB-Tk returned a row but no placement, for example Unclassified Bacteria for a genome with too little of the marker alignment (see warnings);
  • no_result: GTDB-Tk returned no row (see identify/gtdbtk.failed_genomes.tsv).

The command prints the same table, then counts and times. It exits 0 on success and 2 on any failure, with the reason on stderr. If every genome was assigned by the ANI screen, it runs no GTDB-Tk step and writes the table from the screen alone. The work folder must be new, empty, or one an earlier place made (marked by place.json): a rerun replaces place’s own files there and nothing else.

Installing gPlace. gPlace 0.1.1 is not on PyPI or Bioconda (they have 0.1.0, as of 2026-11-10), and place was tested only with 0.1.1. On the test laptop it was built from source with pip into the GTDB-Tk environment, which needs a C compiler (system gcc 16.2.1 there):

micromamba create -n gtdbtk -c conda-forge -c bioconda python=3.12 gtdbtk=2.8.0 skani=0.3.1
micromamba run -n gtdbtk pip install \
git+https://github.com/cmc-aau/gPlace@695a401e0b54cc1bde55a5e1498a0697899b15ea
micromamba run -n gtdbtk gplace check_install # "Installation OK ... bit for bit"

Tested (2026-11-14): see the run below.

2026-11-14, GTDB-Tk 2.8.0 and gPlace 0.1.1 (695a401) in one micromamba environment, gtdbtk-r232-placement-lite, on the 12-core laptop. The input was the 23 MetaBAT 2 bins from Crystal Geyser (MAGs on a laptop), with --skip-assigned set to the gtdbprep ani work folder from that run, --cpus 6, --max-memory 6G. The whole command ran in a systemd scope with MemoryMax=8G and MemorySwapMax=0.

  • 20 bins were skipped as ANI-assigned. The other three were placed, with 3 gPlace calls: the backbone, class-level tree 6 and class-level tree 1.
  • Same classifications as the earlier run: bin.12 Unclassified Bacteria (3.6% of the MSA), bin.13 g__Sideroxyarcus;s__, bin.16 g__OLB19;s__. GTDB-Tk’s gtdbtk.bac120.summary.tsv was byte-identical to that of the run that used a separate shim and launcher outside gtdbprep, and so were the placements in all three placement files.
  • Peak memory 5.60 GiB (the scope’s memory.peak, 6,017,150,976 bytes), no OOM kill. gPlace tiled all three trees; its largest estimate was 5.7 GiB, for class-level tree 1.
  • Wall time 11.3 min: identify 17 s, align 3 s, classify 652 s. The machine was shared, with a load average of 20 to 27 on 12 cores the whole time. The earlier run of the same classify took 270 s at a load of about 1.7, so most of the difference is likely contention.
  • Mockup only. The round trip ran against GTDB’s mockup database (5 reference genomes, full tree). The real R232 package (split backbone and class-level trees) is untested: that needs the ~100 GB package on a large-memory machine (the phased plan, step 2).
  • Three bacterial genomes. No archaea, no translation table 4, no genomes below 10% alignment, no Prodigal failures, no batch where every genome is ANI-assigned, no borderline bac/ar calls.
  • The ANI screen still needs the genomes. classify screens the renamed FASTA files locally, so it runs where the genomes and the skani database both are. (gtdbprep ani runs the screen on the laptop without the database; classify doesn’t use it yet.) Sketches are produced and pinned to skani 0.3.1, but nothing consumes them yet: the server-side shim that maps sketch queries back to opaque IDs (design, section 4) is not built.
  • --place_species is not supported (its classify_wf behaviour differs and is untested).
  • The ANI-screen step calls GTDB-Tk’s internal Python API (ANIScreener.run_aniscreen), which may change in a later GTDB-Tk release. The call is pinned to GTDB-Tk 2.8.0: classify refuses other versions until the API has been checked against them.
  • Placement-side provenance for classify (pplacer/gPlace version, full-package hash) is not written next to the results yet; only the laptop side has a manifest. place records the GTDB-Tk and gPlace versions in place.json, but no reference-data hash.
  • place does not write a merged GTDB-Tk summary. GTDB-Tk’s summary.tsv in <work>/classify holds only the placed genomes; ANI-assigned genomes are merged in place.tsv, not in GTDB-Tk’s format. place was tested on bacteria only (no archaea, no --full-tree), and with gPlace 0.1.1 only.
  • Marker summaries are plain text inside a folder bundle (1.9 KB per genome); use .tar.gz to send them compressed.
  • Linux and macOS only (random IDs come from /dev/urandom; GTDB-Tk doesn’t run on Windows).
  • Tool sources. Bioconda installs skani 0.4.0 with GTDB-Tk 2.8.0; --sketch refuses anything but 0.3.1, which has to be built from its tag.
  • Reference download. The gtdbtk-r232-client entry checks GTDB’s mockup archive against its published MD5 (corrected on 2026-11-11, GTDBTk#726) and the extracted folders against REF_HASHES.