Metadata-Version: 2.4
Name: 3dsem
Version: 0.1.17
Summary: Classify point clouds with pretrained 3D semantic segmentation models, from one command
License: MIT
License-File: LICENSE
Requires-Python: >=3.9
Description-Content-Type: text/markdown

# 3dsem

Classify point clouds with pretrained 3D semantic segmentation models, on
your own machine, from one command.

```
pip install 3dsem

sem install dales-utonia
sem infer dales-utonia tile.las
```

The classified `.laz` appears next to your input file, with per-point
classification, confidence, and every original dimension carried over.

## What you need

- An NVIDIA GPU with a current driver (Windows 527.41+, Linux 525.60.13+)
- Python 3.9 or newer
- About 10 GB of disk per model

`sem install` shows the download size, the license, and a GPU check, then
asks once before any bytes move. It is safe to interrupt and resumes where it
stopped. After install, inference runs fully offline.

## Models

`dales-utonia` is trained on [DALES](https://arxiv.org/abs/2004.11985) aerial
LiDAR and predicts 7 classes: ground, vegetation, vehicle, powerline, fence,
pole, building.

`dales-hag-utonia` predicts the same 7 classes from the same data, with
height above ground added as an input. It scores higher on held-out DALES
scenes (0.86 mIoU against 0.98 overall accuracy) and is the one to reach for
first.

Both models are licensed CC-BY-NC 4.0 (non-commercial), shown before
install.

`sem models` lists what is available and installed. Bare `sem` opens an
interactive picker.

## How do I make it more accurate?

Use a preset. Each one turns on more of the same three ideas: predict the
scene from several augmented views and vote (test-time augmentation), tile
the scene a second time at a half offset so no point sits on a tile edge
(overlapped voting), and clean up the labels afterwards (smoothing, island
removal, geometry rules).

- default: one pass with light smoothing
- `--med`: 4 voting views, overlapped tiling, island removal. Roughly 4x the
  time.
- `--high`: 6 views including flips and rotations, stronger smoothing,
  geometry rules for ground/vegetation/building confusions, per-class
  probability fields. Roughly 6x.
- `--ultra`: 9 views and the strongest smoothing. Roughly 9x.

An explicit flag always wins over a preset, so `--ultra --no-sieve` means
ultra without island removal.

## How accurate is it on my data?

If you have a tile whose classification you trust, classify a copy and
compare the two:

```
sem eval tile_pred.laz tile_truth.laz
```

No inference runs; it reads the `Classification` column of both files and
prints, per class, IoU, precision, recall and F1 with point counts, then
overall accuracy, mIoU, macro F1, Cohen's kappa, and a confusion matrix.
Files written by `sem infer` carry their own class legend, whatever codes
that run actually used, so rows read `2 ground` rather than bare numbers.
A file without a legend shows plain codes, and `--names 1=ground,4=water`
names any code by hand.
The files must be the same points in the same order, which is what
`sem infer`'s own export gives you. `--pred-field` and `--truth-field`
name a different column on either side.

When the truth file uses different codes than the model, `--map` translates
them first, and `--ignore` keeps codes out of the scoring:

```
sem eval tile_pred.laz tile_truth.laz --map 3=5,4=5 --ignore 0,1
```

folds the truth's low and medium vegetation into the model's single
vegetation class and skips points that were never classified. `--map
3,4=5` says the same thing: several truth codes go on the left of the `=`,
one predicted code on the right.

To see whether the cleanup passes help or hurt on your data, run the
inference with `--keep-raw`: it also exports a copy from before
post-processing into a `raw/` folder inside the job, under the same names.
Score each against the same truth and compare:

```
sem eval tile_dales-hag-utonia_predictions/ truth/
sem eval tile_dales-hag-utonia_predictions/raw/ truth/
```

A whole batch works the same way with two folders: files pair by name
(`tile_pred.laz` matches `tile.laz`), each tile gets a summary line, and
the totals pool every point before computing IoU, so a big tile counts
for more than a small one. A predicted file with no truth partner stops
the run and is named. Folders pair with folders and files with files,
never mixed.

## How do I make it faster?

`--low` is a single pass with no cleanup, the fastest option. If a preset is
mostly what you want, `--no-overlap` drops its second tiling pass, which is
about half its extra cost.

## Big buildings come out patchy or cut through. Why?

The scene is processed in square tiles, typically 50 m on a side. An object
bigger than one tile is predicted in pieces, and the pieces can disagree.
Two fixes that combine well:

- `--chunk-xy 100` makes the tiles bigger, so a large building fits in one.
  Costs GPU memory.
- `--overlap` (on automatically with `--med` and up) predicts a second pass
  at a half offset and votes, which removes most seam artifacts.

## It ran out of GPU memory

Lower `--chunk-xy`, try 35 and then 25. Smaller tiles need less VRAM, and
the extra seams they create are what `--overlap` is for.

## Poles or powerlines are disappearing

Presets from `--med` up turn on island removal, which absorbs clusters
smaller than 10 points into their surroundings. Thin objects are exactly
small clusters. Keep the filter but make it gentler with
`--sieve-min-pts 5`, or turn it off with `--no-sieve`.

## Can I hide the model's low-confidence guesses?

`--unclass 0.6` exports every point below 60% confidence as unclassified
instead of its best guess. Bare `--unclass` uses 0.5.
To see where the model is unsure, `--extra-dims` adds per-point diagnostic
fields: `entropy` (0..1 uncertainty), `margin` (gap between the top two
classes), `probs` (one probability field per class, grows the file), and
`ood` (the raw scores behind the gates below). Combine them freely:
`--extra-dims entropy,margin`.

## It confidently labels things it has never seen

A crane or a boat has no class to land in, so the model puts it in the
nearest one it knows, often with high confidence. Confidence alone will not
catch that, because the model is confident and wrong. Two other scores will.

`--unclass-gmm` compares each point against how the training classes actually
looked to the model and unclassifies anything too far from all of them. Bare,
it uses the threshold the model was shipped with; give it a number to
override. `--unclass-maxlogit 4.0` catches the opposite case, points where no
class drew much evidence at all. They find different mistakes, so using both
is normal.

To choose your own numbers, run once with `--extra-dims ood`, which writes
the raw scores into the output file so you can see where your data sits, then
set the thresholds and re-export. Re-exporting never re-runs the model.

## Every model in one folder ran, but the gates did nothing

`--unclass-gmm` needs a model that shipped with those statistics. If a model
predates them, that flag stops with an error naming what is missing rather
than silently exporting ungated results. `--unclass` and `--unclass-maxlogit`
work on any model.

## Part of my file is already classified correctly

`--input-field preserve:Classification:2,9` keeps the codes your file already
carries for those points and writes the model's prediction everywhere else.
They are read from the source file at export, so inference, the unclassified
gates and the cleanup passes never touch them.

Kept codes are written as your file spells them. When a kept code means
something different in the model's output (your water 9 has no model
class), add `--preserve-map 9=2` to write it as ground instead, and the
export warns whenever a kept code is not one the model uses, so a foreign
code never slips through silently.

The export reports how often the model disagreed with what you preserved,
which is a quick check on both.

You can also do it afterwards, on a file that is already classified:

```
sem preserve tile_pred.laz tile.las --keep Classification:2,9
```

That writes `tile_pred_preserved.laz` beside it. Same overlay, decided after
you have seen the result rather than before, with the same `--map 9=2`
option for codes that need translating. The two files have to be the same
points in the same order, which is checked on counts and coordinates. This
does not work with `--raw-codes`: the preserved codes are in your file's
scheme, so the output has to be on codes too, not model indices.

## Can I combine models?

`sem infer dales-utonia+dales-hag-utonia tile.las` runs every model in the
chain and merges their predictions with a vote: each model's per-class
probabilities are averaged, and the strongest combined evidence wins. Where
the models agree, the label sticks; exact ties go to the model you listed
first, so lead with your strongest.
The result carries an `agreement` field (what fraction of models agreed on
each point) and an `ens_member` field (which model drove each label). The
models must share the same class list. Each member also keeps its own
`tile_<model>_predictions` folder, so you can compare them individually.

## Can I run my own trained model?

If a training run left you a folder holding `final_model.pth` and
`run.json`, register it:

```
sem local-model C:\runs\my_run --name myrun
```

It appears in `sem models` and runs like any other: `sem infer myrun
tile.las`. Nothing is copied; the folder is used in place, and
`sem local-model --forget myrun` unregisters it without touching the
files.

The weights are yours but the environment is not: inference borrows the
installed env of a catalog model with the same backbone, so install one
that matches first. Everything else the run needs (grid, features,
classes) is read from the `run.json` beside the weights.

## My file has no coordinate system

It still runs. sem uses the coordinates exactly as they stand, reprojects
nothing, and warns that it is taking their unit on trust: everything below
(tile size, voxel grid, neighbourhoods) is a length in metres, so a file
already in metres is correct and a file in feet is off by that factor. Most
aerial LiDAR is already metre-projected, which is why this usually just works.

`--epsg 26917` declares the projection when you know it (use your own zone's
code), which silences the warning and makes the exported cloud georeferenced.

The one case that stops is lon/lat-shaped coordinates, where the whole scene
spans a fraction of a degree. A 50 m tile there would cover everything at
once, so that is not a slightly worse answer, it is no answer; declare a
projected CRS with `--epsg` and it proceeds.

Reprojection only ever happens to get your data into metres, from degrees or
from feet. A model is not tied to any particular CRS, only to the scale it was
trained at, so a cloud already in a metre projection is never transformed.

## My file is not a LAS. How does sem know what its columns mean?

It does not, and it will not guess. LAS and LAZ name their dimensions in the
format spec, so intensity, return number and classification are read straight
off a LAS with no help from you. Every other format (`.txt`, `.csv`, `.xyz`,
`.pts`, `.npy`, `.npz`, `.ply`, `.pcd`) leaves the meaning of a field entirely
up to whoever wrote it, so sem asks you:

```
sem infer dales-utonia scan.txt \
    --input-field xyz:1,2,3 --input-field intensity:5
```

`--input-field KEY:COLUMN[:VALUE]` says where one thing lives in your file,
repeated per entry. Columns are names or 0-based numbers.

- `xyz:A,B,C` and `rgb:R,G,B` take three columns
- `intensity:C`, `return-number:C` and `hag:C` take one
- `ground:C:V,...` and `preserve:C:V,...` add the value(s) in that column
  that mean it, because which column and which value in it are one idea

A model that wants a channel you did not name stops and lists the fields your
file actually has.

Nothing is inferred from a column's position or from a name that looks
familiar, because a file with `id,x,y,z` in that order and a file with
`x,y,z,id` are indistinguishable to anything except you.

## My colours come out black, or sem asks for --rgb-max

Point clouds store colour as 8, 10, 12 or 16 bit, and no format records which.
Guessing it from the brightest point in the scene turns a 12-bit cloud nearly
black and a dark 16-bit cloud into a blown-out one, so `--rgb-max` states the
full-scale value: `255`, `1023`, `4095`, `65535`, or `1` for float 0 to 1
colour. It is only needed when the model actually consumes colour; a model
that runs on intensity ignores the colour in your file and never asks.

## Height above ground

Models trained with a HAG channel compute one at staging, using a ground
raster whose cell size comes from the linear unit your file's CRS declares: 2
metres, or whatever length equals 2 metres in your file's own unit. A cloud in
US survey feet gets a 6.56 foot cell, which is the same ground resolution.
Nothing is assumed about your units; they are read from the CRS.

`--hag-cell` overrides that when you want a different resolution. The height
error it costs is roughly the cell size times the local slope, so a smaller
cell buys accuracy on steep ground and costs memory. A cloud that declares no
projected CRS has no unit to read, so it stops until you either declare one
with `--epsg` or state the cell with `--hag-cell`.

`--ground-method` picks where the ground comes from (`smrf`, `csf`, `zmin`, or
`labels` when your file already marks ground). `--csf-rigidness` (1 steep, 2 moderate,
3 flat/urban), `--smrf-window` and `--smrf-cut` tune the ground filters. If
your file marks ground with a class code, say so in one entry:
`--input-field ground:Classification:2`. The column alone is not enough,
because it does not say which value in it means ground. More than one code
can count as ground: `ground:Classification:2,9` also folds water under
the ground surface, which keeps HAG flat across lakes and rivers.

## The cleanup rules are wrong for my data

The geometry rules only run when you ask for them and tell them what your
classes mean: `--rules --roles veg=vegetation,building=building`. sem no
longer decides that a class is vegetation because its name contains "tree".
Their thresholds are lengths in your scene's vertical unit and unitless
ratios, all settable: `--rule-ground-hag` (0.1), `--rule-lowveg-hag` (0.35),
`--rule-high-hag` (2.0), `--rule-planar-min` (0.55), `--rule-scatter-min`
(0.4).

## Controlling the exported class codes

Classes export under ASPRS codes where the spec has one, and user-definable
codes (64+) where it does not. For the shipped models that is ground 2,
vegetation 5, building 6, powerline 14, pole 15, vehicle 64, fence 65. A class
this table does not name takes the next free code at or above 64.

`--asprs-map ground=2,building=6` overrides any of them by name, and two
classes landing on one code stops the run rather than merging them silently.

`--unclassified-code` sets the code gated points receive. It defaults to 1,
ASPRS "Unclassified", which is free because no class maps onto it. Export
stops rather than letting that code collide with a real class.

`--raw-codes` exports each class as its raw model index instead. Those indices
mean something different in ASPRS (index 2 is your vehicle, but 2 is Ground in
the spec), so anything reading the file per the spec will misread it. Use it
when you control both ends.

## I don't have a GPU

`sem` classifies on your own machine and needs an NVIDIA GPU. There is no
cloud option here. Conversion and export are CPU work, but the model itself
is not.

## Can I rerun with different settings without reconverting?

Pass `--keep-npz` on the first run, then yes.

The output folder is a self-contained job, named after your file and the
model (`tile_dales-utonia_predictions/`). The staged `.npz` inside it hold
the converted channels, and they are what a re-run reuses. By default they
are deleted once the export is written, because most runs only want the
classified file. `--keep-npz` retains them.

With them kept: quality options (presets, TTA, cleanup, export) never
reconvert. Conversion options (`--epsg`, `--ground-method`, `--hag`, ...)
and changes to the input file itself reconvert automatically; identical
settings reuse the staged files. Every model keeps its own job folder, so
switching models never mixes results.

## Can I just name the output file?

Yes. An output ending in `.laz`, `.las`, `.txt`, `.csv` or `.ply` is the file
itself, and its extension picks the format, so `--format` becomes redundant:

```
sem infer dales-utonia tile.las out/pc.laz
```

The job folder is created beside it (`out/pc_dales-utonia_predictions/`), so
everything the run makes stays where you pointed. This is for a single input
file; a folder of tiles produces one result per scene, so give it a folder.

If you have set a default with `sem output`, a bare filename lands there:
`sem infer dales-utonia tile.las pc.laz` writes `<your dir>/pc.laz`. Put a
folder in the path and it goes exactly there instead.

## A whole folder of tiles?

Pass the folder. Every `.las/.laz/.ply/.pcd` inside becomes one job, and
results land in `<model>_predictions/` inside it.

## Where does everything live?

Downloads go to `~/.trainer` (set `TRAINER_HOME` to move them). Your data
and results never go there: each job is a folder next to your input, or
under a default you set with `sem output <dir>` and undo with
`sem output off`. `sem clean dales-utonia` removes one model;
`sem clean --all` removes everything sem ever downloaded.

## Can I look at the intermediate files?

Run with `--keep-npz`, or the intermediates are gone once the export is
written. Then `sem tolaz job/tile_input.npz` writes `tile_input.laz`
beside it. The xyz becomes the cloud, rgb the color, and every other
per-point channel a named field you can shade by in CloudCompare. That
works on the staged input as well as on predictions, so it is how you
see the features a model actually received, not just what it predicted.

## Every option

`sem infer --help` documents all of it. Add `--pick` to browse and edit
every option with arrow keys before running.

## Licensing

The `sem` tool is MIT licensed. Each model ships a `NOTICE.md` stating its
architecture credits and license terms; some models carry a non-commercial
restriction inherited from their pretrained components, shown before you
install.
