# AbstractMusic

`abstractmusic` is a local-first **text-to-music / text-to-audio** library designed to plug into **AbstractCore** as an optional capability plugin.

## Install

```bash
pip install abstractmusic
```

The base package is currently local-first and includes the ACE-Step runtime stack. The shared
framework profile aliases are available for composition:

```bash
pip install "abstractmusic[apple]"
pip install "abstractmusic[gpu]"
pip install "abstractmusic[all-apple]"
pip install "abstractmusic[all-gpu]"
```

These aliases do not add dependencies yet because the local stack is already in the base package.

## Quickstart (local generation)

```python
from abstractmusic import MusicManager
from abstractmusic.backends import AceStepV15Backend, AceStepV15BackendConfig

backend = AceStepV15Backend(config=AceStepV15BackendConfig())

mm = MusicManager(backend=backend)
wav_bytes = mm.t2m("uplifting synthwave with punchy drums", duration_s=10.0)
open("out.wav", "wb").write(wav_bytes)
```

## Quickstart (AbstractCore integration)

```python
from abstractcore import create_llm

llm = create_llm(
    # Any provider/model works here. The LLM does *not* synthesize audio.
    "ollama",
    model="qwen3:4b-instruct",
    music_backend="acestep",
    music_model_id="ACE-Step/Ace-Step1.5",
)

wav_bytes = llm.music.t2m("ambient lo-fi study music", format="wav", duration_s=10.0)
open("out.wav", "wb").write(wav_bytes)
```

## Notes

- Audio output baseline is **WAV** (no external codecs required).
- Model weights are downloaded on first use via the Hugging Face cache (same workflow as Diffusers-based vision).
- ACE-Step v1.5 loads a **pinned** Hugging Face revision by default for deterministic behavior. Override with `--revision` / `ABSTRACTMUSIC_REVISION` if needed.
- The ACE-Step backend vendors the checkpoint’s custom Transformers model code into `abstractmusic` so we do **not** use `trust_remote_code`.

## CLI / REPL

After installation, `abstractmusic` provides a small CLI:

```bash
# One-shot generation
abstractmusic --backend acestep t2m "ambient lo-fi study music" --out out.wav --duration 10

# Interactive REPL
abstractmusic --backend acestep repl
```

## Licensing note

- The default backend example uses **ACE-Step v1.5** (`ACE-Step/Ace-Step1.5`), tagged `license:mit` on Hugging Face. The vendored custom model code files carry **Apache-2.0** headers (both permissive).
- If you switch to `--backend diffusers`, **model licenses vary** by checkpoint. Choose a model compatible with your intended usage.

### macOS / Apple Silicon note (MPS)

Some Diffusers audio pipelines can fail on the `mps` device due to PyTorch backend limitations (typically during vocoder inference).
`abstractmusic` will **retry on CPU** with a clear warning (`#FALLBACK`) when it detects the known MPS channel-limit error.
To force CPU directly, use `--device cpu`.

For ACE‑Step v1.5 on MPS, `abstractmusic` defaults to **fp16** (bf16 disabled) to keep memory usage reasonable on typical unified‑memory Macs.
If you run into numerical issues, you can override with `--dtype float32` (at the cost of significantly higher memory use).
By default, ACE‑Step caps MPS memory to ~16 GiB by setting `PYTORCH_MPS_HIGH_WATERMARK_RATIO` (configurable via `--mps-max-memory-gb` or `--mps-high-watermark-ratio`).
In addition, ACE-Step text-encoder conditioning is executed on **CPU float32** on MPS builds as a compatibility fallback (`#FALLBACK`) to avoid known mixed-dtype MPSGraph kernel aborts; conditioning tensors are cast back to the model dtype/device before diffusion.
The ACE-Step backend defaults to `infer_method=ode` (turbo `fix_nfe=8`, upstream default) with `shift=3.0` (upstream schedule). For text2music conditioning, source latents are initialized from seeded random noise (instead of silence) to better match expected model behavior, and chunk masks default to zeros to avoid injecting constant features.
By default, the prompt is passed as-is (no SFT wrapper); set `use_sft_prompt=True` in config if you need the previous instruction/metas format.
If a run returns non-finite latents, `abstractmusic` retries once with the alternate infer method using an incremented seed (`#FALLBACK`) instead of writing a silent/invalid WAV.

For instrumental prompts (no explicit lyrics), ACE-Step uses a proper **null lyric condition** (mask=0) rather than synthetic placeholder lyric text.
Decoded waveforms are DC-centered before normalization to avoid one-sided/noisy artifacts from amplifying tiny decoder bias.

Upstream references:
- PyTorch MPS env var `PYTORCH_ENABLE_MPS_FALLBACK=1` (fallback to CPU when an op is unsupported): `https://docs.pytorch.org/docs/stable/mps_environment_variables.html`
- Example upstream issue tracking the specific MPS channel-limit error: `https://github.com/pytorch/pytorch/issues/144445`
