Frontend for the new omni audio API
(`/v1/omni_gen/{models,generate,cost}/audio`).
### New pages
- **Webapp**: `/create-audio` page (desktop layout + mobile
Generate/History tabs with settings drawers), added to sidebar.
- **Desktop**: new `AUDIO` tab ("Create Audio") with `PromptBoxAudio`,
registered in the app menu / TopBar. Audio enqueues over HTTP and polls
the jobs feed (Tauri task queue untouched).
### Models & promptbox
- Suno Music / Remix / Sounds / Sample + Seed Audio 1.0, fetched live
from `GET /v1/omni_gen/models/audio` with per-model capability gating.
- Always-visible **style prompt** row (Suno tags), instrumental /
keep-lyrics / loop toggles, BPM + musical key popover (Suno Sounds),
sample rate + speed/volume/pitch tuning popover (Seed Audio).
- Audio/image reference uploads with server rules mirrored client-side
(remix/sample require exactly 1 audio track; Seed Audio can't combine
audio + image refs).
- One request can return multiple job tokens (Suno multi-clip) - every
token becomes its own pending card.
### Audio playback in galleries
- New **`@storyteller/ui-audio-player`** lib (wavesurfer.js v7):
waveform + seek, play/pause, volume, duration - only one player plays at
a time.
- Lazy waveform decode (IntersectionObserver + peaks cache) and a
no-waveform `<audio>` fallback if CORS/decode fails.
- Audio cards/rows added once in the shared `ui-generation-list`, so
both hosts' grid + list views get them.
- Audio playback in both lightboxes (webapp lightbox + desktop
lightbox-modal); Library gets an **Audio** filter tab and includes audio
in "All"; downloads keep the right extension.
### Shared plumbing
- `OmniGenApi` gains audio types + `getAudioModels` / `generateAudio` /
`estimateAudioCost`.
- `useOmniGenAudioModels`, request builder, enqueue + cost hook shared
via `@storyteller/omni-gen` (used by both hosts).
- `useGenerationJobs` moved from the webapp into `ui-generation-list`
(with re-export shim) and taught the `audio_generation` category - the
desktop audio feed reuses it.
- Suno model names/descriptions/creator icon added to `model-list`.
### New shared lib: `@storyteller/omni-gen`
- The `omni-gen-hooks.ts` file was copy-pasted between `artcraft-webapp`
and
`artcraft-website` and had drifted (website copy was missing outage
toasts,
carried a stale hardcoded display-name map, and had weaker icon
resolution).
- Extracted the newer (webapp) version into a new lib
`frontend/libs/omni-gen`
(generated via `nx g @nx/react:library`): singleton session cache,
`useOmniGenImageModels()` / `useOmniGenVideoModels()`, outage message
constants.
- App-local toasts are injected via `setOmniGenErrorNotifier()`, called
once in
each app's `main.tsx`, since the hooks are singleton-cache based and
each app
has its own toast component.
### Consolidated model metadata into `@storyteller/model-list`
- `getModelDescription()` / `getModelInfo()` (tagline + info-blurb
fallbacks)
moved to `ModelDescriptions.ts`; API values still always win.
- `getModelDisplayName()` gained an optional `fullName` param (API name
wins);
the app-local wrapper is gone.
- Both apps' local `getModelCreatorIconPath()` replaced with the
existing,
stronger `getCreatorIconPathForModelId()` (checks `ALL_MODELS_LIST`
first,
then prefix match). The loader's duplicate `CREATOR_BY_PREFIX` map now
reuses
the same shared `MODEL_ID_PREFIX_CREATORS` table.
### Dead code removed
- `modelReconciliation.ts`: deprecated `reconcile*()` helpers (never
called) and
the empty, retired `BACKEND_TO_TAURI_*` alias maps.
`buildModelsFromListing()`
dropped its alias-map indirection accordingly (backend ids == tauriIds
since
the 2026-07 id migration). `modelCreatorFromBackend()` moved inline into
the
loader, its only consumer.
- Six unused static `*_PAGE_MODEL_LIST` constants in
`model-selectors-for-pages.tsx`
(superseded by the `use*PageModelList` hooks in #1710). The two
`IMAGE_TO_3D_*`
constants stay: 3D models are not served by the omni backend.
- Untracked the accidentally committed
`apps/artcraft/tsconfig.tsbuildinfo` and
gitignored `*.tsbuildinfo`.
### Website-visible improvements (from adopting the newer hooks)
- Model outage toasts now appear on the website (previously
webapp-only).
- Current display names from the canonical map (previously-unmapped ids
like
`gpt_image_2` and Midjourney models rendered as raw ids).
- Model taglines/descriptions now render in the website model picker.
- Midjourney and SwitchX creator icons now resolve.
## Summary
Reworks the desktop (Tauri) app's Create Image / Create Video pages to
match the webapp,
extracts the webapp's generation gallery into a shared library used by
both apps, embeds the
model selector into every prompt box, adds a folders sidebar + marquee
selection to the gallery
modal, restyles the login/pricing/credits modals to the webapp look, and
turns the Apps page
into the default landing tab.
### Shared generation-list lib
- Extracted the webapp's generation feed into
`@storyteller/ui-generation-list`: list + masonry
grid views, pending/failed/completed rows and cards, thumbnail handling,
gallery/prompt data
hooks, and the shared grid/list view store + toggle. Host-specific
actions (recreate,
make-video, share, download, toasts) inject via render props — webapp
markup unchanged, its
old files are now thin wrappers.
### Desktop create pages
- Prompt box fixed at the bottom, `max-w-5xl` like the webapp; pages
renamed to
"Create Image" / "Create Video".
- Generation feed: in-progress/failed from the Tauri task queue, history
from the library API,
rendered as grid or list (toggle in the TopBar next to credits). Typed
Tauri completion events
push finished media in instantly; the poll reconciles and dedupes.
- Row/card quick actions (recreate, make video, copy link, download)
backed by a shared
`desktopMediaActions` module extracted from the TopBar gallery wiring.
- Hero + background act as a splash that fades when the feed loads;
bottom gradient behind the
prompt box. Batch generations show "Generating N images" on the single
pending chip.
- Video prompt box now expands vertically as far as the image prompt box
(viewport-relative).
### Embedded model selector
- `ClassyModelSelector` gained `variant="embedded"`: a compact pill that
opens the webapp-style
rich-list popover while keeping the desktop's provider selection.
Embedded on all five
prompt-input pages; floating selectors removed; legacy "10 sec." badges
dropped.
### Gallery modal
- Left sidebar lists the user's folders (drag-and-drop, right-click
management, new-folder
button) and shows on the Folders tab too, highlighting the active
folder's root branch.
- Marquee (drag-rectangle) selection in view mode, with edge
auto-scroll; starts only on blank
background so it doesn't interfere with drag-to-folder/canvas.
### Restyled modals (desktop)
- Login/signup: webapp 2-column layout (form left, Vimeo showcase
right), no Google button
(Google auth isn't wired for Tauri), `artcraft-icon` logo, eye toggles.
The dismiss button is
removed so login is required; the post-auth Discord + Success onboarding
steps are kept, as is
the existing Tauri auth flow.
- Pricing + credits: match the webapp visually (unified monochrome plan
cards with
Most Popular / Best Value highlight frames; `#161618` credit-pack cards
with Popular badge).
All billing stays on the existing Tauri `invoke` calls — no billing
logic changed.
### Apps page / navigation
- The Apps page is now the first tab and the default landing tab, with a
Home icon. Restyled to
the webapp home: "What will you craft today?" hero, Create / Edit
sections, gradient-hover
cards sized to their content, full-width scrollbar. The TopBar
quick-menu header reads
"Create" instead of "Generate".
### Fixes
- Video Editor no longer crashes on the desktop app (`Tooltip must be
used within
TooltipProvider`): the editor shell now mounts its own Radix
`TooltipProvider`. The webapp got
one app-wide via `SidebarProvider`; the Tauri app had none, so panel
tooltips threw.
### Misc
- `getCreatorIconPathForModelId()` added to `@storyteller/model-list`
(model id → creator icon
path, desktop/webapp aware).
- refactored all markdown content as a shared library for use across
both the ArtCraft website and the app (FAQ, tutorials, news)
- added a news page to the website
- added a news & updates option to the app's bottom right help context
menu
Instructions to add and build News Posts for both the site and app is in
``NEWS_POSTS.md`` but in short:
- To add a new news post, create an .md file in
``libs/markdown-content/src/lib/content/news/`` folder
- Restart the artcraft-website dev server after adding new posts (or in
/frontend directory, run ``node scripts/generate-news-json.mjs``
manually) - this will update the public ``news.json`` file on the
website.
- This is just a temporary workaround - this way if the ArtCraft app
itself hasn't updated, it'll still be able to load the latest public
``news.json`` file from the website to show within the app.
shared component for tutorials
app:
- tutorial modal accessible from any page on the app on the bottom right
- plays right in the modal through youtube embed
- it remembers the play progress and saves it in the zustand state so in
that current session if modal is opened again, it will continue where
the user left off
website:
- navbar updates (home and tutorials link)
- tutorial page with category buttons to filter by product or show all
- plays through youtube embed on website page
- pricing modal with tiers: free, plus, pro and max
- set up and ready to integrate with stripe
- small bug fix with the library modal buttons being squished
- upgrade button in TopBar is commented out for now
- Create 3d model from image works now (accessed via wand button next to
the + asset button).
- Modal pops up, or if you dont have a FAL api set in settings it'll pop
up an action reminder to set it first.
- You can choose image from library (opens gallery select mode) or
upload an image from computer.
- Pressing create enqueues the image to 3d and closes the modal.
Spinning loading toast on the top right says Generating... until it
finishes.
- When done, currently it just pops up in the library and you can drag
it onto the scene yourself.
**Current bugs/missing features:**
1. No 3D object image thumbnail in gallery
2. If you open up the gallery and try add the newly generated 3d object
to the scene, it will be stuck at "loading" text because of a CORS error
(not sure why). Pressing the button to refresh the gallery before
dragging it in seems to fix that though.
- reusable model selector component
- uses zustand with separated states for each tab
Image model list:
- GPT Image 1 (GPT-4o)
- Flux Pro Ultra
- Recraft 3
- Flux.1 Kontext
Video model list:
- Kling 2.0
- Runway
- login now flows automatically smoothly to the next step after login
- settings modal: misc/audio panes styling and toggle switch
- account settings disconnect sora account button
remaining TODO:
- link logout sora endpoint
- login modal is a shared component that
- finished but needs checking session if openai/sora logged in or not
- replaced video for 3d (it had a black screen at the end)
BFlat