Import wizard

The import wizard ingests CSV or Excel files of show data into the catalog. It handles the messy parts — varied column headers, fuzzy publisher names, external artwork URLs, near-duplicates against existing shows — and lands a clean, deduped collection on the other side.

When to use it

• Onboarding a new publisher or network — bulk-loading their full rate card.
• Adding several dozen shows at once where typing each one would be tedious.
• Migrating shows out of an existing spreadsheet system into Catalog as the source of truth.

For one-off additions, just create a row directly in the grid — the import wizard is overkill below ~10 shows.

Opening it

From the shows grid (admin view), click Import in the toolbar. The wizard opens as a modal.

File requirements

• CSV (.csv) or Excel (.xlsx, .xls)
• Max 10 MB
• First sheet only (Excel)
• Headers can be on any row — the AI detects which row is the header.
• Metadata / definition rows above the headers are OK; they’re skipped.

The six steps

Step 1 — Setup

Pick the target organization the shows will belong to. When network attribution is relevant, a “Network attribution” section also lets you pin a single network for the whole file up front.

Step 2 — Upload

Drag a file onto the dropzone or click to browse. The file is parsed immediately:

• CSV uses PapaParse with header detection and UTF-8 / windows-1252 fallback.
• Excel uses ExcelJS. Formula cells use their evaluated value; Excel dates become ISO strings.

If parse errors happen, they’re listed for review but the wizard still advances — most parse errors are recoverable.

Step 3 — Process

One step now runs the whole AI pipeline, with a ConsoleLog showing live progress throughout:

Header & network detection. AI detects which row holds the headers. If the file carries multiple networks, a network-column picker appears and each unique network name is matched against existing organizations:

Artwork rehost. External artwork URLs are downloaded and re-uploaded to Frequency storage so they’re served from our CDN — not the original publisher’s server, which might disappear. URLs process one at a time ([1/N] Rehosting 'Show Name'...); failures are left as-is and can be retried.

Transform. AI converts each raw row into a structured Show record:

• Maps varied column names to canonical fields using the show schema’s aliases annotations.
• Cleans up casing, whitespace, locale-formatted numbers, etc.
• Returns a per-cell confidence score so you can spot weak transformations.

Step 4 — Map columns

Review the source-column → field mappings the AI chose, with a confidence indicator per mapping. Exclude any column that mapped badly before it pollutes the import.

Step 5 — Review (duplicates)

Two passes of fuzzy matching run on the transformed batch:

1. Within the import — finds rows that probably refer to the same show (e.g. “The Joe Rogan Experience” vs “Joe Rogan Experience”).
2. Against existing shows — finds rows that match shows already in your catalog.

Matching is weighted: name 0.50, network 0.30, host_name 0.20. The threshold is user-adjustable with three presets:

Per pair, you choose:

• Merge — opens the Merge Strategy modal (see below). The “loser” row is removed from the import.
• Dismiss — mark this pair as “not duplicates”. Persisted in cata_dismissed_duplicate so the same pair won’t flag again in future imports.
• Skip — defer the decision; both rows continue separately.

Once every pair is resolved, the Import button enables.

Merge strategy

When you merge a pair, an AI-recommended per-field choice is shown — for each field, the AI picks which value (or combination) should win. You can override any field choice before confirming.

Step 6 — Finalize

Clicking the import button runs:

1. Slug generation with collision avoidance (<name> → <name>-2, …).
2. Org resolution — creates any “New” orgs from the Process step and reuses matched ones.
3. Field-level merges applied to existing shows (from the duplicate review).
4. Any remaining un-rehosted artwork URLs get a final rehost attempt.
5. Categories auto-created if they don’t exist; show ↔ category links populated.
6. Batch insert of all new shows, followed by an RSS enhance pass over the new rows.
7. A collection is created automatically, named <filename> — <date>, with every imported and merged show linked.
8. The wizard closes and navigates to the new collection.

The wizard is minimizable — a dock titled “Import in progress” keeps the status visible while you work elsewhere in the app.

Auditing past imports

Admins can review every past import from the Admin dashboard (documented on the internal site):

• Filename, file size, row counts (imported / merged / skipped / errored)
• Who ran it and when
• Resulting collection (one click to open)

Failed imports keep their import_job row with the error message for debugging.

Edge cases & gotchas

• Header detection wrong — if the AI misidentifies the header row, override it manually from the dropdown on the Process step.
• Artwork rehost fails — leaves the original URL on the show. Re-import or run RSS Enhance later to populate.
• Duplicate review long — at Loose threshold with a large file, you may have 100+ pairs to triage. Use Merge all suggested if you trust the AI’s choices, then audit afterwards.
• Org name typos — if you create a “new” org during import that turns out to be a typo of an existing one, an admin can merge orgs after the fact.
• Mid-step abandonment — closing the modal mid-import is safe before the final insert. After insert begins, partial completes will land in the DB.
• Test fixtures — apps/catalog/test-data/ ships small CSV files for manual testing: a 3-show happy path, a metadata-row file, a messy-data file, a missing-fields file, and a duplicate-with-seed file. QA: start there.