NormalizeMessageID now returns empty for a value containing CR or LF.
A source_id with an embedded line break would render as a malformed
In-Reply-To/References header on outbound replies. The IMAP path never
sees such values; the HTTP API can, so the helper rejects them. An empty
result leaves the column NULL, so a bad value degrades threading rather
than producing invalid mail headers.
Add an optional source_id to createConversationRequest and messageReq and
store it on the created contact message, mirroring the IMAP ingestion path
which is the only place source_id is otherwise populated.
BuildEmailThreadingHeaders builds every outgoing reply's References and
In-Reply-To from prior messages' source_id. A desk fed over the API, as
ours is, therefore references only its own previous sends, so the customer's
mail client has nothing of theirs to thread on and every reply arrives
looking standalone. Letting the API set source_id fixes this for any API-fed
inbox without a direct database write.
The value is normalized (whitespace and angle brackets stripped) since
source_id is stored unbracketed and the brackets are re-added when composing
the threading headers. An empty value leaves the column NULL as before, so
existing callers are unaffected.
enmime's charset detection was overriding the declared UTF-8 charset on long mostly-ASCII bodies, storing accented text as mojibake (issue #512). Parse with detection disabled so the declared charset wins, bump enmime to v2, and swap the abandoned jaytaylor/html2text for its inbucket fork. Pin behavior with charset tests across locales, encodings, and malformed inputs.
A batch of correctness fixes from a code review of the help center branch.
Public pages:
- hide the article author in JSON-LD when the theme hides the byline, so
the author name no longer leaks in structured data.
- make agent and AI assistant avatars public on upload (and backfill AI
assistants in the v2.7.0 migration) so author avatars don't 401 for
anonymous visitors.
- return the same 401 for missing and existing private uploads to an
anonymous caller, so /uploads can't be used to probe which private
media exists.
- switch the bare /hc/{slug} home redirect from 301 to 302 since the
default locale is mutable and a permanent redirect gets cached.
- match the request host against the public URL by hostname, ignoring
any port, so robots.txt and sitemaps resolve on custom domains.
- fix the lazy-load regex so an img with data-loading or alt text like
"loading=x" still gets loading="lazy" and decoding="async".
- stop logging searches shorter than the minimum length, which never
actually run, so they don't pollute the no-result insights.
Admin and data integrity:
- take the help center row lock in UpdateArticle, MoveArticle,
CreateCollection and UpdateCollection, and run the collection depth
check inside that transaction, so concurrent edits can't create
duplicate slugs or overshoot the max depth.
- default the collection locale before validating the parent, so
creating a child collection without a locale no longer 400s.
- keep a hand-edited help center slug when the name is edited again.
- render the Snippets page title in plural.
- release the feedback dedup key when the insert fails, so a retry
isn't swallowed as a duplicate.
Batch of fixes from a review of the help-center branch.
Slugs. A long title made a slug longer than the 200 char cap, so the
save failed with a slug error on a form that has no slug field. The
"-2" dedup suffix could also push a slug past the cap after validation
ran, which locked the article out of every later edit. Slug generation
now truncates to fit, and collections get the same dedup pass articles
already had. Accented titles kept losing letters, so accents now fold
to their ASCII base. Titles in scripts with no ASCII form still fall
back to a random slug.
Locales. Removing a language from a help center stranded its
collections and articles. They had no admin UI path and no public URL,
but stayed in the AI index. Dropping a language that still holds
content is now rejected. Only languages the help center allows today
are checked, so an existing orphan can still be saved.
Theme. The luminance check only parsed 3 and 6 digit hex while the
sanitizer accepts 4 and 8 digit too, so an 8 digit header colour fell
back to dark text on a dark header. The free text colour fields had no
validation, so a bad value was silently thrown away on save while the
toast said it worked. The article editor was missing --hc-accent-ink,
which made button links invisible while editing.
Embeddings. The chunker changed the text it emits but the fingerprint
did not change, so nothing already indexed was ever re-chunked. The
chunker version now feeds the fingerprint. Deleting a collection or a
help center left the cascaded articles' vectors behind until the next
sweep, so those now clean up on delete.
Public API. The public JSON endpoints returned author name and avatar
even when the theme had authors turned off.
Media and search. Anonymous /uploads requests now hit a rate limit
before the DB lookup, and served files carry a one day cache header.
S3 redirects are no-store because a presigned URL expires. Public
search now enforces the same two character floor the typeahead uses,
so a one character query cannot seq-scan every article body.
Each help center gets an optional public_url. When set, canonical links, hreflang
alternates, og:url, the sitemap and robots.txt point at that host instead of the
app root URL. Stored media URLs are rewritten to root-relative paths so logos,
favicons, header images and article images resolve on whichever host serves the
page. The URL is validated on both the form and the backend, and trailing slashes
are trimmed.
Also fixes issues found in code review:
- the help center template used a locale link as the data root inside the
alternates loop, which broke every public page on a help center with two or
more locales
- creating an article posted the clicked collection id, so changing the
collection in the sheet filed the article in the wrong place
- the collection field's error message was hidden when the locale had no
collections, so Create did nothing with no error shown
- switching the header type or unticking popular articles dropped those theme
values on save, because vee-validate unsets fields that unmount
- closing a new collection sheet with Esc left the parent id set, so the next
collection you edited got re-parented
- a table row with th labels in the first column was treated as a header row
when preparing article content for embeddings
- the img tag regex stopped at a > inside an attribute value and injected
loading/decoding in the middle of an attribute
Resolved the AI embedding conflicts by keeping this branch's generic embedSource path and layering main's tag indexing on top. Search now covers snippets and help articles, tags stay excluded. Dropped SetContentID since main's link-message-media query stamps content_id itself.
Public pages were missing most of what a search engine needs. They now serve
a sitemap index at /sitemap.xml plus a robots.txt, and every page carries a
canonical URL, hreflang alternates, an x-default, and JSON-LD. Article pages
also send published and modified times. Search pages and the markdown view
are marked noindex. Requests from bots no longer bump the view counters, so
the insights numbers reflect real readers. Offscreen images in articles are
lazy loaded.
Public page text is now translated per locale instead of always using the
desk language, and pages with more than one translation show a language
switcher.
In the admin, collections and articles can be dragged to reorder, and an
article can be dragged into a different collection. Both save through new
sort order endpoints. Collections can have a lucide icon, picked from a new
icon picker backed by a sprite sheet. The landing page has layout options
for card grid or one per row, cards per row, and which details to show.
The help center list is a data table now, so HelpCenterCard is gone. The
tree page moved into AdminSplitLayout, its actions sit on their own row
below the breadcrumb, and it gained visit site and expand/collapse all.
Transcripts flattened links to "text ( url )", so the model copied that shape and drafts came back with the URL in brackets instead of a link. HTML to text for the LLM now writes "[text](url)", which Markdown2HTML turns back into an anchor.
AI grammar fix rewrote the draft as plain text, so links in the reply box were dropped. It now sends the editor HTML and asks the model to keep tags. Conversation transcripts, the AI agent history and copilot context also stripped link URLs when converting HTML to text, so the model never saw them. They now keep links as "text ( url )".
Review pass over the help center commit. Bug fixes:
- Image-only or video-only articles were rejected on save because the
editor blanked the content when it had no text.
- An article could be moved into a collection in another help center.
Slugs are only unique per help center, so two articles could end up
on the same public URL.
- The article editor fired the "agent is typing" indicator into
whatever conversation was last open, which the contact could see.
- Unsupported locales like /hc/x/zz-ZZ returned an empty 200 page with
a canonical URL, so crawlers saw unlimited URLs. They 404 now.
- Article feedback was accepted for paused help centers.
- French pages announced lang="en".
- Editing a help center popped validation errors mid-keystroke.
Shared code that was duplicated:
- Help articles had their own copy of the snippet embedding lifecycle.
The copy was missing the wait group, the concurrency cap and the
generation gating, so article embeds were killed at shutdown, all hit
the provider at once, and a slow embed could overwrite a newer edit or
bring back a deleted article. Both content types now go through one
embedSource lifecycle. Fingerprints are unchanged, so nothing
re-embeds on deploy.
- Callout and collapsible CSS was written twice, once for the editor and
once for the public page. They now share article-content.css, so the
editor stays WYSIWYG.
- The excerpt was computed three ways. The page uses the stored one.
Less work at runtime:
- The admin tree no longer sends every article's HTML. The edit sheet
loads the one article it needs. Publishing an article used to
re-download the whole help center.
- Embedding reconcile no longer reads every article body every minute.
- The tree view fetched the help center twice on load.
Cleanups: one update-article route instead of two, media privacy comes
from the model type instead of a bool threaded through the handler, a
LinkListField component replaces three copies of the same repeater,
TreeNode renders the article row once, and 12 duplicate or dead i18n
keys are gone.
Agents can now publish a public knowledge base per help center.
Backend:
- new internal/helpcenter package for help centers, collections and articles
- admin JSON API behind a new help_center:manage permission
- public /hc/{slug} pages, JSON API, search and sitemap, all rate limited
- migration v2.7.0 for the new tables
AI:
- published articles with the AI flag on get embedded, so the agent answers
from them. Unpublishing or clearing the flag removes the embeddings.
Media:
- media rows now carry a private flag. Help article images are public so the
public pages can serve them without auth, and only agents with
help_center:manage can upload them.
Frontend:
- admin help center list, tree view and article editor
- the old TextEditor is split into ArticleEditor and ConversationEditor over a
shared useTextEditor composable, so article-only tools like callouts,
collapsibles and YouTube embeds stay out of the reply box
Final review pass before taking the AI agent branch live.
Knowledge base:
- Text not wrapped in a block tag was never collected, so prose around a
table or list never reached the index. The assistant answered "no
relevant information" for questions the snippet covered.
- Blocks over the token limit were truncated and the remainder dropped. They
are split into several chunks now.
- Trimming an oversized block ran one rune at a time and re-tokenized the
whole string each step. A large table took minutes. It uses a binary
search now.
- Overlap text was not escaped, so a sentence containing markup swallowed
the rest of the chunk.
- SVG and template text no longer reaches the index.
AI agent:
- Verification codes are capped per address and per conversation. The cap
was per conversation only, so a customer correcting a mistyped email was
told to check an inbox that never got a code.
- Livechat verification sends synchronously. A queued send returned nil even
when SMTP failed, so a failure counted as a sent code.
- Queued jobs drain on shutdown and hand off to a human instead of being
dropped with no reply.
- Deleting an assistant no longer moves resolved and closed conversations
into the fallback team.
- Image decode is capped at 25 MP. The old bound allowed a 400 MB decode per
attachment.
Auth and admin:
- A blank OIDC client secret no longer overwrites the stored one. Blank id
or secret is rejected instead.
- OIDC token exchange uses the SSRF guarded client with a timeout.
- Renaming a tool auth header no longer attaches the secret of whichever row
now sits at that position.
- Clearing embedding dimensions no longer refills 1536 on the next load,
which pushed a wrong value to the provider on the next save.
- Copilot conversation lookups filter by access before capping at 10.
When the agent built conversation history and mined FAQs, it used the
raw message text, which included the full quoted reply chain from every
email. That wasted tokens and confused the model with old back-and-forth.
Add emailquote.go to strip quoted blocks. HTML messages get their quote
containers pruned (gmail, yahoo, protonmail, outlook markers, and
blockquotes); plain text gets trailing ">" lines and "On ... wrote:" /
"Original Message" markers trimmed. If stripping leaves nothing (a
quote-only reply or forward), we fall back to the full text so the
message is not dropped. Add the matching protonmail_quote selector to
the frontend hide-quoted-text styles so the two stay in sync.
Also fix knowledge base chunking: plain text with no block structure was
kept as one chunk and could overflow the model limit. It now packs into
size-bound pieces on sentence boundaries, and oversized atomic blocks are
flushed and truncated on their own so they can never sneak through.
Fix a reindex race: snippet embedding runs outside the lock, so a slower
job from an older edit could commit stale vectors after a newer edit.
Split Reindex into embed (lock-free) and commit (locked), and gate the
commit with a per-snippet generation counter so only the latest edit
wins. Delete drops the counter so an in-flight job can't re-insert
vectors for a deleted snippet.
Clear the DB avatar reference before deleting the avatar media file. The
old order deleted the file first, so a failed DB update left the DB
pointing at a missing file and a broken image.
Also handle unchecked errors flagged by errcheck (tx.Rollback,
html.Render, fmt.Fprintf), guard capToTokens against a negative limit
that would panic, and stop logging the full import URL since it can
carry credentials.
Snippet import: new "Import from URL" flow fetches a page and stores its
readable content as a snippet. Extraction uses the mackee/go-readability
library (Mozilla Readability port) and outputs Markdown, so nav/footer
boilerplate is dropped. The snippet list shows the source, and the edit
dialog is now wider with a taller content box.
Summarize: new "Summarize with AI" action on a conversation calls the AI and
adds the result as a private note. It shows an info toast right away so the
user knows it started, since the call can take a few seconds. This adds an
"info" toast variant that any feature can use.
Assistant languages: assistants can be given a list of allowed reply
languages. The assistant replies in the customer's language when it is one of
them, otherwise it falls back to the first. The preview also lists the
knowledge sources it used.
Cleanup: replace hardcoded gray/zinc/white colors with theme tokens
(text-muted-foreground, text-foreground, bg-accent) across several components.
Replies from the AI agent are now converted from markdown to HTML with
goldmark before queueing, so bold, links, and lists render properly in the
widget, agent app, and email. The prompt now allows simple markdown. Raw
HTML in model output is escaped by goldmark, and both frontends sanitize
on render anyway.
Other fixes bundled in:
- validate avatar type and size before creating or updating an assistant,
and roll back the assistant if the avatar upload fails after create
- return 404 from agent update and API key endpoints for AI assistant
identity users, and hide assistants from mention and SLA user pickers
- reserve the autonomous assistant's built-in tool names so custom tools
cannot shadow them
- apply resolve after the reply is posted so the CSAT survey follows the
answer instead of preceding it
- unassign the assistant on handoff even when the fallback team is the
same team
- count reopens by status category instead of status name, and exclude
CSAT messages from the turn cap
- split oversized wrapper divs into child blocks when chunking KB HTML
instead of truncating them
- return 404 when soft-deleting an already-deleted agent, and keep AI
assistants (which have no email) visible in the compact users list
Adds two agent-facing AI features: a copilot chat panel in the conversation
sidebar and a generate-reply button in the reply box. Both run an agentic
tool-calling loop whose first tool searches the knowledge base.
Snippets are chunked and embedded on save, then searched in memory with
brute-force cosine similarity (no pgvector). Providers are split into completion
and embedding types. Both are OpenAI-compatible and the API key is encrypted at
rest. Admins can also register custom HTTP tools the model can call.
A new admin AI settings page covers provider config, snippets, and tools. The
v2.6.0 migration and schema add the ai_knowledge_base, embeddings, and ai_tools
tables plus the ai_providers type column.
Swap k3a/html2text for jaytaylor/html2text in stringutil.HTML2Text.
k3a returns empty on HTML with orphan tags (e.g. stray </a>), which
made text_content blank for some emails.
- Update all SQL queries to add missing columns
- Update the create conversation API to allow setting the initiator of a conversation. For example, we might want to use this API to create a conversation on behalf of a customer, with the first message coming from the customer instead of the agent. This param allows this.
- Minor refactors and clean up
- Tidy go.mod
- Rename structs to reflect purpose
- Create focus structs for scanning JSON payloads for clarity.
- New permission `contacts:manage`
- Views for contact list and single contact view.
- Ability to block contacts that in inturn stops new messages from the contact.
- Make all DB transactions that run dynamically generated SQL query `readonly`
- Rename `/admin/teams/users` to `/admin/users/agents/` for consistency. (ref #50)
- Fix UI glitches for long emails (refs #54)
- Fix empty created_at date for agents admin table (refs #51)
- Migrations for v0.6.0
Before this feature the only way to create a conversation was by adding inbox and sending an email.
Agents first search contacts by email, see a dropdown select an existing contact or fill a new email for new contact.
The backend creates contact if it does not exist, creates a conversation, sends a reply to the conversation.
Optinally assigns conversation to a user / team.
fix: Replies to emails create a new conversation instead of attaching to the previous one.
Was not happening in gmail, as gmail was sending the references headers in all replies and I missed this completely. So when libredesk searches a conversation by references headers it worked!
Instead the right way is to generate the outgoing email message id and saving it in DB. This commit fixes that.
There could be more backup strategies like putting reference number in the subject but that can be explored later.
chore: new role `conversatons:write` that enables the create conversations feature for an agent.
chore: migrations for v0.4.0.
malpou