Commit Graph

48 Commits

Author SHA1 Message Date
malpou 5d84d5af5c Reject line breaks in source_id
NormalizeMessageID now returns empty for a value containing CR or LF.
A source_id with an embedded line break would render as a malformed
In-Reply-To/References header on outbound replies. The IMAP path never
sees such values; the HTTP API can, so the helper rejects them. An empty
result leaves the column NULL, so a bad value degrades threading rather
than producing invalid mail headers.
2026-08-29 11:08:19 +02:00
malpou 3b0d3d4415 api: accept source_id on conversation and message endpoints
Add an optional source_id to createConversationRequest and messageReq and
store it on the created contact message, mirroring the IMAP ingestion path
which is the only place source_id is otherwise populated.

BuildEmailThreadingHeaders builds every outgoing reply's References and
In-Reply-To from prior messages' source_id. A desk fed over the API, as
ours is, therefore references only its own previous sends, so the customer's
mail client has nothing of theirs to thread on and every reply arrives
looking standalone. Letting the API set source_id fixes this for any API-fed
inbox without a direct database write.

The value is normalized (whitespace and angle brackets stripped) since
source_id is stored unbracketed and the brackets are re-added when composing
the threading headers. An empty value leaves the column NULL as before, so
existing callers are unaffected.
2026-08-29 10:59:03 +02:00
Abhinav Raut a3d29011b6 fix mojibake in incoming quoted-printable emails
enmime's charset detection was overriding the declared UTF-8 charset on long mostly-ASCII bodies, storing accented text as mojibake (issue #512). Parse with detection disabled so the declared charset wins, bump enmime to v2, and swap the abandoned jaytaylor/html2text for its inbucket fork. Pin behavior with charset tests across locales, encodings, and malformed inputs.
2026-08-20 11:40:02 +05:30
Abhinav Raut 95346c7514 Merge branch 'main' into help-center 2026-08-13 01:30:39 +05:30
Abhinav Raut e853cd39b4 preserve unicode in attachment filenames and escape MIME headers
properly

Fixes #483
2026-08-12 01:28:11 +05:30
Abhinav Raut aa54bfa48d fix slug, locale, theme and embedding bugs in help center
Batch of fixes from a review of the help-center branch.

Slugs. A long title made a slug longer than the 200 char cap, so the
save failed with a slug error on a form that has no slug field. The
"-2" dedup suffix could also push a slug past the cap after validation
ran, which locked the article out of every later edit. Slug generation
now truncates to fit, and collections get the same dedup pass articles
already had. Accented titles kept losing letters, so accents now fold
to their ASCII base. Titles in scripts with no ASCII form still fall
back to a random slug.

Locales. Removing a language from a help center stranded its
collections and articles. They had no admin UI path and no public URL,
but stayed in the AI index. Dropping a language that still holds
content is now rejected. Only languages the help center allows today
are checked, so an existing orphan can still be saved.

Theme. The luminance check only parsed 3 and 6 digit hex while the
sanitizer accepts 4 and 8 digit too, so an 8 digit header colour fell
back to dark text on a dark header. The free text colour fields had no
validation, so a bad value was silently thrown away on save while the
toast said it worked. The article editor was missing --hc-accent-ink,
which made button links invisible while editing.

Embeddings. The chunker changed the text it emits but the fingerprint
did not change, so nothing already indexed was ever re-chunked. The
chunker version now feeds the fingerprint. Deleting a collection or a
help center left the cascaded articles' vectors behind until the next
sweep, so those now clean up on delete.

Public API. The public JSON endpoints returned author name and avatar
even when the theme had authors turned off.

Media and search. Anonymous /uploads requests now hit a rate limit
before the DB lookup, and served files carry a one day cache header.
S3 redirects are no-store because a presigned URL expires. Public
search now enforces the same two character floor the typeahead uses,
so a one character query cannot seq-scan every article body.
2026-08-08 23:24:05 +05:30
Abhinav Raut 6aa221a4fc merge origin/main into help-center
Resolved the AI embedding conflicts by keeping this branch's generic embedSource path and layering main's tag indexing on top. Search now covers snippets and help articles, tags stay excluded. Dropped SetContentID since main's link-message-media query stamps content_id itself.
2026-08-01 11:47:59 +05:30
Abhinav Raut 2e89586a11 send links to the LLM as markdown so drafts keep real hyperlinks
Transcripts flattened links to "text ( url )", so the model copied that shape and drafts came back with the URL in brackets instead of a link. HTML to text for the LLM now writes "[text](url)", which Markdown2HTML turns back into an anchor.
2026-07-31 18:02:22 +05:30
Abhinav Raut a5a029d894 fix lossy text conversions in AI features 2026-07-31 16:36:04 +05:30
Abhinav Raut 72c3f29830 fix hyperlinks getting lost across AI features
AI grammar fix rewrote the draft as plain text, so links in the reply box were dropped. It now sends the editor HTML and asks the model to keep tags. Conversation transcripts, the AI agent history and copilot context also stripped link URLs when converting HTML to text, so the model never saw them. They now keep links as "text ( url )".
2026-07-31 16:08:19 +05:30
Abhinav Raut 76db5aa851 fix help center bugs and share the embedding and article-content code
Review pass over the help center commit. Bug fixes:

- Image-only or video-only articles were rejected on save because the
  editor blanked the content when it had no text.
- An article could be moved into a collection in another help center.
  Slugs are only unique per help center, so two articles could end up
  on the same public URL.
- The article editor fired the "agent is typing" indicator into
  whatever conversation was last open, which the contact could see.
- Unsupported locales like /hc/x/zz-ZZ returned an empty 200 page with
  a canonical URL, so crawlers saw unlimited URLs. They 404 now.
- Article feedback was accepted for paused help centers.
- French pages announced lang="en".
- Editing a help center popped validation errors mid-keystroke.

Shared code that was duplicated:

- Help articles had their own copy of the snippet embedding lifecycle.
  The copy was missing the wait group, the concurrency cap and the
  generation gating, so article embeds were killed at shutdown, all hit
  the provider at once, and a slow embed could overwrite a newer edit or
  bring back a deleted article. Both content types now go through one
  embedSource lifecycle. Fingerprints are unchanged, so nothing
  re-embeds on deploy.
- Callout and collapsible CSS was written twice, once for the editor and
  once for the public page. They now share article-content.css, so the
  editor stays WYSIWYG.
- The excerpt was computed three ways. The page uses the stored one.

Less work at runtime:

- The admin tree no longer sends every article's HTML. The edit sheet
  loads the one article it needs. Publishing an article used to
  re-download the whole help center.
- Embedding reconcile no longer reads every article body every minute.
- The tree view fetched the help center twice on load.

Cleanups: one update-article route instead of two, media privacy comes
from the model type instead of a bool threaded through the handler, a
LinkListField component replaces three copies of the same repeater,
TreeNode renders the article row once, and 12 duplicate or dead i18n
keys are gone.
2026-07-25 15:50:47 +05:30
Abhinav Raut baff30fc28 add help center with public pages, admin management and AI article indexing
Agents can now publish a public knowledge base per help center.

Backend:
- new internal/helpcenter package for help centers, collections and articles
- admin JSON API behind a new help_center:manage permission
- public /hc/{slug} pages, JSON API, search and sitemap, all rate limited
- migration v2.7.0 for the new tables

AI:
- published articles with the AI flag on get embedded, so the agent answers
  from them. Unpublishing or clearing the flag removes the embeddings.

Media:
- media rows now carry a private flag. Help article images are public so the
  public pages can serve them without auth, and only agents with
  help_center:manage can upload them.

Frontend:
- admin help center list, tree view and article editor
- the old TextEditor is split into ArticleEditor and ConversationEditor over a
  shared useTextEditor composable, so article-only tools like callouts,
  collapsibles and YouTube embeds stay out of the reply box
2026-07-25 14:19:38 +05:30
Abhinav Raut dad99bf0ee render AI agent replies as markdown and fix assistant identity, handoff, and stats bugs
Replies from the AI agent are now converted from markdown to HTML with
goldmark before queueing, so bold, links, and lists render properly in the
widget, agent app, and email. The prompt now allows simple markdown. Raw
HTML in model output is escaped by goldmark, and both frontends sanitize
on render anyway.

Other fixes bundled in:
- validate avatar type and size before creating or updating an assistant,
  and roll back the assistant if the avatar upload fails after create
- return 404 from agent update and API key endpoints for AI assistant
  identity users, and hide assistants from mention and SLA user pickers
- reserve the autonomous assistant's built-in tool names so custom tools
  cannot shadow them
- apply resolve after the reply is posted so the CSAT survey follows the
  answer instead of preceding it
- unassign the assistant on handoff even when the fallback team is the
  same team
- count reopens by status category instead of status name, and exclude
  CSAT messages from the turn cap
- split oversized wrapper divs into child blocks when chunking KB HTML
  instead of truncating them
- return 404 when soft-deleting an already-deleted agent, and keep AI
  assistants (which have no email) visible in the compact users list
2026-07-12 23:12:44 +05:30
Abhinav Raut cfc032f387 add SplitName util and use it to parse email contact names 2026-07-10 17:07:50 +05:30
Abhinav Raut 870f66c229 fix: sanitize invalid UTF-8 in incoming email, fix widget scroll flicker 2026-05-29 21:00:48 +05:30
Abhinav Raut ddc82d9a1a fix: extract text from malformed email HTML
Swap k3a/html2text for jaytaylor/html2text in stringutil.HTML2Text.
k3a returns empty on HTML with orphan tags (e.g. stray </a>), which
made text_content blank for some emails.
2026-05-28 19:03:26 +05:30
Abhinav Raut f6a4ad537f fix auto-replies and csat fanning out to last message's cc list
Both now send to the contact only and carry meta.is_automated so the
reply-box prefill skips them and inherits CCs from the last human message.
2026-05-15 16:16:32 +05:30
Abhinav Raut ffa5a22f3f fix s3 avatar_url storing bucket path instead of /uploads/<uuid>
Trim spaces around reply to config
2026-04-24 20:12:04 +05:30
Abhinav Raut dce33238e2 exclude inbox reply_to from reply recipients to prevent self-loop 2026-04-24 19:14:55 +05:30
Abhinav Raut 3d2da2a11f add reply_to on email inboxes for routing replies off the SMTP From domain 2026-04-24 18:39:16 +05:30
Abhinav Raut 7958d7e212 Merge branch 'main' into feat/live-chat-channel 2026-01-14 00:33:51 +05:30
Abhinav Raut 27d929c302 refactor add util funcs to extract conversation uuid from to address 2025-12-26 03:15:07 +05:30
Abhinav Raut a63c073c15 fix strip conversation uuid from receipient address as it could be the inbox email with plus variant
- adds test cases for util functions that strip inbox email.
- update make file to include frontend unit tests
2025-12-24 16:08:25 +05:30
Abhinav Raut 22af66c44d Feat: match incoming messages to conversations using reference number in the subject.
refactor: update GetConversation method to include reference number parameter
2025-12-24 16:08:25 +05:30
Abhinav Raut bf1cf025e0 Merge branch 'main' into feat/live-chat-channel 2025-10-04 13:38:42 +05:30
Abhinav Raut 16ca6b6df7 Send batched unread messages in conversation continuity email after contact has not seen a conversation for configured amount of time, instead of sending a single message immediately when contact is disconnected on websocket.
Make the batching interval and time threshold for unread messages configurable
2025-10-03 01:54:17 +05:30
Abhinav Raut d7067bce7d feat: add email fallback / continutiy inbox feature for live chat inbox 2025-09-21 00:51:19 +05:30
Abhinav Raut 634fc66e9f Translate welcome to libredesk email subject
- Update all SQL queries to add missing columns

- Update the create conversation API to allow setting the initiator of a conversation. For example, we might want to use this API to create a conversation on behalf of a customer, with the first message coming from the customer instead of the agent. This param allows this.

- Minor refactors and clean up

- Tidy go.mod

- Rename structs to reflect purpose

- Create focus structs for scanning JSON payloads for clarity.
2025-08-28 00:34:56 +05:30
Abhinav Raut 6c9eca3d81 fix: do not computed bcc from latest message. 2025-05-11 20:26:33 +05:30
Abhinav Raut 07b185050e fix: empty recipients in automated replies
- Make recipients list from the latest message recipients for automated replies
2025-05-11 18:51:34 +05:30
Abhinav Raut 6b2be57049 fix: set correct recipients when a 3rd email is involved in conversation, link to thread discussing this - https://github.com/abhinavxd/libredesk/issues/74#issue-3021419913
refactor move recipient computation to /utils/email-recipients
2025-05-10 23:46:19 +05:30
Abhinav Raut b2a53b18d5 feat: manage contacts
- New permission `contacts:manage`
- Views for contact list and single contact view.
- Ability to block contacts that in inturn stops new messages from the contact.
- Make all DB transactions that run dynamically generated SQL query `readonly`
- Rename `/admin/teams/users` to `/admin/users/agents/` for consistency. (ref #50)
- Fix UI glitches for long emails (refs #54)
- Fix empty created_at date for agents admin table (refs #51)
- Migrations for v0.6.0
2025-04-10 04:13:11 +05:30
Abhinav Raut aeef7d4ad7 feat: configurable SLA alerts per SLA. 2025-03-21 23:23:03 +05:30
Abhinav Raut 8551b65a27 fix: set references header in all outgoing emails, set the last 20 messages.
feat: set conversation reference number in the subject of conversation for better thread matching.
fix: hide CSAT link from conversation last message.
2025-03-05 03:49:22 +05:30
Abhinav Raut 494bc15b0a feat: Enable agents to create conversations from the UI
Before this feature the only way to create a conversation was by adding inbox and sending an email.

Agents first search contacts by email, see a dropdown select an existing contact or fill a new email for new contact.

The backend creates contact if it does not exist, creates a conversation, sends a reply to the conversation.
Optinally assigns conversation to a user / team.

fix: Replies to emails create a new conversation instead of attaching to the previous one.

Was not happening in gmail, as gmail was sending the references headers in all replies and I missed this completely. So when libredesk searches a conversation by references headers it worked!

Instead the right way is to generate the outgoing email message id and saving it in DB. This commit fixes that.

There could be more backup strategies like putting reference number in the subject but that can be explored later.

chore: new role `conversatons:write` that enables the create conversations feature for an agent.

chore: migrations for v0.4.0.
2025-03-05 01:17:42 +05:30
Abhinav Raut 356a206110 fix: new incoming messages having 0 attachments in API responses, upload attachments first before sending out websocket updates.
- refactor: stringutils package remove unncessary `string` in function names.
- minor refactors.
2025-02-08 18:04:02 +05:30
Abhinav Raut d3c2cc2527 feat: add CSAT sending functionality and improve error handling in templates
feat: automation action to send CSAT.
- minor refactors.
2025-02-02 21:03:09 +05:30
Abhinav Raut 52e401c601 feat: email cc and bcc in conversation replies
- feat: adds cc and bcc inputs in conversation text editor.
- feat: mask conversation id in api responses.
- customize shadnc dialog to hide close button & cross icon.
2025-01-25 03:47:17 +05:30
Abhinav Raut 586c7355f9 feat: new automation fields for conversations - hours since created, hours since resolved.
- feat: adds new actions - send private note, send reply.
- minor fixes.
2024-11-03 21:47:47 +05:30
Abhinav Raut ed314eb1a5 feat: reset password flow
- feat: notification emails including - welcome email, assigned conversation email, reset password email.
- feat: adds outer app component which does not have a navbar
- feat: improve email templating, adds contact and conversations fields to email template
- fix: evaluator - ensure "contains" checks evaluate individual tokens in the string rather than the entire string.
- feat: logo URL for the app in general settings.
- General fixes and refactors.
- fix: hide trailing prose mirror line breaks.
2024-11-03 00:40:51 +05:30
Abhinav Raut ee7be54c0d fix: hide passwords in API response
- Minor refactors.
- Fixes bugs in SelectTag component.
- Update vite.
2024-10-23 05:25:10 +05:30
Abhinav Raut 2b25ce32b6 feat: adds nonce check for oidc token exchange
- feat: adds filters support on conversations list
- refactor middlewares.go
- Adds new paginate.go for generating filtered paginated SQL queries this removes sql generation code from the conversations package.
- rename some components
- removes hardcoded `/uploads/`
2024-10-17 04:21:22 +05:30
Abhinav Raut 7e15995048 fix: file uploads with same file names replacing previous files, switches to uuids for stored filenames.
- minor refactor and fixes.
2024-10-10 02:59:00 +05:30
Abhinav Raut a38edb1572 refactor: notifications package.
- minor refactors in conversations pkg.
2024-08-27 04:16:34 +05:30
Abhinav Raut e698f010e1 refactor. 2024-08-05 11:59:06 +05:30
Abhinav Raut 79db632c3a refactor before merging conversations / messages pkg. 2024-07-29 02:00:10 +05:30
Abhinav Raut 7ac9a2fe67 refactor entire backend 2024-07-25 02:03:56 +05:30
Abhinav Raut d7fb9be211 some more commits? 2024-07-01 03:08:45 +05:30