Commit Graph

48 Commits

Author SHA1 Message Date
Abhinav Raut 2e89586a11 send links to the LLM as markdown so drafts keep real hyperlinks
Transcripts flattened links to "text ( url )", so the model copied that shape and drafts came back with the URL in brackets instead of a link. HTML to text for the LLM now writes "[text](url)", which Markdown2HTML turns back into an anchor.
2026-07-31 18:02:22 +05:30
Abhinav Raut a5a029d894 fix lossy text conversions in AI features 2026-07-31 16:36:04 +05:30
Abhinav Raut 72c3f29830 fix hyperlinks getting lost across AI features
AI grammar fix rewrote the draft as plain text, so links in the reply box were dropped. It now sends the editor HTML and asks the model to keep tags. Conversation transcripts, the AI agent history and copilot context also stripped link URLs when converting HTML to text, so the model never saw them. They now keep links as "text ( url )".
2026-07-31 16:08:19 +05:30
Abhinav Raut b569777cf6 match email quote classes exactly and enforce password length on set password 2026-07-25 21:46:17 +05:30
Abhinav Raut 6b8a0f9521 fix content loss in knowledge base chunking and harden AI agent limits
Final review pass before taking the AI agent branch live.

Knowledge base:
- Text not wrapped in a block tag was never collected, so prose around a
  table or list never reached the index. The assistant answered "no
  relevant information" for questions the snippet covered.
- Blocks over the token limit were truncated and the remainder dropped. They
  are split into several chunks now.
- Trimming an oversized block ran one rune at a time and re-tokenized the
  whole string each step. A large table took minutes. It uses a binary
  search now.
- Overlap text was not escaped, so a sentence containing markup swallowed
  the rest of the chunk.
- SVG and template text no longer reaches the index.

AI agent:
- Verification codes are capped per address and per conversation. The cap
  was per conversation only, so a customer correcting a mistyped email was
  told to check an inbox that never got a code.
- Livechat verification sends synchronously. A queued send returned nil even
  when SMTP failed, so a failure counted as a sent code.
- Queued jobs drain on shutdown and hand off to a human instead of being
  dropped with no reply.
- Deleting an assistant no longer moves resolved and closed conversations
  into the fallback team.
- Image decode is capped at 25 MP. The old bound allowed a 400 MB decode per
  attachment.

Auth and admin:
- A blank OIDC client secret no longer overwrites the stored one. Blank id
  or secret is rejected instead.
- OIDC token exchange uses the SSRF guarded client with a timeout.
- Renaming a tool auth header no longer attaches the secret of whichever row
  now sits at that position.
- Clearing embedding dimensions no longer refills 1536 on the next load,
  which pushed a wrong value to the provider on the next save.
- Copilot conversation lookups filter by access before capping at 10.
2026-07-25 03:52:32 +05:30
Abhinav Raut 40f8284e98 strip quoted reply chains from AI agent message context
When the agent built conversation history and mined FAQs, it used the
raw message text, which included the full quoted reply chain from every
email. That wasted tokens and confused the model with old back-and-forth.

Add emailquote.go to strip quoted blocks. HTML messages get their quote
containers pruned (gmail, yahoo, protonmail, outlook markers, and
blockquotes); plain text gets trailing ">" lines and "On ... wrote:" /
"Original Message" markers trimmed. If stripping leaves nothing (a
quote-only reply or forward), we fall back to the full text so the
message is not dropped. Add the matching protonmail_quote selector to
the frontend hide-quoted-text styles so the two stay in sync.

Also fix knowledge base chunking: plain text with no block structure was
kept as one chunk and could overflow the model limit. It now packs into
size-bound pieces on sentence boundaries, and oversized atomic blocks are
flushed and truncated on their own so they can never sneak through.
2026-07-22 21:22:35 +05:30
Abhinav Raut d02b65f2bd address CodeRabbit findings in AI embedding and assistant flows
Fix a reindex race: snippet embedding runs outside the lock, so a slower
job from an older edit could commit stale vectors after a newer edit.
Split Reindex into embed (lock-free) and commit (locked), and gate the
commit with a per-snippet generation counter so only the latest edit
wins. Delete drops the counter so an in-flight job can't re-insert
vectors for a deleted snippet.

Clear the DB avatar reference before deleting the avatar media file. The
old order deleted the file first, so a failed DB update left the DB
pointing at a missing file and a broken image.

Also handle unchecked errors flagged by errcheck (tx.Rollback,
html.Render, fmt.Fprintf), guard capToTokens against a negative limit
that would panic, and stop logging the full import URL since it can
carry credentials.
2026-07-16 12:50:02 +05:30
Abhinav Raut 8cdfe861e8 add AI snippet URL import, conversation summarize, and assistant reply languages
Snippet import: new "Import from URL" flow fetches a page and stores its
readable content as a snippet. Extraction uses the mackee/go-readability
library (Mozilla Readability port) and outputs Markdown, so nav/footer
boilerplate is dropped. The snippet list shows the source, and the edit
dialog is now wider with a taller content box.

Summarize: new "Summarize with AI" action on a conversation calls the AI and
adds the result as a private note. It shows an info toast right away so the
user knows it started, since the call can take a few seconds. This adds an
"info" toast variant that any feature can use.

Assistant languages: assistants can be given a list of allowed reply
languages. The assistant replies in the customer's language when it is one of
them, otherwise it falls back to the first. The preview also lists the
knowledge sources it used.

Cleanup: replace hardcoded gray/zinc/white colors with theme tokens
(text-muted-foreground, text-foreground, bg-accent) across several components.
2026-07-16 01:39:45 +05:30
Abhinav Raut dad99bf0ee render AI agent replies as markdown and fix assistant identity, handoff, and stats bugs
Replies from the AI agent are now converted from markdown to HTML with
goldmark before queueing, so bold, links, and lists render properly in the
widget, agent app, and email. The prompt now allows simple markdown. Raw
HTML in model output is escaped by goldmark, and both frontends sanitize
on render anyway.

Other fixes bundled in:
- validate avatar type and size before creating or updating an assistant,
  and roll back the assistant if the avatar upload fails after create
- return 404 from agent update and API key endpoints for AI assistant
  identity users, and hide assistants from mention and SLA user pickers
- reserve the autonomous assistant's built-in tool names so custom tools
  cannot shadow them
- apply resolve after the reply is posted so the CSAT survey follows the
  answer instead of preceding it
- unassign the assistant on handoff even when the fallback team is the
  same team
- count reopens by status category instead of status name, and exclude
  CSAT messages from the turn cap
- split oversized wrapper divs into child blocks when chunking KB HTML
  instead of truncating them
- return 404 when soft-deleting an already-deleted agent, and keep AI
  assistants (which have no email) visible in the compact users list
2026-07-12 23:12:44 +05:30
Abhinav Raut 3ba2b8281e Merge remote-tracking branch 'origin/main' into feat/ai-agent 2026-07-12 04:50:24 +05:30
Abhinav Raut cfc032f387 add SplitName util and use it to parse email contact names 2026-07-10 17:07:50 +05:30
Abhinav Raut 0ecd783941 add AI copilot backed by a knowledge base and custom tools
Adds two agent-facing AI features: a copilot chat panel in the conversation
sidebar and a generate-reply button in the reply box. Both run an agentic
tool-calling loop whose first tool searches the knowledge base.

Snippets are chunked and embedded on save, then searched in memory with
brute-force cosine similarity (no pgvector). Providers are split into completion
and embedding types. Both are OpenAI-compatible and the API key is encrypted at
rest. Admins can also register custom HTTP tools the model can call.

A new admin AI settings page covers provider config, snippets, and tools. The
v2.6.0 migration and schema add the ai_knowledge_base, embeddings, and ai_tools
tables plus the ai_providers type column.
2026-07-10 15:51:27 +05:30
Abhinav Raut a629beaf0b resolve sql builder date filters in the app timezone instead of UTC 2026-06-18 11:56:43 +05:30
Abhinav Raut 870f66c229 fix: sanitize invalid UTF-8 in incoming email, fix widget scroll flicker 2026-05-29 21:00:48 +05:30
Abhinav Raut ddc82d9a1a fix: extract text from malformed email HTML
Swap k3a/html2text for jaytaylor/html2text in stringutil.HTML2Text.
k3a returns empty on HTML with orphan tags (e.g. stray </a>), which
made text_content blank for some emails.
2026-05-28 19:03:26 +05:30
Abhinav Raut f6a4ad537f fix auto-replies and csat fanning out to last message's cc list
Both now send to the contact only and carry meta.is_automated so the
reply-box prefill skips them and inherits CCs from the last human message.
2026-05-15 16:16:32 +05:30
Abhinav Raut ffa5a22f3f fix s3 avatar_url storing bucket path instead of /uploads/<uuid>
Trim spaces around reply to config
2026-04-24 20:12:04 +05:30
Abhinav Raut dce33238e2 exclude inbox reply_to from reply recipients to prevent self-loop 2026-04-24 19:14:55 +05:30
Abhinav Raut 3d2da2a11f add reply_to on email inboxes for routing replies off the SMTP From domain 2026-04-24 18:39:16 +05:30
Abhinav Raut 7958d7e212 Merge branch 'main' into feat/live-chat-channel 2026-01-14 00:33:51 +05:30
Abhinav Raut 27d929c302 refactor add util funcs to extract conversation uuid from to address 2025-12-26 03:15:07 +05:30
Abhinav Raut a63c073c15 fix strip conversation uuid from receipient address as it could be the inbox email with plus variant
- adds test cases for util functions that strip inbox email.
- update make file to include frontend unit tests
2025-12-24 16:08:25 +05:30
Abhinav Raut 22af66c44d Feat: match incoming messages to conversations using reference number in the subject.
refactor: update GetConversation method to include reference number parameter
2025-12-24 16:08:25 +05:30
Abhinav Raut bf1cf025e0 Merge branch 'main' into feat/live-chat-channel 2025-10-04 13:38:42 +05:30
Abhinav Raut 16ca6b6df7 Send batched unread messages in conversation continuity email after contact has not seen a conversation for configured amount of time, instead of sending a single message immediately when contact is disconnected on websocket.
Make the batching interval and time threshold for unread messages configurable
2025-10-03 01:54:17 +05:30
Abhinav Raut d7067bce7d feat: add email fallback / continutiy inbox feature for live chat inbox 2025-09-21 00:51:19 +05:30
Abhinav Raut 634fc66e9f Translate welcome to libredesk email subject
- Update all SQL queries to add missing columns

- Update the create conversation API to allow setting the initiator of a conversation. For example, we might want to use this API to create a conversation on behalf of a customer, with the first message coming from the customer instead of the agent. This param allows this.

- Minor refactors and clean up

- Tidy go.mod

- Rename structs to reflect purpose

- Create focus structs for scanning JSON payloads for clarity.
2025-08-28 00:34:56 +05:30
Abhinav Raut 6c9eca3d81 fix: do not computed bcc from latest message. 2025-05-11 20:26:33 +05:30
Abhinav Raut 07b185050e fix: empty recipients in automated replies
- Make recipients list from the latest message recipients for automated replies
2025-05-11 18:51:34 +05:30
Abhinav Raut 6b2be57049 fix: set correct recipients when a 3rd email is involved in conversation, link to thread discussing this - https://github.com/abhinavxd/libredesk/issues/74#issue-3021419913
refactor move recipient computation to /utils/email-recipients
2025-05-10 23:46:19 +05:30
Abhinav Raut b2a53b18d5 feat: manage contacts
- New permission `contacts:manage`
- Views for contact list and single contact view.
- Ability to block contacts that in inturn stops new messages from the contact.
- Make all DB transactions that run dynamically generated SQL query `readonly`
- Rename `/admin/teams/users` to `/admin/users/agents/` for consistency. (ref #50)
- Fix UI glitches for long emails (refs #54)
- Fix empty created_at date for agents admin table (refs #51)
- Migrations for v0.6.0
2025-04-10 04:13:11 +05:30
Abhinav Raut 1875a62e00 feat: test cases for SLA calculator, string utilities and attachments 2025-03-26 23:57:56 +05:30
Abhinav Raut aeef7d4ad7 feat: configurable SLA alerts per SLA. 2025-03-21 23:23:03 +05:30
Abhinav Raut 8551b65a27 fix: set references header in all outgoing emails, set the last 20 messages.
feat: set conversation reference number in the subject of conversation for better thread matching.
fix: hide CSAT link from conversation last message.
2025-03-05 03:49:22 +05:30
Abhinav Raut 494bc15b0a feat: Enable agents to create conversations from the UI
Before this feature the only way to create a conversation was by adding inbox and sending an email.

Agents first search contacts by email, see a dropdown select an existing contact or fill a new email for new contact.

The backend creates contact if it does not exist, creates a conversation, sends a reply to the conversation.
Optinally assigns conversation to a user / team.

fix: Replies to emails create a new conversation instead of attaching to the previous one.

Was not happening in gmail, as gmail was sending the references headers in all replies and I missed this completely. So when libredesk searches a conversation by references headers it worked!

Instead the right way is to generate the outgoing email message id and saving it in DB. This commit fixes that.

There could be more backup strategies like putting reference number in the subject but that can be explored later.

chore: new role `conversatons:write` that enables the create conversations feature for an agent.

chore: migrations for v0.4.0.
2025-03-05 01:17:42 +05:30
Abhinav Raut 356a206110 fix: new incoming messages having 0 attachments in API responses, upload attachments first before sending out websocket updates.
- refactor: stringutils package remove unncessary `string` in function names.
- minor refactors.
2025-02-08 18:04:02 +05:30
Abhinav Raut d3c2cc2527 feat: add CSAT sending functionality and improve error handling in templates
feat: automation action to send CSAT.
- minor refactors.
2025-02-02 21:03:09 +05:30
Abhinav Raut 52e401c601 feat: email cc and bcc in conversation replies
- feat: adds cc and bcc inputs in conversation text editor.
- feat: mask conversation id in api responses.
- customize shadnc dialog to hide close button & cross icon.
2025-01-25 03:47:17 +05:30
Abhinav Raut 586c7355f9 feat: new automation fields for conversations - hours since created, hours since resolved.
- feat: adds new actions - send private note, send reply.
- minor fixes.
2024-11-03 21:47:47 +05:30
Abhinav Raut ed314eb1a5 feat: reset password flow
- feat: notification emails including - welcome email, assigned conversation email, reset password email.
- feat: adds outer app component which does not have a navbar
- feat: improve email templating, adds contact and conversations fields to email template
- fix: evaluator - ensure "contains" checks evaluate individual tokens in the string rather than the entire string.
- feat: logo URL for the app in general settings.
- General fixes and refactors.
- fix: hide trailing prose mirror line breaks.
2024-11-03 00:40:51 +05:30
Abhinav Raut ee7be54c0d fix: hide passwords in API response
- Minor refactors.
- Fixes bugs in SelectTag component.
- Update vite.
2024-10-23 05:25:10 +05:30
Abhinav Raut 2b25ce32b6 feat: adds nonce check for oidc token exchange
- feat: adds filters support on conversations list
- refactor middlewares.go
- Adds new paginate.go for generating filtered paginated SQL queries this removes sql generation code from the conversations package.
- rename some components
- removes hardcoded `/uploads/`
2024-10-17 04:21:22 +05:30
Abhinav Raut 7e15995048 fix: file uploads with same file names replacing previous files, switches to uuids for stored filenames.
- minor refactor and fixes.
2024-10-10 02:59:00 +05:30
Abhinav Raut a38edb1572 refactor: notifications package.
- minor refactors in conversations pkg.
2024-08-27 04:16:34 +05:30
Abhinav Raut e698f010e1 refactor. 2024-08-05 11:59:06 +05:30
Abhinav Raut 79db632c3a refactor before merging conversations / messages pkg. 2024-07-29 02:00:10 +05:30
Abhinav Raut 7ac9a2fe67 refactor entire backend 2024-07-25 02:03:56 +05:30
Abhinav Raut d7fb9be211 some more commits? 2024-07-01 03:08:45 +05:30