mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-09-25 03:42:06 +00:00
xarmian
d1560606cb
* feat(urlimport): generic HTML→Markdown converter (TASK-1470)
Adds ConvertGeneric to internal/urlimport — the v1 catch-all converter
for "non-OpenAPI" URLs. Pipeline:
1. go-shiori/go-readability strips chrome/nav/ads/scripts and returns
the page's primary article.
2. JohannesKaufmann/html-to-markdown/v2 converts the cleaned HTML to
markdown.
3. cleanupMarkdown normalizes line endings, trims trailing whitespace,
collapses blank-line runs, and ensures a single trailing newline.
Fallback path: when Readability cannot identify an article (directory
listings, single paragraphs, pages with no clear content container),
the converter falls back to a whole-body conversion so callers still
get usable markdown.
Dependencies (license-checked):
- github.com/JohannesKaufmann/html-to-markdown/v2 v2.5.1 (MIT)
- github.com/go-shiori/go-readability (Apache-2.0)
Fixtures + tests:
- testdata/availity-shape.html — Availity-style div soup with heavy
chrome (nav, ads, sidebar, footer, analytics script). Asserts the
article content survives and the chrome is stripped.
- testdata/mdn-shape.html — MDN-style semantic HTML (article/main +
proper heading levels + code fences). Asserts structure preserved.
- TestConvertGeneric_EmptyBody — empty-input rejection.
- TestConvertGeneric_PlainTextFallback — Readability-can't-find-article
fallback path.
- TestCleanupMarkdown — 5-case table-driven cleanup verification.
Parent: PLAN-1467.
* fix(urlimport): preserve hard-line-break markers + apply WithDomain on fallback per Codex review (round 1)
- MEDIUM: cleanupMarkdown was stripping the markdown two-trailing-
spaces hard-line-break idiom. html-to-markdown emits <br> as
" \n" — bulk-stripping trailing whitespace was demoting hard
breaks to soft wraps. Now the cleanup steps line-by-line, keeps
exactly-two trailing spaces (no tab), strips 1/3+/tab-mixed runs.
- MEDIUM: The raw-HTML fallback path (used when Readability cannot
identify an article) now passes converter.WithDomain(pageURL) so
relative links/images resolve against the source URL rather than
the Pad host where they'd 404.
Tests added:
- cleanupMarkdown: hard-line-break preserved, single/triple trailing
spaces stripped, tab-mixed spaces stripped, blank-line-with-spaces
collapsed (5 new cases).
- TestConvertGeneric_RelativeURLsResolvedOnFallback: relative href
in non-article HTML resolves to absolute URL via pageURL.
77 lines
2.9 KiB
HTML
77 lines
2.9 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="utf-8">
|
|
<title>Eligibility & Benefits API — Availity-shape Docs</title>
|
|
<meta name="description" content="Eligibility and benefits inquiry endpoint reference.">
|
|
</head>
|
|
<body>
|
|
<header class="site-header">
|
|
<nav class="top-nav">
|
|
<a href="/">Home</a> | <a href="/docs">Docs</a> | <a href="/contact">Contact</a>
|
|
</nav>
|
|
<div class="ads"><img src="ad.gif" alt="ad"></div>
|
|
</header>
|
|
<div class="layout">
|
|
<aside class="sidebar">
|
|
<ul>
|
|
<li><a href="#auth">Authentication</a></li>
|
|
<li><a href="#endpoints">Endpoints</a></li>
|
|
<li><a href="#errors">Errors</a></li>
|
|
</ul>
|
|
</aside>
|
|
<div class="content">
|
|
<div class="container">
|
|
<div class="row">
|
|
<div class="col-12">
|
|
<div class="page-title">
|
|
<div class="page-title-inner">
|
|
<h1>Eligibility & Benefits API</h1>
|
|
<p>Submit an eligibility inquiry to a payer and receive coverage details for a member.</p>
|
|
</div>
|
|
</div>
|
|
<div class="page-body">
|
|
<div class="section" id="auth">
|
|
<h2>Authentication</h2>
|
|
<p>All requests require a bearer token obtained from the OAuth 2.0 client-credentials flow.</p>
|
|
<pre><code>POST /v1/token
|
|
client_id=YOUR_ID&client_secret=YOUR_SECRET&grant_type=client_credentials</code></pre>
|
|
</div>
|
|
<div class="section" id="endpoints">
|
|
<h2>Endpoints</h2>
|
|
<div class="endpoint">
|
|
<h3>POST /v1/eligibility</h3>
|
|
<p>Submit an eligibility inquiry.</p>
|
|
<p>Required fields:</p>
|
|
<ul>
|
|
<li><code>memberId</code> — payer-assigned member identifier</li>
|
|
<li><code>payerId</code> — Availity payer identifier</li>
|
|
<li><code>serviceTypes</code> — array of service-type codes (e.g. <code>30</code> for general health benefits)</li>
|
|
</ul>
|
|
</div>
|
|
<div class="endpoint">
|
|
<h3>GET /v1/eligibility/{id}</h3>
|
|
<p>Retrieve a previously submitted inquiry's response.</p>
|
|
</div>
|
|
</div>
|
|
<div class="section" id="errors">
|
|
<h2>Errors</h2>
|
|
<p>Errors follow the standard <a href="https://tools.ietf.org/html/rfc7807">RFC 7807</a> problem-details shape.</p>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<footer class="site-footer">
|
|
<p>© 2026 Availity, LLC.</p>
|
|
<div class="social">
|
|
<a href="https://twitter.com/availity">Twitter</a>
|
|
<a href="https://github.com/availity">GitHub</a>
|
|
</div>
|
|
</footer>
|
|
<script>console.log('analytics');</script>
|
|
</body>
|
|
</html>
|