diff --git a/LICENSE b/LICENSE index 241dafb..3dd92b0 100644 --- a/LICENSE +++ b/LICENSE @@ -1,22 +1,679 @@ -MIT License +HAProxy OpenManager +Copyright (C) 2025-2026 Taylan Bakırcıoğlu and HAProxy OpenManager Contributors -Copyright (c) 2025 HAProxy OpenManager Contributors +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU Affero General Public License as published +by the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU Affero General Public License for more details. -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. +You should have received a copy of the GNU Affero General Public License +along with this program. If not, see . -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. +--- + GNU AFFERO GENERAL PUBLIC LICENSE + Version 3, 19 November 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU Affero General Public License is a free, copyleft license for +software and other kinds of works, specifically designed to ensure +cooperation with the community in the case of network server software. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +our General Public Licenses are intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + Developers that use our General Public Licenses protect your rights +with two steps: (1) assert copyright on the software, and (2) offer +you this License which gives you legal permission to copy, distribute +and/or modify the software. + + A secondary benefit of defending all users' freedom is that +improvements made in alternate versions of the program, if they +receive widespread use, become available for other developers to +incorporate. Many developers of free software are heartened and +encouraged by the resulting cooperation. However, in the case of +software used on network servers, this result may fail to come about. +The GNU General Public License permits making a modified version and +letting the public access it on a server without ever releasing its +source code to the public. + + The GNU Affero General Public License is designed specifically to +ensure that, in such cases, the modified source code becomes available +to the community. It requires the operator of a network server to +provide the source code of the modified version running there to the +users of that server. Therefore, public use of a modified version, on +a publicly accessible server, gives the public access to the source +code of the modified version. + + An older license, called the Affero General Public License and +published by Affero, was designed to accomplish similar goals. This is +a different license, not a version of the Affero GPL, but Affero has +released a new version of the Affero GPL which permits relicensing under +this license. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU Affero General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Remote Network Interaction; Use with the GNU General Public License. + + Notwithstanding any other provision of this License, if you modify the +Program, your modified version must prominently offer all users +interacting with it remotely through a computer network (if your version +supports such interaction) an opportunity to receive the Corresponding +Source of your version by providing access to the Corresponding Source +from a network server at no charge, through some standard or customary +means of facilitating copying of software. This Corresponding Source +shall include the Corresponding Source for any work covered by version 3 +of the GNU General Public License that is incorporated pursuant to the +following paragraph. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the work with which it is combined will remain governed by version +3 of the GNU General Public License. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU Affero General Public License from time to time. Such new versions +will be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU Affero General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU Affero General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU Affero General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If your software can interact with users remotely through a computer +network, you should also make sure that it provides a way for users to +get its source. For example, if your program is a web application, its +interface could display a "Source" link that leads users to an archive +of the code. There are many ways you could offer source, and different +solutions will be better for different programs; see section 13 for the +specific requirements. + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU AGPL, see +. diff --git a/README.md b/README.md index ca9f1b1..48833f8 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,7 @@ Modern, web-based management interface for HAProxy load balancers with multi-clu - [Apply Management & Version Control](#apply-management--version-control) - [Security & Certificate Management](#security--certificate-management) - [ACME Automation](#acme-automation---automated-ssl-certificates) + - [Site Wizard - Guided Multi-Step Host Setup](#site-wizard---guided-multi-step-host-setup) - [WAF Management](#waf-management) - [IP Inventory](#ip-inventory---cross-cluster-ip-search) 3. [Key Capabilities](#key-capabilities) @@ -36,6 +37,7 @@ Modern, web-based management interface for HAProxy load balancers with multi-clu - [Agent Management](#agent-management---haproxy-agent-deployment--monitoring) - [Dashboard](#dashboard---main-overview--real-time-monitoring) - [Frontend Management](#frontend-management---virtual-host--routing-configuration) + - [New Site Wizard](#new-site-wizard---guided-multi-step-host-setup) - [Backend Servers](#backend-servers---server-pool-management) - [Configuration](#configuration---haproxy-config-file-management) - [Apply Management](#apply-management---change-tracking--deployment) @@ -97,11 +99,13 @@ This architecture provides better security (no inbound connections to HAProxy se ✅ **Agent-Based Pull Architecture** - Secure, scalable management without inbound connections ✅ **Multi-Cluster & Pool Management** - Organize and manage multiple HAProxy clusters from one interface ✅ **Frontend/Backend/Server CRUD** - Complete entity management with visual UI +✅ **New Site Wizard** - Step-by-step guided setup for a complete proxied host (frontend + backend + servers + SSL/ACME) in one consolidated atomic apply ✅ **Bulk Config Import** - Import existing `haproxy.cfg` files with smart SSL auto-assignment ✅ **Version Control & Rollback** - Every change versioned with one-click restore capability ✅ **Real-Time Monitoring** - Live stats, health checks, and performance dashboards ✅ **SSL Certificate Management** - Centralized SSL with expiration tracking ✅ **ACME Auto SSL (Let's Encrypt)** - Automated certificate issuance, renewal, and deployment via ACME protocol +✅ **ACME Certificate Diagnostic Panel** - Automated preflight that checks agent readiness, DNS resolution, port 80 reachability, and ACME challenge ACL before issuing certificates ✅ **WAF Rules** - Web Application Firewall management and deployment ✅ **Agent Script Versioning** - Update agents via UI (Monaco editor) with auto-upgrade ✅ **Token-Based Agent Auth** - Secure token management with revoke/renew @@ -192,6 +196,11 @@ This architecture provides better security (no inbound connections to HAProxy se **ACME Settings** - ACME/SSL Automation configuration with provider selection, staging mode, and auto-renewal settings ![ACME Settings](docs/screenshots/acme-settings.png) +### Site Wizard - Guided Multi-Step Host Setup + +**New Site Wizard** - Multi-step guided form for creating a complete proxied host (domains + backend pool + servers + frontend + SSL/ACME) with live HAProxy config validation and atomic apply +![New Site Wizard](docs/screenshots/site-wizard.png) + ### WAF Management **WAF Management** - WAF rule configuration with request filtering, rate limiting, and advanced options @@ -638,6 +647,23 @@ The dashboard displays comprehensive real-time metrics collected by agents from - **Advanced Options**: Connection limits, timeouts, and performance tuning - **Search & Filter**: Real-time search and status-based filtering +### New Site Wizard - Guided Multi-Step Host Setup + +The New Site Wizard is the recommended entry point for creating a complete proxied host. It collects every piece of configuration a host needs across a five-step guided form and creates the matching frontend + backend + servers + SSL binding as a single atomic apply. + +- **5-step guided flow**: + 1. **Domains** - host header / SNI / wildcard validation, IDN/Punycode normalization, port + bind-address collision check + 2. **Backend** - pool name, load-balance algorithm, health check method/URI, cookie-based persistence (RFC 6265 and HAProxy-parser-safe character validation) + 3. **Servers** - per-server address/port/weight, optional CA bundle reference (cluster-RBAC enforced), backup-server + cookie-value validators + 4. **SSL** - three modes: existing certificate (cluster-RBAC), PEM upload (chain validation + SAN/CN match), or ACME order (account binding + preflight) + 5. **Review + Dry-Run** - live `POST /api/sites/preview` runs the proposed config through `haproxy -c` and surfaces every WARNING/ERROR with a marker comment that pinpoints the offending block before anything touches the database +- **Draft persistence**: every step auto-saves to a server-side draft (30-day TTL, 50 drafts per user cap, cluster-scoped, PEM material stripped at rest) +- **Resume across sessions**: drafts can be reopened from a different browser; cluster swap mid-wizard surfaces a confirmation prompt to prevent cross-cluster contamination +- **Atomic apply**: frontend + backend + servers + SSL binding land as a single PENDING change reviewable from Apply Management - accept and apply, or reject as a whole +- **Cluster RBAC**: every step honours `user_pool_access` and cluster RBAC; SSL certificate references are validated against the target cluster at both preview AND create time +- **Hardening**: IDN/Punycode safe slug generation, cookie/header injection guards, path-traversal protection on SSL filenames, per-user-per-minute rate limit on create + preview, advisory locks against concurrent creation +- **Backwards compatible URLs**: legacy `/proxied-hosts/new` route continues to resolve to the new wizard + ### Backend Servers - Server Pool Management - **Server Management**: Add, edit, remove, and configure backend servers - **Health Checks**: HTTP/TCP health check configuration and monitoring @@ -837,6 +863,20 @@ sequenceDiagram **Key behavior**: Auto-renewed certificates follow the **exact same Apply pipeline** as manual SSL updates. The cert transitions `PENDING -> APPLIED` automatically, agents are notified, and cross-cluster propagation works identically to manual Apply. No separate deployment mechanism is used. +#### ACME Certificate Diagnostic Panel + +Available from the ACME Automation list (`Diagnose` button on each order, or by clicking the order's status tag), the Diagnostic Panel runs a full preflight checklist before a certificate is issued or renewed and surfaces every blocker in a single operator-friendly view: + +- **Agent reachability** - verifies that at least one agent in each target cluster is online and within the last-seen window +- **DNS resolution** - resolves every domain on the order (A / AAAA / CNAME) and flags wildcards and non-resolvable names with the exact upstream resolver error +- **Port 80 reachability** - checks that the HTTP-01 challenge port is reachable from the public internet via the agent's egress path +- **ACME challenge ACL preview** - renders the `acl1 !acl1` + `http-request return` block that will be injected at apply time, so the operator can see exactly what HAProxy will receive +- **HSTS / rate-limit policy collision check** - warns if existing frontend rules would short-circuit the challenge route +- **Account binding check** - confirms a valid ACME account exists for the selected provider + cluster combination +- **Event Log timeline** - merged view of typed `acme_order_events` rows and correlated `user_activity_logs` entries; auto-tails every 5s while the order is in `pending`/`processing` +- **Humanized error display** - covers 11+ RFC8555 problem types (`badNonce`, `caa`, `connection`, `rateLimited`, `unauthorized`, ...) with full backwards compatibility for legacy plain-string error details +- **Operator-friendly error envelope** - every failure returns `{ correlation_id, field, cause, remediation }` so root-causing is one API call away + #### ACME Features | Feature | Description | @@ -2169,7 +2209,13 @@ journalctl -u keepalived ## License -This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. +This project is licensed under the **GNU Affero General Public License v3.0 (or later)** — see the [LICENSE](LICENSE) file for the full text. + +A short summary: + +- You are free to use, modify, and distribute this software. +- If you run a modified version as a network service (e.g., SaaS), you must make the modified source available to its users (AGPL §13). +- Any redistribution or derivative work must remain under AGPL-3.0-or-later. ## Author @@ -2199,744 +2245,7 @@ Developed with ❤️ for the HAProxy community ## Release Notes -### v1.5.0 — ACME Diagnostics & Site Wizard - -#### Highlights - -- **Issue #13: ACME Diagnostic Panel.** From the ACME Automation list, click the new `Diagnose` button (or the order's status tag) to launch a Modal with three Tabs: - 1. **Pre-flight Checks** — DNS resolution, port-80 reachability, HAProxy routing, ACME account status, agent health. Each check has its own `Re-run` button. - 2. **Event Log** — Merged timeline of typed `acme_order_events` rows (added in v1.5.0) and correlated `user_activity_logs` entries; auto-tails every 5s while the order is in `pending`/`processing`. - 3. **Raw Error** — Humanized error display covering 11+ RFC8555 problem types (`badNonce`, `caa`, `connection`, `rateLimited`, `unauthorized`, ...) with full backwards compatibility for legacy plain-string `error_detail`. -- **Issue #14: New Site Setup Wizard.** A single guided flow (`/sites/new`) that creates a Backend + Servers + HTTP Frontend (and optional HTTPS Frontend) in one atomic transaction. SSL choice supports four modes: - - `acme` — defers HTTPS frontend creation to a deferred `post_completion_actions` block on a wizard-staged ACME order; the order is promoted to a real LE call only after the agent confirms the gating `bulk-site-create-{ts}` config version (legacy `bulk-proxied-host-create-{ts}` is still recognised by the reject path for historical APPLIED versions). - - `upload` — uploads PEM cert+key in the same transaction. - - `existing` — reuses an admin-uploaded cert and ensures the cluster junction is set. - - `none` — HTTP-only host. - The wizard ships with the same visual ACL rule builder used by the standalone Frontend Management page (Routing & ACLs section on the Frontend step) so operators define `acl` / `use_backend` / `redirect` rules from cluster-scoped backend dropdowns instead of free-text HAProxy directives. Drafts persist for 30 days with PEM material stripped at rest. Reject of the wizard's PENDING version cleanly rolls back ALL wizard entities (backends, servers, frontend(s), SSL row if any, AND the wizard-staged `letsencrypt_orders` row). - -#### Migration Release Notes - -This release adds **idempotent** migrations only — no destructive schema changes: - -- New columns on `letsencrypt_orders`: - - `post_completion_actions JSONB` (deferred actions for wizard ACME mode) - - `wizard_staged_until TIMESTAMPTZ` (24h timeout for wizard-staged orders) - - `pending_apply_version_name VARCHAR(255)` + partial index `WHERE status='wizard_staged'` - - `created_by INTEGER REFERENCES users(id) ON DELETE SET NULL` -- New tables: `acme_order_events` (typed event log, 90d retention), `wizard_drafts` (30d retention). -- New composite index `idx_user_activity_logs_user_action_time` for the per-user-per-minute rate-limit COUNT(*) used by both new features. - -The `letsencrypt_orders.status` column has no CHECK constraint; the new `wizard_staged` value coexists with all existing statuses (`pending`, `ready`, `processing`, `valid`, `invalid`, ...). - -The reject path's force-delete fallback now also covers `entity_type='letsencrypt_order'` snapshots so wizard-staged ACME orders are removed when their parent PENDING version is rejected. - -#### Rollback Considerations - -- **Forward compatibility (v1.5.0 → future).** All new columns/tables are additive; older code paths that do not know about them are unaffected. -- **Backward rollback (v1.5.0 → v1.4.0).** The new columns/tables remain in the database harmlessly; v1.4.0 simply ignores them. Wizard-staged ACME orders that were never promoted to `pending` will not progress on v1.4.0 (the v1.4.0 background task does not select `status='wizard_staged'`); admins can either: - 1. Wait for the 24h `wizard_staged_until` timeout to fire (v1.5.0 only) — only relevant if rolling back temporarily, OR - 2. Manually `DELETE FROM letsencrypt_orders WHERE status='wizard_staged'` and re-run the wizard once you re-deploy v1.5.0. -- **Wizard ACME failure scenarios.** If the agent never confirms the gating config version (e.g. agent down), the wizard-staged ACME order will time out and transition to `status='invalid'` after 24h with `error_detail='wizard staged timeout (>24h with no agent confirm)'` — surfaced in the new Diagnostic Panel. -- **Drafts.** PEM material is server-side stripped from `wizard_drafts.payload`; rolling back will not leak keys at rest. - -#### v1.5.x — Site Wizard module rename + ACL UX parity - -A non-breaking follow-up to v1.5.0 that retires the internal "Proxied Host" namespace in favour of "Site" everywhere it used to leak into operators' workflow: - -- **Module file rename.** `backend/routers/proxied_host.py` and `backend/models/proxied_host.py` are now `site_wizard.py`. The Pydantic class `ProxiedHostCreate` (and its sibling `ProxiedHostPreflightAcme` / `ProxiedHostDraftCreate`) was renamed to `SiteCreate` etc. with a module-level alias `ProxiedHostCreate = SiteCreate` so existing imports keep working. -- **API URL prefix rename.** The wizard now mounts at `/api/sites/*` (canonical). The legacy `/api/proxied-hosts/*` slug is preserved as a hidden `308 Permanent Redirect` alias on `main.py`, so external integrators keep working through the redirect during the transition window. The frontend axios calls all target `/api/sites/*` directly. -- **Audit-log version-name rename.** Wizard-applied versions now carry the prefix `bulk-site-create-{ts}`. The cluster reject path on `cluster.py` recognises BOTH the new prefix and the legacy `bulk-proxied-host-create-{ts}` so historical APPLIED versions still clean up. -- **Activity-log action + resource_type.** The wizard's explicit `log_user_activity` call now writes `action='wizard_create_site'` and `resource_type='site'` (was `wizard_create_proxied_host` / `proxied_host`). Older audit rows already in the database keep their pre-rename strings. -- **Rate-limit dual-name aliasing.** The wizard's per-user-per-minute rate-limit (`COUNT(*)` over `user_activity_logs`) now passes `ANY($::text[])` so it counts BOTH the canonical `site_*` action_name and its legacy `proxied_host_*` companion. A deploy that lands mid-minute cannot reset the quota, and the limit cannot be bypassed by an attacker who picks the legacy name. -- **DB schema rebrand (Phase I).** The `wizard_drafts.wizard_type` column's schema-level `DEFAULT` flipped from `'proxied_host'` to `'site'`. New rows land with the canonical value via an explicit `INSERT … VALUES ($1, 'site', …)`. The list / cap / cluster-delete-purge queries all filter on `wizard_type IN ('site', 'proxied_host')` so pre-rebrand drafts owned by the same user remain visible and remain rejectable. **Existing rows are NOT row-rewritten** — the migration is a metadata-only `ALTER TABLE … SET DEFAULT 'site'` that takes a non-blocking lock and is idempotent. -- **Wizard ACL UX parity.** The Frontend step now embeds the same `ACLRuleBuilder` component used by the Frontend Management page, with cluster-scoped existing backends populated automatically and the wizard's brand-new backend surfaced as a virtual entry in the use_backend dropdown. Drafts persist the three rule arrays so a resumed draft hydrates with the same routing config. - -Backward compatibility is preserved at every layer: the DB-level `wizard_drafts.wizard_type='proxied_host'` enum value (still valid for pre-rebrand rows), the `LEGACY_WIZARD_DRAFT_SESSION_KEY` browser sessionStorage key, frontend route aliases (`/proxied-hosts/new`, `/proxied-hosts/drafts`), and the legacy `/api/proxied-hosts/*` URL all keep working. - -##### Phase J — UI mount-time race fix ("clusters don't appear after deploy") - -**Reported symptom.** After every rolling deploy, operators saw the cluster -selector empty for "a long time" — closing and re-opening the browser did -not help, but waiting ~30s did. The user diagnosed it as a UI problem. - -**Root cause.** A React mount-time race between `` (parent) -and `` (child). React's useEffect commit phase fires -CHILD effects before PARENT effects, so `ClusterProvider.useEffect` — -which dispatches the very first `axios.get('/api/clusters')` — ran BEFORE -`AuthProvider.useEffect` set `axios.defaults.headers.common['Authorization']`. -The first request went out un-authenticated → backend returned 401 → -ClusterContext's `catch` block silently committed `clusters=[]`. The -operator-visible UI rendered "no clusters" until the 30-second -auto-refresh interval re-fired the request, by which point auth had -hydrated and the call succeeded. Restarting the browser kept hitting the -same race because localStorage carried the token but the useEffect -ordering was identical. - -**Fix (3 layers of defence).** - -1. **`src/index.js` module-level axios bootstrap.** Runs before - `` is rendered, so no React tree (and therefore no useEffect) - can fire before it. Synchronously seeds - `axios.defaults.headers.common['Authorization']` from localStorage - AND installs an `axios.interceptors.request` that re-reads the token - on every outbound request. The interceptor is the belt-and-suspenders - defence — it cannot be raced by mount ordering and survives any - future code path that mutates `axios.defaults`. -2. **`AuthContext` synchronous useState lazy initialisers.** The - `_hydrateAuthSync` helper runs during the AuthProvider RENDER phase, - which precedes ANY child useEffect. It reads localStorage and seeds - `loading=false`, `isAuthenticated=true`, and the user object — - eliminating the post-mount async hydration that produced the race. -3. **`ClusterContext` auth-gate + exponential-backoff retry.** The - first fetch is gated on `isAuthenticated && !authLoading`, and a - transient 5xx / network failure now triggers up to 4 fast retries - (1s, 2s, 4s, 8s — total ~15s) instead of immediately blanking the - cluster list and depending on the 30s auto-refresh interval. 401/403 - intentionally do NOT retry (re-auth is the user's job). The previous - cluster list is preserved on transient hiccups so periodic refreshes - no longer flash an "empty state". - -**Verification.** 22 static-source pin tests in -`backend/tests/test_frontend_auth_bootstrap_phase_j.py` cover all three -layers (bootstrap order, AuthContext lazy init, ClusterContext retry + -auth-gate) plus the six audit-loop hotfixes below. 590 backend tests -pass; frontend `npm run build` clean. - -**Operator-visible outcome.** After deploy, the cluster selector -populates on the FIRST fetch — no 30-second wait. A transient kube-proxy -convergence window collapses to a few seconds (covered by retries) instead -of being masked by the 30-second interval. - -##### Phase J audit hotfixes (audit fix #2 → #6) - -Successive audit loops surfaced six follow-on issues that each -reproduced one or more of the original symptoms in narrower windows. -Each fix is pinned in the same Phase J pin-test file: - -- **Audit fix #2 — stale closure in `fetchClusters`.** Wrapping - `fetchClusters` in `useCallback(…, [])` froze `selectedCluster` at - its mount-time value (`null`), so the 30-second auto-refresh - reported stale agent-health data forever. Bridged via - `selectedClusterRef`, updated in a passive effect, and read inside - the callback. -- **Audit fix #3 — missing `setLoading(true)` on the auth-gated - first fetch.** During the login flow, the ClusterContext effect - fired with `loading=false` (the initial value), so the cluster - selector briefly rendered "No Cluster Selected" before the spinner - came back. Now the effect explicitly seeds `setLoading(true)` when - the auth gate flips open. -- **Audit fix #4 — `loading=false` between retry waves.** The - `finally` clause unconditionally released the loading flag, so the - spinner blinked off between each backoff attempt and the UI - flashed "No Cluster Selected" for up to ~15 seconds — the very - symptom Phase J was meant to eliminate. The release is now gated - on `retryTimerRef.current === null` so the spinner stays on across - the entire retry budget. -- **Audit fix #5 — exhausted retry budget left counter at 4.** - `retryAttemptRef` was reset only on a successful fetch and on the - auth-gate transition. After 4 transient failures in a row the - counter stayed at 4 for the rest of the session, so any subsequent - invocation (the 30s background refresh, an explicit refetch from a - mutator like `deleteCluster`) skipped the retry pattern entirely - on the first transient failure. The settle-into-empty-state branch - now resets the counter so each fresh invocation gets a full retry - budget. -- **Audit fix #6 — page-content "No Cluster Selected" during the - fetch window.** The cluster selector itself already showed a - spinner via `loading`, but page-level components - (`SSLManagement`, `Configuration`, `DashboardV2`, - `BulkConfigImport`, `BulkVersionHistory`) checked - `!selectedCluster` directly and rendered a permanent warning - affordance. During the 15-second retry budget the page therefore - read as "you forgot to pick a cluster" even though the cluster - list was simply still being fetched. Each page now also consumes - `loading: clustersLoading` from `useCluster()` and shows a neutral - "Loading clusters…" affordance until the fetch settles, only then - flipping to the warning. This is the fix that fully closes the - user-visible loop on the original "clusters don't appear after - deploy" report. - -##### Phase K — Site Wizard validation hardening + UX simplification - -Operators reported that completing the wizard and clicking **Create & -Apply** repeatedly surfaced opaque 422 errors at the final step: - -``` -HTTP→HTTPS redirect cannot be combined with custom redirect rules. -body -> frontend -> acl_rules -> 0: Input should be a valid dictionary -body -> frontend -> use_backend_rules -> 0: Input should be a valid dictionary -``` - -Root causes (each fixed by Phase K): - -1. **Contract mismatch on rule fields.** `ACLRuleBuilder.js` - serialised ACL / use_backend / redirect rules as `string[]` while - `FrontendStep` typed them as `List[dict]`. Every wizard POST - carrying a single ACL rule failed Pydantic validation. The - downstream renderer in `services/haproxy_config.py` had always - expected strings, so the schema mismatch was the stale side. -2. **Step 2 mutex was advisory only.** The - `https_redirect ⊕ redirect_rules` validator existed at the model - level but the wizard let the operator advance through Step 2 → 3 → - 4 with the conflict in place, only to be punted back at Create. -3. **No HAProxy validation before Create.** `/api/sites/preview` - only checked collisions; the real validator ran inside the - `create_site` transaction *after* entity inserts. Operators - discovered errors at apply time. -4. **HTTPS step overcrowded.** 11 SSL bind-line knobs flat on Step 3 - without a defaults summary or any visible grouping. - -**What changed** - -- **Phase A — Backend contract + safety validators** - (`backend/models/site_wizard.py`). - - `acl_rules`, `use_backend_rules` are now `List[str]`. - `redirect_rules` stays `List[Union[str, dict]]` to preserve the - structured-redirect path used by the renderer's - `_format_redirect_rule`. - - Per-element safety validators reject embedded newlines (HAProxy - directive injection prevention), shell-substitution patterns - (`system`, `exec`, `eval`, `$(`, backtick — same set the - manual frontend API has been blocking since pre-R14), 4 KB - string limit, and empty / whitespace-only strings. - - Two new cross-field model validators close silent-bug gaps: - `FrontendStep.reject_tcp_mode_with_https_redirect` (the renderer - used to emit an HTTP-only directive into a TCP frontend) and - `SSLChoice.reject_inverted_tls_versions` (when both `ssl_min_ver` - and `ssl_max_ver` are set, reject `min > max`). -- **Phase B — Step 2 hard-block + TCP-mode guard** - (`SiteWizard.js`, `ACLRuleBuilder.js`). - - The Step 2 Next handler now hard-blocks the - `https_redirect ⊕ redirect_rules` and `mode='tcp' ⊕ - https_redirect` combinations with one-click resolve buttons - ("Disable HTTP→HTTPS switch" / "Remove redirect rules"). - - Switching the frontend to TCP mode auto-clears `https_redirect`; - the Switch is also `disabled` while `mode==='tcp'` with an - explanatory tooltip. - - `ACLRuleBuilder` accepts a new `disableRedirectRules` prop that - visually disables the Redirect Rules section (`aria-disabled`, - greyed-out cards, tooltip) when the parent passes - `https_redirect=true`. The rules data stays in component state - so toggling the switch off restores them. -- **Phase C — Real HAProxy dry-run gate before Create** - (`backend/routers/site_wizard.py`, `SiteWizard.js`). - - New shared helper `_synthesize_candidate_haproxy_config(body, - conn, *, entities_already_inserted)` is used by both - `create_site` (post-insert validation gate) and a new dry-run - path on `POST /api/sites/preview`. Two callsites pinned by - `test_phase_k_create_site_and_preview_use_same_synthesis_helper` - so the apply gate and the dry-run gate cannot silently desync. - - `POST /api/sites/preview` now accepts an optional - `validate_haproxy_config=true` query param. When set, the - endpoint runs `HAProxyConfigValidator` against the synthesised - candidate config and returns a `validation: {is_valid, - error_count, warning_count, errors, warnings, infos}` block in - the same 200 OK envelope. Validator crashes return - `is_valid: null` + `validator_error` (matches `create_site`'s - non-fatal posture). The dry-run path is rate-limited at 5/min - via `_enforce_rate_limit` and emits structured ENTER/EXIT - `logger.info` lines for telemetry. Legacy preview callers - (`SiteDrafts.handlePreview`) are unaffected — they pass no flag. - - The wizard auto-fires the dry-run on Step 4 entry with an - `AbortController` so rapid Step 4 → Step 2 → Step 4 navigation - cancels the in-flight request. A six-state validation card - renders inline: idle / loading / clean (green) / - `warnings_only` (yellow) / `errors` (red, blocks Create) / - `pydantic_error` (red, body-parse failures from Phase A's new - validators or PEM-stripped resume drafts) / `unavailable` - (orange, advisory — Create stays enabled to mirror the - validator-crash-is-non-fatal contract). Each error / - `pydantic_error` row gets an `Edit Step N` jumpback button via - a static directive→step + loc→step mapping table. - - Audit-fix #1 (post-implementation review): the wizard also - resets `dryRunResult.status` to `idle` whenever the operator - leaves Step 4. The ACL builder lives outside the antd Form - so its mutations don't fire `Form.onValuesChange`; without - this reset a stale `clean`/`errors`/`warnings_only` status - survives Step 4 → Step 2 (ACL edit) → Step 4 round-trips - and the auto-fire branch suppresses the next fetch. With - the reset every Step 4 entry triggers a fresh dry-run - (rate-limit-safe — entry is operator-initiated, not - programmatic). - - Audit-fix #2 (post-implementation review): the - `Edit Step N` jumpback now also resolves the target step - from the error **message text** when `loc` cannot pinpoint - it. Pydantic v2 raises `model_validator(mode="after")` - errors with `loc=()`; FastAPI prepends `'body'` so the - operator-visible envelope is `loc=['body']` (length 1). - The legacy `_locPathToStep` early-returned null for this - case, dropping the jumpback for PEM-stripped resume - ("ssl.mode='upload' requires a non-empty PEM-encoded - certificate_content …") and every - `enforce_acme_apply_and_http` cross-field rejection. A - small ordered pattern table recovers the step from the - failure message text so operators always get a working - "fix-from-here" button. - - Audit-fix #2 round 3 (post-implementation review): the - pattern table is ordered so cross-field ACME messages - route to the step the operator must EDIT to fix the - error, not the step that "feels related". A naive ordering - ("SSL first because every cross-field message starts with - `ssl.mode='acme'`") would route every cross-field hit to - Step 3, defeating the jumpback. Order is now: - `apply_immediately` (Step 4) → `wildcard`/`domains` (Step - 0) → `frontend.*`/`bind_port` (Step 2) → `backend.*` (Step - 1) → SSL catch-all (Step 3, LAST). With this ordering, - "ssl.mode='acme' requires apply_immediately=true" routes - to Step 4 (toggle the switch), "ssl.mode='acme' requires - frontend.bind_port=80" routes to Step 2 (edit FE port), - and "(HTTP-01) cannot issue wildcard certs" routes to - Step 0 (remove wildcard). PEM-stripped and other SSL-only - errors still hit Step 3 via the final catch-all. - - Audit-fix #2 round 4 (post-implementation review): the - pydantic_error renderer no longer emits a stray - `: ` orphan-colon prefix when the failing - error has no field path. SiteCreate-level model_validator - errors land with `loc=['body']` (length 1); after dropping - the leading `'body'` marker the joined path is empty. - Pre-fix the renderer wrapped that empty string in - `...: `, producing a visually broken " : " - prefix in front of every PEM-stripped resume message and - every `enforce_acme_apply_and_http` cross-field rejection. - Post-fix the strong/colon prefix renders only when a real - field path exists. -- **Phase D — HTTPS step simplification + UI parity** - (`SiteWizard.js`). - - TLS bounds (`ssl_min_ver`, `ssl_max_ver`) and the HSTS quartet - stay first-class on the SSL step; rarely-used knobs - (`https_bind_port`, `https_frontend_name_suffix`, `ssl_alpn`, - `ssl_ciphers`, `ssl_ciphersuites`, `ssl_strict_sni`, - `ssl_verify`) move into a nested **Advanced TLS settings - (rarely needed)** Collapse that defaults to closed. A read-only - summary line ("Port 443, ALPN h2,http/1.1, …") shows the safe - defaults that apply unless overridden. - - The Advanced Collapse auto-opens (`defaultActiveKey`) when a - saved draft has any non-default value, so resumed drafts - surface their custom tuning instead of silently hiding it. - - HSTS UI parity for the Phase A - `reject_hsts_preload_without_hsts` validator: the - `hsts_preload` Switch is `disabled` until HSTS is enabled, - `max-age ≥ 31536000`, AND `includeSubDomains=true`. - `hsts_max_age` and `hsts_include_subdomains` are also disabled - while `hsts_enabled=false`. - - TLS min/max ordering UI parity: the `ssl_min_ver` / - `ssl_max_ver` Selects use Antd `dependencies` + a custom - validator that rejects min > max client-side with the same - wording the Phase A model validator uses. - -**Backward compatibility** - -- The existing `/api/sites` POST envelope is unchanged. -- The existing `/api/sites/preview` POST envelope gains an optional - `validation` field that legacy callers can ignore. The - `validate_haproxy_config` flag defaults to `false`, so - `SiteDrafts.handlePreview` and any external integrators keep - their pre-Phase K behaviour. -- `redirect_rules` retains its `List[Union[str, dict]]` shape, so - any historical caller (or saved draft) that used the structured - dict form continues to work. -- The Pydantic safety validators (`system`, `exec`, `eval`, `$(`, - backtick) match the manual frontend API's existing - `validate_acl_rules` posture, which has been in production - blocking the same substrings since pre-R14 with no operator - complaint. No existing wizard payload that previously round- - tripped through `services/haproxy_config.py` can be rejected by - these new validators. -- The `_synthesize_candidate_haproxy_config` helper in - `entities_already_inserted=True` mode is functionally identical - to the previous inline `generate_haproxy_config_for_cluster` - call inside `create_site`. The refactor is pure DRY plumbing. - -##### Rollback considerations (Phase K) - -If you must roll back to a pre-Phase-K v1.5.x build: - -- **Saved drafts** with the new `acl_rules: List[str]` shape are - forward- and backward-compatible: the legacy build also expected - string elements at the renderer level, the rejection only ever - happened at the wizard model boundary. Operators on the legacy - build hit the same 422 the new build is fixing — no DB rewrite - needed. -- **`/api/sites/preview` `validation` block** is a new optional - field; legacy frontend callers ignore unknown fields. The - `validate_haproxy_config` query param default is `false`, so - legacy callers do not exercise the dry-run branch. -- **No DB migrations** are introduced by Phase K. The - `frontends.acl_rules` / `redirect_rules` / `use_backend_rules` - JSONB columns remain unchanged. - -##### Phase K Phase D — Operator-feedback follow-ups (Bulgu #1–#6) - -Operator review of the Phase A–C release surfaced six additional -issues. Each is rooted in a UX inconsistency or a residual stuck -state, and the fixes converge on a "single source of truth + ref- -based dry-run lifecycle" architecture: - -- **Bulgu #1 — Cluster scope.** Pre-fix Step 0 had its own cluster - Select dropdown decoupled from the header. Operators routinely - picked cluster A in the header and cluster B in the wizard with - zero visual signal that the wizard would target a different - cluster than every other tool. Phase D pipes the wizard through - the SAME `ClusterContext` that FrontendManagement / BackendServers - / SSLManagement consume, hides Step 0's `cluster_id` `Form.Item`, - and replaces the picker with a read-only `` display + hint - to change cluster via the header. A `useEffect` keeps - `form.cluster_id` synchronised with `selectedCluster.id` so mid- - wizard header changes propagate; the existing cluster-transition - cleanup effect handles cert-id orphan reconciliation. Resume from - a draft that targets a different cluster now auto-swaps the - header cluster (best-effort `selectCluster()` call) so post- - resume edits stay cluster-consistent. - -- **Bulgu #2 — SSL CA bundle dropdown filter.** Backend's - `BackendServers.js` filters the CA-bundle Select with - `?usage_type=server`, so operators only see certs imported with - the right purpose. The wizard pre-fix surfaced EVERY cert in the - cluster regardless of usage, letting an operator submit a payload - that apply-time HAProxy would parse-error on (`unable to load - SSL private key`). Phase D filters explicitly: - * Per-server CA bundle Select → `usage_type === 'server'`. - * SSL & ACME step's "Existing certificate" Select → - `usage_type === 'frontend'`. - The empty-state Alert was also updated to reason about only the - filtered list so a cluster with N server-side certs but zero - frontend certs renders the "no certs imported" hint correctly. - -- **Bulgu #3 — Stuck "Validating against HAProxy…".** The root - cause was a self-cancel race in the auto-fire `useEffect`. The - effect deps array included `dryRunResult.status`, and the effect - body called `setDryRunResult({status: 'loading'})` at the top. - The status change re-triggered the effect; React's cleanup of the - previous run fired BEFORE the new body, aborting the in-flight - controller; the new body returned early because `status !== - 'idle'`; the aborted fetch's `.catch` block detected - `signal.aborted` and returned without setting state. Status - stayed `'loading'` forever. Audit-fix #1 (round 1) had addressed - the leave-Step-4 cleanup branch but the enter-Step-4 self-abort - was a separate failure mode that only surfaced on a real backend. - Phase D switches the lifecycle to a ref-driven model: - * `dryRunStatusRef` shadows the latest status (synced via a - passive `useEffect`). - * `dryRunInvalidationTick` is the external re-trigger channel; - `onValuesChange` bumps it when the operator edits a Step-4- - visible field (e.g. the Apply Immediately switch). - * The main effect's deps array drops `dryRunResult.status` and - becomes `[step, form, aclBuilderData, dryRunInvalidationTick]` - — none of these change on a self-issued setDryRunResult, so - the self-cancel race is structurally impossible. - * Cleanup nulls the abort ref only if it still points to the - torn-down controller, so a fresh fetch's ref is never - accidentally cleared. - -- **Bulgu #4 — Preview missing fields.** The /api/sites/preview - response previously echoed only a sparse subset of fields, so the - SiteDrafts Preview modal could not show whether per-server - timings, backend cookie persistence, frontend maxconn, HSTS, or - ciphersuites would actually land on disk. Phase D enriches both - the backend response (additive — all existing keys preserved) - AND the SiteDrafts UI: - * Backend: emits the full operator-settable surface area on - `would_create` (backend cookie/timeouts/options, per-server - timings + SSL+CA-bundle details, frontend maxconn/timeouts/ - compression/ACL counts, HTTPS ciphersuites, etc.). - * Frontend: replaces the four flat Descriptions blocks with a - typed renderer that only surfaces NON-DEFAULT values - (`isMeaningful` predicate) so the modal stays scannable. A - dedicated per-server card surfaces every per-server field - the operator customised. HSTS gets its own section when - enabled. - -- **Bulgu #5 — Resume hydration regressions.** Two issues: - 1. Existing certificate was wiped on resume. Root cause was - the orphan-detect effect running on the SAME render that - the resume effect committed the new cluster_id. existingCerts - was still `[]` (fetch in flight), so `certIds = new Set()` - and the freshly-resumed `ssl.ssl_certificate_id` looked like - an orphan and got cleared. Phase D fix: short-circuit the - orphan-detect when `existingCertsLoading=true` and add the - loading flag to the effect deps so the check re-runs after - the fetch settles. ALSO: pin `prevClusterRef.current` to - `merged.cluster_id` BEFORE `form.setFieldsValue(merged)` so - the cluster-transition cleanup effect does not misread the - hydration as a user-driven cluster switch. - 2. The same stuck "Validating against HAProxy…" — resolved by - the Bulgu #3 self-cancel-race fix above. - -- **Bulgu #6 — Create as PENDING button removed.** Pre-fix the - wizard had TWO submit buttons. The "Create as PENDING" button - bypassed the standard manual-flow convention (entity Create → - PENDING version → Apply Management review → operator Apply). The - "Create & Apply" button bypassed Apply Management entirely. - Operators were trained to "always Create & Apply", defeating the - change-review benefit of Apply Management. Phase D consolidates: - * Single button: "Create Site" (or "Create & Apply (ACME)" when - sslMode='acme', because ACME forces the immediate apply for - the HTTP-01 challenge). - * `handleSubmit` derives `effectiveApply` from `sslModeAtSubmit - === 'acme'` — no button-driven branching. - * Non-ACME flow: `apply_immediately=false` → backend returns - `created_pending` → operator is navigated to /apply-management - where they review the bulk version and click Apply (same - Agent-pull cadence as manual entity creation). - * ACME flow: `apply_immediately=true` (M22 model_validator - enforces this) → standard `created_applied` response. - * The `acmeBlocksDraft` derivation that gated the (now-removed) - PENDING button is retired — handleSubmit's `effectiveApply` - replaces the gate. - -##### Phase K Phase D — Backward compatibility / rollback - -- **Cluster picker change.** Operators who relied on the wizard- - internal cluster Select must switch via the header instead. No - data-layer change. Drafts saved on a different cluster - auto-swap the header on resume. -- **`/api/sites/preview` response shape.** Additive only — every - pre-existing key keeps the same shape; new keys are - `cluster_id`, `domains`, additive fields on `backend` / `servers` - / `frontend_http` / `frontend_https`. Legacy frontend callers - ignore unknown fields. -- **`/api/sites` request shape.** Unchanged. -- **No DB migrations** are introduced by Phase K Phase D. - -##### Phase K Phase D — Follow-up audit findings (Bulgu #7–#8) - -A deeper post-implementation audit surfaced two additional -race conditions that were not visible in the first pass. Both -are now resolved on the same `pilot` branch: - -- **Bulgu #7 — Resume cluster swap race on cold mount.** On a - browser refresh of `/sites/new` while a Resume click had - already pre-populated sessionStorage, the wizard mount races - against `ClusterContext`'s `fetchClusters()`. The resume - effect ran with `clustersFromContext=[]`, so - `selectCluster(draftCluster)` was silently skipped. Then - `ClusterContext` finished loading and `selectedCluster` - became the user's `defaultCluster` (NOT the draft's - cluster). The naive header sync then overwrote - `form.cluster_id` with the default cluster, and the - cluster-transition cleanup effect read that overwrite as a - user-driven switch and wiped the draft's cert selections — - the Bulgu #5 second-order failure that survived the - short-circuit fix on a cold mount path. - - Fix: header sync effect grew a one-shot post-resume swap - branch keyed on `resumedFromDraft && !resumeClusterSynced`. - When the draft's `cluster_id` is in the freshly-loaded - `clustersFromContext`, the swap pushes the HEADER to the - draft cluster instead of forcing the form to follow the - header. The `resumeClusterSynced` state gates this to - exactly ONE attempt so a later operator-driven header - cluster change is honoured normally. `selectClusterRef` - (a `useRef(selectCluster)` updated by a tiny sync effect) - keeps the dep set small so the header sync effect does not - re-run on every `ClusterProvider` render. - - Pin: `tests/test_frontend_auth_bootstrap_phase_j.py:: - test_phase_k_phase_d_resume_cluster_swap_race_fix`. - -- **Bulgu #8 — Mid-wizard cluster change leaves stale dry-run.** - When an operator on Step 4 changes the header cluster, the - wizard's cluster_id transitions through `form.setFieldsValue` - (the header sync effect's standard force path). Antd's - `setFieldsValue` is a SILENT update that does NOT fire - `onValuesChange`, so the dry-run invalidation tied to - `onValuesChange` never ran. Result: the Step 4 validation - card kept displaying the PREVIOUS cluster's "clean" verdict - even though the wizard payload now targeted a different - cluster. - - Fix: the cluster-transition cleanup effect (which already - detected the change to wipe stale cert ids) now also resets - `dryRunResult` to idle and bumps `dryRunInvalidationTick` - whenever `dryRunStatusRef.current !== 'idle'`. The dry-run - effect's dep list picks up the tick bump and re-fires - against the new cluster as soon as the operator reaches - Step 4. - - Pin: `tests/test_frontend_auth_bootstrap_phase_j.py:: - test_phase_k_phase_d_cluster_change_invalidates_dry_run`. - -Both fixes are additive (no API or DB changes) and rollback -without leaving residual state — disabling the new effects -simply restores the previous (racy) behaviour. - -##### Phase K Phase D — Operator-feedback round 2 (Bulgu #9–#11) - -A second operator-feedback round surfaced one parity gap and two -follow-ups on the wizard's HAProxy validation experience: - -- **Bulgu #9 — Wizard PEM upload parity with SSL Management page.** - Pre-fix `services.ssl_service.create_cert_row` (the helper the - wizard calls when `ssl.mode='upload'`) was a thin INSERT that - never parsed the PEM. It stored `primary_domain` / `all_domains` - from the operator-entered FRONTEND domains (not the cert SAN), - left `expiry_date` / `issuer` / `fingerprint` NULL, hard-coded - `status='valid'` and `days_until_expiry=0`, never validated the - private key or chain, never checked name uniqueness (so a - duplicate name would 500 at the DB unique constraint), and could - not reactivate a soft-deleted row of the same name. The - resulting cert showed up on the SSL Management page with empty - expiry/issuer columns and a permanent "valid" status — confusing - UX and clearly inconsistent with the dedicated SSL Management - upload flow (`POST /api/ssl/certificates`). - - Fix: `create_cert_row` now mirrors `routers/ssl.py:: - create_ssl_certificate`: - - parses the PEM via `utils.ssl_parser.parse_ssl_certificate` - (raises HTTPException 400 on parse failure), - - validates private_key + chain via `validate_private_key` - / `validate_certificate_chain`, - - computes status / days_until_expiry from the normalised - timezone-naive UTC `expiry_date`, - - enforces name uniqueness within the target cluster (returns - 400 instead of a DB-level 500), - - reactivates soft-deleted rows of the same name (preserves - the row id for downstream references). - - Pin: `tests/test_ssl_service_extraction.py` — 11 tests cover - the happy path, all 6 negative paths (parse fail, empty content, - bad private key, bad chain, duplicate active name, soft-delete - reactivation), and the "metadata comes from PEM, not payload" - contract. - -- **Bulgu #10 — Heuristic validator rejected wizard's own default - timeouts.** The wizard's config synthesis emits `timeout connect - 10000ms` / `timeout server 60000ms` / `timeout client 100ms` - (millisecond suffix is canonical HAProxy syntax). The pre-fix - heuristic regex was `^\d+[smhd]?$`, which only allowed the - single-character suffixes `s`/`m`/`h`/`d` — `ms` was rejected - outright even though the same validator's own suggestion text - said "Use format like '5s', '30000ms', '1m'". Operators saw - 10+ FALSE-POSITIVE "Invalid timeout value '10000ms'" errors on - the wizard's defaults at Step 4 and could not click Create. - - Fix: `utils/haproxy_validator.py::_validate_timeout_directive` - regex relaxed to `^\d+(us|ms|s|m|h|d)?$` — accepting the full - set of HAProxy time-format suffixes (per the HAProxy docs Time - format chapter) while still rejecting malformed values like - `10000xx`, `abc`, `-100ms`, `1.5s`, and bare `ms`. - - Pin: `tests/test_haproxy_validator_timeout_units.py` — 17 - parametrised cases (11 valid formats, 5 invalid formats, plus - the exact operator-reported failure mode). - -- **Bulgu #11 — Operator reported "Previous loses values".** - Architectural review confirmed the wizard's contract is sound: - every step is rendered into a long-lived `
` whose only - step-driven prop is the CSS `display` toggle (`block` vs - `none`). React does NOT unmount the children, Antd's Form.Item - registrations stay intact, and the Antd default `preserve=true` - keeps values in form state even for the inner Form.Items that - conditional-render inside `` (SSL mode - branches, TCP/http frontend mode toggle). All wizard - `setFieldsValue` call-sites are guarded by domain triggers - (cluster change, sslMode change, TCP-mode-clears-https_redirect, - resume hydration) — none fire on a Previous/Next click alone. - - No code regression was identified. Most likely operator - perception driver: with Bulgu #10 fixed, the `timeout - connect=10000` / `timeout server=60000` values the operator - saw in the "Advanced backend settings" Collapse after coming - back from Step 4 are simply the wizard's pre-existing defaults - (`backend.timeout_connect=10000`, `backend.timeout_server= - 60000`, `backend.timeout_queue=60000`), not regressed values - — these were never operator-entered, just defaults the - operator did not notice in the collapsed Advanced section on - the forward pass. - - Defensive measure: a static-source pin test asserts the - architectural contract so a future refactor cannot regress - to per-step conditional rendering or sneak a - `preserve={false}` in: - `tests/test_frontend_auth_bootstrap_phase_j.py:: - test_phase_k_phase_d_wizard_preserves_form_state_across_step_navigation`. - - If the operator can reproduce specific field-level state loss - on a Previous click after the Bulgu #10 fix, please file the - repro steps so we can target the actual scenario. - -##### Rollback considerations (Phase I) - -If you must roll back to a pre-rebrand v1.5.x build after operators have already saved drafts on the new build: - -- New rows on `wizard_drafts` with `wizard_type='site'` will be invisible to the legacy code path that filters on `wizard_type='proxied_host'` only. Operators will see those new drafts disappear from the listing AND will not be counted against the 50-draft cap. The rows themselves are not deleted — they expire via the standard 30-day TTL prune. -- Pre-rebrand rows with `wizard_type='proxied_host'` continue to work on the legacy build because their value never changed. -- The schema-level `DEFAULT` is not rolled back automatically. Operators rolling back can either (a) leave it at `'site'` (harmless — the legacy build hard-codes `'proxied_host'` in every INSERT, so the default is never consulted) or (b) re-run an `ALTER TABLE wizard_drafts ALTER COLUMN wizard_type SET DEFAULT 'proxied_host'` to restore the original schema. - -##### Phase K Phase D — Operator-feedback round 3 (Bulgu #12) - -**Operator-reported failure flow** (May 11, 2026): - -The wizard's Step 4 dry-run showed 8 WARNINGs but no ERRORs, so Create proceeded; the operator then applied via Apply Management and the real `haproxy -c` parse rejected the config: - -``` -[ALERT] parsing [/tmp/haproxy-new-config.cfg:79] : error detected while parsing ACL 'acl1' : failed to open pattern file . -[ALERT] parsing [/tmp/haproxy-new-config.cfg:87] : error detected while parsing switching rule : no such ACL : 'acl1'. -[ALERT] Fatal errors found in configuration. -``` - -The 8 WARNINGs were ALSO operator-confusing false positives: - -``` -[frontend] Directive 'stick-table' may not be valid in 'frontend' section -[frontend] Directive 'tcp-request' may not be valid in 'frontend' section (×2) -[backend] Directive 'cookie' may not be valid in 'backend' section (×2) -[backend] Missing 'global' section - recommended for production -``` - -**Two root causes:** - -1. **Heuristic validator `valid_directives` was incomplete** — `stick-table`, `tcp-request`, `tcp-response`, `cookie`, `http-after-response`, `errorfile`, `description`, `id`, `filter`, etc. are perfectly valid in their respective sections but the validator's small hand-picked sets did not list them. Every wizard / manual page that emitted them flagged a spurious "may not be valid" WARNING. The wizard's pre-persist apply-time gate uses the same validator; even though it only blocks on ERROR-level findings, the noise polluted the operator-visible response trail and the version-history page. - -2. **ACL `-f ` pattern-file references** — the visual ACL builder offered `-f (from file)` as a selectable flag, and neither the manual Frontend API's Pydantic validator (`models/frontend.py::validate_acl_rules`) nor the wizard's Pydantic validator (`models/site_wizard.py::_validate_haproxy_directive_string`) rejected `-f`. HAProxy OpenManager is a fully-managed product: it does NOT provision pattern files onto the HAProxy node's filesystem, so any operator-typed `-f /path/...` ALWAYS resolves to "file not found" at HAProxy reload time. The UI made it trivial to author an unsupported state. - -**Three-layer fix:** - -**Layer A — Heuristic validator** (`backend/utils/haproxy_validator.py`): - -- Expanded `valid_directives['frontend']` to include `stick-table`, `stick`, `tcp-request`, `tcp-response`, `http-after-response`, `errorfile`, `errorloc`, `errorloc302`, `errorloc303`, `http-error`, `description`, `id`, `filter`, `monitor`, `unique-id-format`, `unique-id-header`, `declare`, `http-buffer-request`, plus a long-tail of less-common-but-valid directives. -- Expanded `valid_directives['backend']` to include `cookie`, `appsession`, `tcp-request`, `tcp-response`, `tcp-check`, `retries`, `fullconn`, `dispatch`, `redirect`, `use-server`, `acl`, `capture`, `errorfile`, `description`, `id`, `filter`, `rate-limit`, `email-alert`, `force-persist`, `transparent`, `source`, plus a long-tail. -- Added `partial_fragment: bool = False` parameter to `HAProxyConfigValidator.validate_config()` and the module-level `validate_haproxy_config()`. When True (or auto-detected via the wizard's marker comment), the validator suppresses the "Missing 'global' section" / "Consider adding 'defaults' section" diagnostics — the wizard / cluster synthesis intentionally OMITS those blocks because the agent merges them with its local copy on disk. -- Both the wizard's `/preview` dry-run AND the apply-time pre-persist gate now pass `partial_fragment=True` (`backend/routers/site_wizard.py`). - -**Layer B — ACL `-f` rejection in Pydantic** (server-side gate): - -- `backend/models/site_wizard.py`: Added `_ACL_FILE_FLAG_PATTERN = re.compile(r"(^|\s)-f(\s|$)")` and rejected the pattern inside `_validate_haproxy_directive_string` with an operator-friendly message explaining why the product cannot support pattern files. This covers `acl_rules`, `use_backend_rules`, and string-shaped `redirect_rules`. -- `backend/models/frontend.py::validate_acl_rules`: Mirrored the same rejection on the manual Frontend API so both create paths return the identical 400/422 envelope. - -**Layer C — ACL `-f` removal from the visual builder + UI gates** (client-side authoring guardrail): - -- `frontend/src/components/ACLRuleBuilder.js`: Removed `-f` from the selectable `FLAGS` list. Updated `FLAG_HINTS` to drop the `-f` mention. Existing rules that already carry `-f` (loaded from saved drafts pre-fix) keep the tag visible as `-f (deprecated — remove)` so operators can SEE and REMOVE the flag, but cannot re-add it once removed. Added a section-level red `Alert` that counts every rule carrying `-f` and explains the failure mode + remediation. Inline rule-card error decoration (`status='error'` + red border + inline description) surfaces the same message at the per-rule level. Mirrored the regex client-side so raw-mode typed `-f` immediately flags inline. -- `frontend/src/components/SiteWizard.js`: Added a Step 2 → Step 3 hard-gate on the Next button — if ANY rule still carries `-f`, the click surfaces the same operator-friendly error and refuses to advance. -- `frontend/src/components/FrontendManagement.js::handleSubmit`: Mirrored the same gate so the manual Frontend page rejects submit identically. - -**Backward compatibility:** - -- Existing drafts that contain `-f`-flagged rules still load — the ACLRuleBuilder displays them visibly so operators can remove them. Submit is blocked until they do. -- Existing PERSISTED frontend rows in the DB that already carry `-f` (created before this fix) continue to work at the agent level — the validator changes do NOT retroactively reject them. They can still be EDITED through the UI (which will block save until `-f` is removed) or read via the API for visibility / audit. -- The expanded `valid_directives` sets only ADD entries; nothing previously accepted is now flagged. Pre-existing tests that asserted "Directive X is valid" continue to pass. - -**Tests added:** - -- `backend/tests/test_haproxy_validator_bulgu12.py` (27 new tests): - - Per-directive false-positive regression pins for both frontend and backend sections. - - `partial_fragment=True` suppression + marker-comment auto-detect. - - Wizard Pydantic `-f` rejection across spacing/position variants. - - Anchor-correctness pin: regex must NOT match `-foo` / `-file` substrings inside other tokens. - - Manual Frontend API parity pin. - - End-to-end pin replaying the user's actual config (minus `-f`) with zero spurious WARNINGs. - -- `backend/tests/test_site_wizard_phase2_validator_gate.py`: Widened the pre-window lookback from 400 to 1500 chars to accommodate the partial-fragment forwarding comment block. - -**Rollback considerations:** - -- Reverting the `valid_directives` expansion brings back operator-visible WARNING noise but does NOT break apply (which only gates on ERROR). Safe to roll back if a regression is discovered. -- Reverting the `-f` Pydantic rejection ALLOWS operators to author the failure mode again, but does not break anything that worked before. Roll back ONLY if a customer has pre-provisioned pattern files and a tightly-controlled need to reference them. -- Reverting the ACLRuleBuilder UI changes is a pure visual revert; the Pydantic gate keeps the safety net. - -### Earlier Releases - -For earlier release notes (v1.4.0 ACME stability + enterprise audit, v1.3.0, ...) see the [GitHub Releases](https://github.com/taylanbakircioglu/haproxy-openmanager/releases) page. +For full release notes and the list of features delivered in each version (v1.5.x Site Wizard + ACME Diagnostic Panel, v1.4.0 ACME stability + enterprise audit, v1.3.0, ...) see the [GitHub Releases](https://github.com/taylanbakircioglu/haproxy-openmanager/releases) page. --- diff --git a/backend/main.py b/backend/main.py index 5476063..aa7b334 100644 --- a/backend/main.py +++ b/backend/main.py @@ -8,7 +8,7 @@ import redis import asyncio from datetime import datetime, timedelta -_version_info = {"version": "1.5.1", "releaseName": "Round-23 + Round-24 audit follow-ups", "releaseDate": "2026-05-13"} +_version_info = {"version": "1.5.2", "releaseName": "ACME Diagnostics Panel Hardening", "releaseDate": "2026-05-13"} for _vpath in ["/app/version.json", os.path.join(os.path.dirname(__file__), "..", "version.json")]: try: with open(_vpath) as _vf: diff --git a/backend/routers/acme_diagnostics.py b/backend/routers/acme_diagnostics.py index 6a729df..7d06497 100644 --- a/backend/routers/acme_diagnostics.py +++ b/backend/routers/acme_diagnostics.py @@ -16,9 +16,11 @@ Per-user 5/min rate-limit via user_activity_logs SQL count (M18 / R50). import json import logging +import uuid from datetime import datetime from typing import List, Optional +import asyncpg from fastapi import APIRouter, Header, HTTPException from auth_middleware import check_user_permission, get_current_user_from_token @@ -58,16 +60,39 @@ async def _enforce_rate_limit(conn, user_id: int, action: str) -> None: async def _load_order(conn, order_id: int) -> dict: - row = await conn.fetchrow( - """ - SELECT id, account_id, status, domains, cluster_ids, error_detail, - post_completion_actions, pending_apply_version_name, - wizard_staged_until, created_by - FROM letsencrypt_orders - WHERE id = $1 - """, - order_id, - ) + """Fetch the order row, or raise a clean 404. + + Bulgu #96 (prod-canary audit): `letsencrypt_orders.id` is a Postgres + int4 column. A path-param `order_id` outside the int4 range + (e.g. > 2_147_483_647) used to bubble up as + `asyncpg.exceptions.DataError: invalid input for query argument $1: + ... (value out of int32 range)` — which the diagnostics endpoint + then surfaced in a `diagnostics_unavailable` envelope, leaking the + raw Postgres / asyncpg error string ("query argument $1", + "int32 range") into the operator-visible response body. + Semantically an out-of-range ID can never reference a real order, + so we treat it identically to "row not found" and return a clean + 404 — same shape as the not-found path, no SQL detail leakage. + """ + try: + row = await conn.fetchrow( + """ + SELECT id, account_id, status, domains, cluster_ids, error_detail, + post_completion_actions, pending_apply_version_name, + wizard_staged_until, created_by + FROM letsencrypt_orders + WHERE id = $1 + """, + order_id, + ) + except asyncpg.exceptions.DataError as exc: + logger.info( + "ACME order lookup rejected by Postgres (out-of-range / " + "uncastable id): order_id=%s exc=%s", + order_id, + exc, + ) + raise HTTPException(status_code=404, detail=f"Order {order_id} not found") if not row: raise HTTPException(status_code=404, detail=f"Order {order_id} not found") return dict(row) @@ -86,35 +111,153 @@ def _parse_jsonb_list(raw, default): return default +def _diagnostic_failure_envelope( + order_id: int, + correlation_id: str, + exc: Exception, + *, + stage: str, +) -> dict: + """Build a structured response when the diagnostic suite itself + cannot run. Bulgu #94 (Round-25): we return HTTP 200 with this + envelope rather than 500 so the UI can still SHOW the operator + what happened — the panel's whole purpose is to surface failure + causes, and the panel itself silently 500-ing is the worst-case + UX. The server-side log carries the full traceback keyed by + correlation_id for operator follow-up. + """ + return { + "order_id": order_id, + "status": "diagnostics_unavailable", + "checks": [ + { + "id": "diagnostics_runner", + "label": "Diagnostic runner", + "status": "fail", + "severity": "error", + "message": ( + f"Diagnostics could not run ({stage}): " + f"{exc.__class__.__name__}: {exc}" + ), + "details": { + "stage": stage, + "exception_type": exc.__class__.__name__, + "exception_message": str(exc), + "correlation_id": correlation_id, + "hint": ( + "Check the backend log for correlation_id " + f"{correlation_id} for the full traceback." + ), + }, + "duration_ms": None, + } + ], + "humanized_error": { + "title": "Diagnostic panel could not run", + "message": ( + "The diagnostic runner itself crashed before any check " + "could complete. This is independent of whether the ACME " + "provider is reachable from this cluster." + ), + "hint": ( + "Share the correlation_id below with the platform team; " + "they can grep the API log for the full traceback." + ), + "correlation_id": correlation_id, + }, + "meta": { + "correlation_id": correlation_id, + "error_stage": stage, + "error_type": exc.__class__.__name__, + "error_message": str(exc), + }, + "generated_at": datetime.utcnow().isoformat() + "Z", + } + + @router.post("/orders/{order_id}/diagnostics") async def run_diagnostics(order_id: int, authorization: str = Header(None)): - """Run the full pre-flight + post-failure diagnostic suite.""" + """Run the full pre-flight + post-failure diagnostic suite. + + Bulgu #94 (Round-25 audit): this endpoint must NEVER return HTTP 500 + for an in-suite failure. The diagnostic panel exists precisely to + explain what is broken; producing an opaque 500 defeats the entire + feature. Authentication / authorisation / rate-limit / not-found + errors still raise the appropriate 4xx, but any unexpected + exception from check execution is converted to a 200 response with + a structured failure envelope so the UI can display the cause. + """ current_user = await get_current_user_from_token(authorization) if not await check_user_permission(current_user["id"], "ssl", "read"): raise HTTPException(status_code=403, detail="Insufficient permissions: ssl.read required") + correlation_id = uuid.uuid4().hex[:12] conn = await get_database_connection() try: await _enforce_rate_limit(conn, current_user["id"], "acme_diagnostics_run") order = await _load_order(conn, order_id) + except HTTPException: + await close_database_connection(conn) + raise + except Exception as exc: # noqa: BLE001 — diagnostic boundary + logger.exception( + "ACME diagnostics setup failed for order=%s correlation_id=%s", + order_id, + correlation_id, + ) + try: + return _diagnostic_failure_envelope(order_id, correlation_id, exc, stage="load_order") + finally: + await close_database_connection(conn) + try: domains = _parse_jsonb_list(order["domains"], []) cluster_ids = _parse_jsonb_list(order["cluster_ids"], []) - results = await run_checks( - conn, - domains=domains, - cluster_ids=cluster_ids, - account_id=order["account_id"], - ) + try: + results = await run_checks( + conn, + domains=domains, + cluster_ids=cluster_ids, + account_id=order["account_id"], + ) + except Exception as exc: # noqa: BLE001 — diagnostic boundary + # run_checks now wraps individual checks, but a top-level + # crash (e.g. lost DB connection) still needs to be visible. + logger.exception( + "ACME diagnostics top-level failure for order=%s correlation_id=%s", + order_id, + correlation_id, + ) + return _diagnostic_failure_envelope(order_id, correlation_id, exc, stage="run_checks") - humanized_error = humanize_error_detail(order["error_detail"]) + try: + humanized_error = humanize_error_detail(order["error_detail"]) + except Exception as exc: # noqa: BLE001 — defensive + logger.warning( + "humanize_error_detail failed for order=%s correlation_id=%s: %s", + order_id, + correlation_id, + exc, + ) + humanized_error = { + "title": "ACME error (raw)", + "message": str(order["error_detail"]) if order["error_detail"] else "", + "hint": "", + "parse_error": exc.__class__.__name__, + } return { "order_id": order_id, "status": order["status"], "checks": results, "humanized_error": humanized_error, + "meta": { + "correlation_id": correlation_id, + "checks_total": len(results), + "checks_failed": sum(1 for r in results if r.get("status") == "fail"), + "checks_warn": sum(1 for r in results if r.get("status") == "warn"), + }, "generated_at": datetime.utcnow().isoformat() + "Z", } finally: @@ -127,7 +270,15 @@ async def rerun_diagnostic_check( check_id: str, authorization: str = Header(None), ): - """Re-run a single check (DNS / port80 / routing / account / agents).""" + """Re-run a single check (DNS / port80 / routing / account / agents). + + Bulgu #94 follow-up (Round-25 audit): the rerun path is just as + sensitive to opaque 500s as the full-suite POST. If + `_enforce_rate_limit` / `_load_order` / `run_checks` raises an + unexpected exception, we surface a structured `fail` row in the + same shape the table already renders — so the operator clicking + "Re-run" never sees an opaque toast and the row updates in place. + """ current_user = await get_current_user_from_token(authorization) if not await check_user_permission(current_user["id"], "ssl", "read"): raise HTTPException(status_code=403, detail="Insufficient permissions: ssl.read required") @@ -138,30 +289,119 @@ async def rerun_diagnostic_check( detail=f"Unknown check_id '{check_id}'. Valid: {', '.join(CHECK_IDS)}", ) + correlation_id = uuid.uuid4().hex[:12] conn = await get_database_connection() try: - await _enforce_rate_limit(conn, current_user["id"], "acme_diagnostic_check_rerun") - order = await _load_order(conn, order_id) + try: + await _enforce_rate_limit(conn, current_user["id"], "acme_diagnostic_check_rerun") + order = await _load_order(conn, order_id) + except HTTPException: + raise + except Exception as exc: # noqa: BLE001 — diagnostic boundary + logger.exception( + "ACME rerun setup failed for order=%s check=%s correlation_id=%s", + order_id, + check_id, + correlation_id, + ) + return { + "order_id": order_id, + "check": { + "id": check_id, + "label": check_id, + "status": "fail", + "severity": "error", + "message": ( + f"Re-run setup failed: " + f"{exc.__class__.__name__}: {exc}" + ), + "details": { + "stage": "setup", + "exception_type": exc.__class__.__name__, + "exception_message": str(exc), + "correlation_id": correlation_id, + }, + "duration_ms": None, + }, + "meta": {"correlation_id": correlation_id, "error_stage": "setup"}, + } - domains = _parse_jsonb_list(order["domains"], []) - cluster_ids = _parse_jsonb_list(order["cluster_ids"], []) - - results = await run_checks( - conn, - domains=domains, - cluster_ids=cluster_ids, - account_id=order["account_id"], - only=[check_id], - ) + try: + domains = _parse_jsonb_list(order["domains"], []) + cluster_ids = _parse_jsonb_list(order["cluster_ids"], []) + results = await run_checks( + conn, + domains=domains, + cluster_ids=cluster_ids, + account_id=order["account_id"], + only=[check_id], + ) + except Exception as exc: # noqa: BLE001 — diagnostic boundary + logger.exception( + "ACME rerun run_checks failed for order=%s check=%s correlation_id=%s", + order_id, + check_id, + correlation_id, + ) + return { + "order_id": order_id, + "check": { + "id": check_id, + "label": check_id, + "status": "fail", + "severity": "error", + "message": ( + f"Re-run crashed: " + f"{exc.__class__.__name__}: {exc}" + ), + "details": { + "stage": "run_checks", + "exception_type": exc.__class__.__name__, + "exception_message": str(exc), + "correlation_id": correlation_id, + }, + "duration_ms": None, + }, + "meta": {"correlation_id": correlation_id, "error_stage": "run_checks"}, + } return { "order_id": order_id, "check": results[0] if results else None, + "meta": {"correlation_id": correlation_id}, } finally: await close_database_connection(conn) +async def _user_activity_columns(conn) -> set: + """Return the set of column names actually present on user_activity_logs. + + Bulgu #95 (Round-25 audit) — the canonical migration for + `user_activity_logs` defines `id, user_id, action, resource_type, + resource_id, details, ip_address, user_agent, created_at, timestamp`. + There is NO `status` column. The original `/events` SELECT pulled + `status` directly, so every diagnostic-panel open against an order + that had any user-activity-log correlation raised + `UndefinedColumnError: column "status" does not exist` and the API + returned HTTP 500. We now introspect the schema and only project + the columns that exist, so deployments at any migration level keep + rendering the diagnostic panel. + """ + try: + rows = await conn.fetch( + """ + SELECT column_name + FROM information_schema.columns + WHERE table_name = 'user_activity_logs' + """ + ) + return {r["column_name"] for r in rows} + except Exception as exc: # noqa: BLE001 — schema introspection is best-effort + logger.warning("user_activity_logs schema introspection failed: %s", exc) + return set() + + @router.get("/orders/{order_id}/events") async def get_order_events( order_id: int, @@ -174,6 +414,12 @@ async def get_order_events( resource_id=order_id) for context. Sorted by created_at ASC (oldest first) so the timeline reads naturally. + + Bulgu #94/#95 (Round-25 audit): every sub-query is wrapped so that a + partial failure (missing column, missing table, malformed JSONB) is + surfaced via `meta.errors[]` rather than collapsing the whole panel + to HTTP 500. The diagnostic UI is a debugging surface — it must not + itself become opaque when one of its data sources is degraded. """ current_user = await get_current_user_from_token(authorization) if not await check_user_permission(current_user["id"], "ssl", "read"): @@ -182,95 +428,168 @@ async def get_order_events( if limit <= 0 or limit > 500: limit = 100 + correlation_id = uuid.uuid4().hex[:12] conn = await get_database_connection() + section_errors: List[dict] = [] try: - # Existence check - await _load_order(conn, order_id) - - # Detect whether acme_order_events exists (zero-impact for envs that - # have not yet run the v1.5.0 migration). Returns empty event_log when - # not yet present rather than 500-ing. - events_table_exists = await conn.fetchval( - """ - SELECT EXISTS ( - SELECT 1 FROM information_schema.tables WHERE table_name = 'acme_order_events' + try: + await _load_order(conn, order_id) + except HTTPException: + raise + except Exception as exc: # noqa: BLE001 — surface, don't 500 + logger.exception( + "ACME events load_order failed order=%s correlation_id=%s", + order_id, + correlation_id, ) - """ - ) + return { + "order_id": order_id, + "events": [], + "count": 0, + "meta": { + "correlation_id": correlation_id, + "errors": [ + { + "section": "load_order", + "exception_type": exc.__class__.__name__, + "message": str(exc), + } + ], + }, + } events: List[dict] = [] - if events_table_exists: - event_rows = await conn.fetch( + + # --- Section 1: acme_order_events --- + try: + events_table_exists = await conn.fetchval( """ - SELECT id, event_type, severity, message, details, correlation_id, created_at - FROM acme_order_events - WHERE order_id = $1 - ORDER BY created_at ASC, id ASC - LIMIT $2 - """, + SELECT EXISTS ( + SELECT 1 FROM information_schema.tables + WHERE table_name = 'acme_order_events' + ) + """ + ) + if events_table_exists: + event_rows = await conn.fetch( + """ + SELECT id, event_type, severity, message, details, correlation_id, created_at + FROM acme_order_events + WHERE order_id = $1 + ORDER BY created_at ASC, id ASC + LIMIT $2 + """, + order_id, + limit, + ) + for r in event_rows: + # R18c round 8 (Bulgu A): asyncpg returns JSONB columns as + # raw JSON strings (no codec on the pool). For the FE + # contract the `details` field MUST be either a dict or + # null — otherwise the React renderer ends up trying to + # access `details.foo` on a plain string and silently + # gets undefined. + _det = r["details"] + if isinstance(_det, str): + try: + _det = json.loads(_det) + except Exception: + _det = {} + if not isinstance(_det, (dict, list)): + _det = {} if _det is None else {"raw": str(_det)} + events.append({ + "source": "acme_order_event", + "id": r["id"], + "event_type": r["event_type"], + "severity": r["severity"], + "message": r["message"], + "details": _det, + "correlation_id": r["correlation_id"], + "created_at": r["created_at"].isoformat().replace("+00:00", "Z") + if r["created_at"] else None, + }) + except Exception as exc: # noqa: BLE001 — surface and continue + logger.exception( + "ACME events acme_order_events query failed order=%s correlation_id=%s", order_id, - limit, + correlation_id, ) - for r in event_rows: - # R18c round 8 (Bulgu A): asyncpg returns JSONB columns as - # raw JSON strings (no codec on the pool). For the FE - # contract the `details` field MUST be either a dict or - # null — otherwise the React renderer ends up trying to - # access `details.foo` on a plain string and silently - # gets undefined. - _det = r["details"] - if isinstance(_det, str): - try: - _det = json.loads(_det) - except Exception: - _det = {} - if not isinstance(_det, (dict, list)): - _det = {} if _det is None else {"raw": str(_det)} - events.append({ - "source": "acme_order_event", - "id": r["id"], - "event_type": r["event_type"], - "severity": r["severity"], - "message": r["message"], - "details": _det, - "correlation_id": r["correlation_id"], - "created_at": r["created_at"].isoformat().replace("+00:00", "Z") - if r["created_at"] else None, + section_errors.append({ + "section": "acme_order_events", + "exception_type": exc.__class__.__name__, + "message": str(exc), + }) + + # --- Section 2: user_activity_logs (best-effort, schema-aware) --- + try: + ua_columns = await _user_activity_columns(conn) + if "resource_id" in ua_columns and "resource_type" in ua_columns: + # Project only columns we know exist. `status` is NOT in + # the canonical schema and was the original 500 cause. + projection_candidates = [ + "id", "action", "resource_type", "resource_id", + "details", "created_at", "user_id", "status", + ] + projection = [c for c in projection_candidates if c in ua_columns] + if "id" not in projection or "created_at" not in projection: + raise RuntimeError( + "user_activity_logs is missing required columns " + "(id / created_at) — skipping correlation" + ) + sql = ( + f"SELECT {', '.join(projection)} " + "FROM user_activity_logs " + "WHERE resource_type = 'letsencrypt_order' AND resource_id = $1 " + "ORDER BY created_at ASC, id ASC LIMIT $2" + ) + ua_rows = await conn.fetch(sql, str(order_id), limit) + for r in ua_rows: + rd = dict(r) + raw_details = rd.get("details") + if isinstance(raw_details, str): + msg = raw_details[:500] + details_obj = {} + try: + parsed = json.loads(raw_details) + if isinstance(parsed, (dict, list)): + details_obj = parsed + except Exception: + details_obj = {} + elif isinstance(raw_details, (dict, list)): + msg = "" + details_obj = raw_details + else: + msg = "" + details_obj = {} + raw_status = rd.get("status") or "" + severity = "info" if str(raw_status).lower() in ("success", "ok", "") else "warn" + events.append({ + "source": "user_activity_log", + "id": rd.get("id"), + "event_type": rd.get("action"), + "severity": severity, + "message": msg, + "details": details_obj, + "correlation_id": None, + "created_at": rd["created_at"].isoformat().replace("+00:00", "Z") + if rd.get("created_at") else None, + }) + else: + section_errors.append({ + "section": "user_activity_logs", + "exception_type": "SchemaMissing", + "message": "user_activity_logs lacks resource_type/resource_id columns", }) - - # User activity rows correlated by resource — schema is permissive - # (`resource_type`/`resource_id` may not always be populated for older - # rows) so this query stays best-effort. - ua_rows = await conn.fetch( - """ - SELECT id, action, resource_type, resource_id, status, details, created_at, user_id - FROM user_activity_logs - WHERE resource_type = 'letsencrypt_order' AND resource_id = $1 - ORDER BY created_at ASC, id ASC - LIMIT $2 - """, - str(order_id), - limit, - ) if await conn.fetchval( - """ - SELECT EXISTS ( - SELECT 1 FROM information_schema.columns - WHERE table_name = 'user_activity_logs' AND column_name = 'resource_id' + except Exception as exc: # noqa: BLE001 — surface and continue + logger.exception( + "ACME events user_activity_logs query failed order=%s correlation_id=%s", + order_id, + correlation_id, ) - """ - ) else [] - - for r in ua_rows: - events.append({ - "source": "user_activity_log", - "id": r["id"], - "event_type": r["action"], - "severity": "info" if (r["status"] or "").lower() in ("success", "ok", "") else "warn", - "message": (r["details"] or "")[:500] if isinstance(r["details"], str) else "", - "details": r["details"] if not isinstance(r["details"], (str, type(None))) else {}, - "correlation_id": None, - "created_at": r["created_at"].isoformat().replace("+00:00", "Z") - if r["created_at"] else None, + section_errors.append({ + "section": "user_activity_logs", + "exception_type": exc.__class__.__name__, + "message": str(exc), }) events.sort(key=lambda e: (e["created_at"] or "", e.get("id") or 0)) @@ -279,6 +598,10 @@ async def get_order_events( "order_id": order_id, "events": events, "count": len(events), + "meta": { + "correlation_id": correlation_id, + "errors": section_errors, + }, } finally: await close_database_connection(conn) diff --git a/backend/services/acme_diagnostics.py b/backend/services/acme_diagnostics.py index 2324619..5120b15 100644 --- a/backend/services/acme_diagnostics.py +++ b/backend/services/acme_diagnostics.py @@ -632,6 +632,61 @@ async def check_agents(conn, cluster_ids: List[int]) -> Dict[str, Any]: CHECK_IDS = ("dns", "port80", "routing", "account", "agents") +def _coerce_cluster_ids(raw) -> List[int]: + """Coerce a cluster_ids list to ints, dropping non-integer-compatible + values. JSONB-stored lists occasionally land as ["1", "2"] (string form) + due to legacy paths; asyncpg's `$1::int[]` cast then fails the diagnostic + query with InvalidTextRepresentationError. We normalise here so the + diagnostic surface is the same regardless of how the order was written. + """ + out: List[int] = [] + if not raw: + return out + for v in raw: + try: + out.append(int(v)) + except (TypeError, ValueError): + continue + return out + + +# Bulgu #94 (Round-25 audit) — the entire point of the diagnostic panel +# is to SHOW the operator what went wrong. Pre-fix, a single check raising +# an uncaught exception (e.g. an asyncpg cast error from a malformed +# cluster_ids JSONB, a DNS resolver outage, an SSRF-guard glitch) would +# propagate up to the router's `try/finally` block, which had no `except` +# clause, and return HTTP 500 with no body. The operator saw only +# "Internal Server Error" in DevTools — the inverse of what a diagnostic +# panel should ever produce. We now wrap every check inside `run_checks` +# so that a check crash becomes a structured `fail` row instead of +# bubbling up; the operator gets the exception type + message in the +# UI and can carry it forward, and the rest of the panel still renders. +async def _safe_check(check_id: str, label: str, coro): + """Run an awaitable that produces a check result; swallow exceptions + and convert them to a structured `fail` result so the diagnostic + response is never short-circuited by a single broken check.""" + started = time.time() + try: + return await coro + except Exception as exc: # noqa: BLE001 — diagnostic boundary + duration_ms = int((time.time() - started) * 1000) + logger.exception( + "ACME diagnostic check %s raised", check_id + ) + return _check_result( + check_id, + label, + "fail", + f"Diagnostic check crashed: {exc.__class__.__name__}: {exc}", + severity="error", + details={ + "exception_type": exc.__class__.__name__, + "exception_message": str(exc), + }, + duration_ms=duration_ms, + ) + + async def run_checks( conn, *, @@ -642,19 +697,32 @@ async def run_checks( ) -> List[Dict[str, Any]]: """Execute the full pre-flight check suite. `only` lets callers re-run a subset (per-check rerun in the UI). + + Every individual check is wrapped in `_safe_check` so the diagnostic + endpoint NEVER 500s because of one broken check — the operator gets + a structured `fail` row identifying which check crashed and why. """ selected = set(only) if only else set(CHECK_IDS) results: List[Dict[str, Any]] = [] + # Normalise inputs once so the per-check error stays in the right + # bucket (a malformed cluster_ids should not crash routing/agents). + safe_domains = [d for d in (domains or []) if isinstance(d, str) and d] + safe_cluster_ids = _coerce_cluster_ids(cluster_ids) + try: + safe_account_id = int(account_id) if account_id is not None else None + except (TypeError, ValueError): + safe_account_id = None + if "dns" in selected: - results.append(await check_dns(domains)) + results.append(await _safe_check("dns", "DNS resolution", check_dns(safe_domains))) if "port80" in selected: - results.append(await check_port80(domains)) + results.append(await _safe_check("port80", "Port 80 reachability", check_port80(safe_domains))) if "routing" in selected: - results.append(await check_routing(conn, domains, cluster_ids)) + results.append(await _safe_check("routing", "HAProxy routing", check_routing(conn, safe_domains, safe_cluster_ids))) if "account" in selected: - results.append(await check_account(conn, account_id)) + results.append(await _safe_check("account", "ACME account", check_account(conn, safe_account_id))) if "agents" in selected: - results.append(await check_agents(conn, cluster_ids)) + results.append(await _safe_check("agents", "HAProxy agents", check_agents(conn, safe_cluster_ids))) return results diff --git a/backend/tests/test_acme_diagnostics.py b/backend/tests/test_acme_diagnostics.py index 80be221..d83cc54 100644 --- a/backend/tests/test_acme_diagnostics.py +++ b/backend/tests/test_acme_diagnostics.py @@ -549,3 +549,180 @@ async def test_run_checks_unknown_only_returns_empty(): account_id=None, only=["bogus"], ) assert out == [] + + +# ---------------------------------------------------------------------------- +# Bulgu #94 / #95 (Round-25 audit) — diagnostic-runner robustness +# ---------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_bulgu94_run_checks_swallows_single_check_crash(monkeypatch): + """Bulgu #94 — a single check exception must NOT collapse the suite. + + Pre-fix, an asyncpg UndefinedColumnError from check_agents (e.g. the + Bulgu #84 ``a.last_heartbeat`` typo on an older deploy) propagated + up to the FastAPI router which had no `except`, so the operator saw + HTTP 500 with no body. The diagnostic panel is precisely the place + that should SURFACE this — never opaque-500 it. We now wrap each + check; the failing one becomes a structured `fail` row and the + other four still render. + """ + monkeypatch.setattr(socket, "gethostbyname_ex", + lambda d: (d, [], ["10.0.0.1"])) + + def _ctor(*args, **kwargs): + return _FakeSession(statuses=[200]) + monkeypatch.setattr("aiohttp.ClientSession", _ctor) + + conn = AsyncMock() + + # check_routing + check_agents both call conn.fetch; explode on the + # FIRST call (which is check_routing) and return rows on the second. + call_count = {"n": 0} + + async def _fetch(*args, **kwargs): + call_count["n"] += 1 + if call_count["n"] == 1: + raise RuntimeError("simulated: column a.last_heartbeat does not exist") + return [] + + conn.fetch = _fetch + conn.fetchrow.return_value = None + + out = await run_checks( + conn, + domains=["a.example.com"], + cluster_ids=[1], + account_id=None, + ) + # All five checks must still appear in the response. + ids = [c["id"] for c in out] + assert ids == ["dns", "port80", "routing", "account", "agents"] + routing = next(c for c in out if c["id"] == "routing") + assert routing["status"] == "fail" + assert "Diagnostic check crashed" in routing["message"] + assert routing["details"]["exception_type"] == "RuntimeError" + assert "last_heartbeat" in routing["details"]["exception_message"] + + +@pytest.mark.asyncio +async def test_bulgu94_run_checks_coerces_string_cluster_ids(monkeypatch): + """Bulgu #94 — cluster_ids stored as JSONB strings (legacy paths) + must not crash check_routing / check_agents with + ``invalid input syntax for type integer: "1"``.""" + monkeypatch.setattr(socket, "gethostbyname_ex", + lambda d: (d, [], ["10.0.0.1"])) + + def _ctor(*args, **kwargs): + return _FakeSession(statuses=[200]) + monkeypatch.setattr("aiohttp.ClientSession", _ctor) + + captured_args = [] + + async def _fetch(*args, **kwargs): + captured_args.append(args) + return [] + + conn = AsyncMock() + conn.fetch = _fetch + conn.fetchrow.return_value = None + + out = await run_checks( + conn, + domains=["a.example.com"], + cluster_ids=["1", "2", "garbage", None, 3], + account_id=None, + ) + # The list passed to asyncpg should already be a pure-int list. + # check_routing is the first call that uses cluster_ids. + routing_call_args = [a for a in captured_args if "frontends" in a[0]] + assert routing_call_args, "check_routing should have queried frontends" + cluster_ids_arg = routing_call_args[0][1] + assert cluster_ids_arg == [1, 2, 3], ( + f"cluster_ids must be coerced to ints before being passed to " + f"asyncpg's ::int[] cast; got: {cluster_ids_arg!r}" + ) + assert all(c["status"] != "fail" or c["id"] != "routing" + for c in out + if c["id"] == "routing" and "Diagnostic check crashed" in (c.get("message") or "") + ), "routing should not have crashed on coerced cluster_ids" + + +@pytest.mark.asyncio +async def test_bulgu94_safe_check_does_not_swallow_cancellation(monkeypatch): + """`_safe_check` must catch `Exception` but NOT `BaseException`. + + asyncio.CancelledError is a BaseException (Python 3.8+) so it must + propagate out of `_safe_check` — otherwise a request that the + client cancelled mid-flight would silently keep running diagnostic + checks instead of unwinding cleanly. We hand `_safe_check` a coro + that raises CancelledError and assert it bubbles up. + """ + from services.acme_diagnostics import _safe_check + + async def _cancelled_coro(): + raise asyncio.CancelledError() + + with pytest.raises(asyncio.CancelledError): + await _safe_check("dns", "DNS resolution", _cancelled_coro()) + + +@pytest.mark.asyncio +async def test_bulgu94_safe_check_handles_keyboardinterrupt(monkeypatch): + """`_safe_check` must also not swallow `KeyboardInterrupt`.""" + from services.acme_diagnostics import _safe_check + + async def _interrupt_coro(): + raise KeyboardInterrupt() + + with pytest.raises(KeyboardInterrupt): + await _safe_check("dns", "DNS resolution", _interrupt_coro()) + + +@pytest.mark.asyncio +async def test_bulgu94_coerce_cluster_ids_handles_none(): + """Coercion must handle None input without raising.""" + from services.acme_diagnostics import _coerce_cluster_ids + assert _coerce_cluster_ids(None) == [] + assert _coerce_cluster_ids([]) == [] + assert _coerce_cluster_ids([1, 2, 3]) == [1, 2, 3] + assert _coerce_cluster_ids(["1", "2"]) == [1, 2] + assert _coerce_cluster_ids([1.5]) == [1] # int() truncates floats + assert _coerce_cluster_ids(["abc", None, "5"]) == [5] + + +@pytest.mark.asyncio +async def test_bulgu94_run_checks_filters_invalid_domains(monkeypatch): + """Non-string entries in `domains` must not reach the DNS resolver.""" + seen_domains = [] + + def fake_gethostbyname_ex(domain): + seen_domains.append(domain) + return (domain, [], ["10.0.0.1"]) + + monkeypatch.setattr(socket, "gethostbyname_ex", fake_gethostbyname_ex) + + def _ctor(*args, **kwargs): + return _FakeSession(statuses=[200]) + monkeypatch.setattr("aiohttp.ClientSession", _ctor) + + conn = AsyncMock() + conn.fetch.return_value = [] + conn.fetchrow.return_value = None + + out = await run_checks( + conn, + domains=["a.example.com", None, "", 42, "b.example.com"], + cluster_ids=[1], + account_id=None, + ) + # Both check_dns and check_port80 resolve DNS, so each valid domain + # may appear multiple times — but invalid entries (None, "", 42) + # must never reach the resolver. + assert set(seen_domains) == {"a.example.com", "b.example.com"} + assert None not in seen_domains + assert "" not in seen_domains + assert 42 not in seen_domains + dns_check = next(c for c in out if c["id"] == "dns") + assert dns_check["status"] == "ok" diff --git a/backend/tests/test_acme_diagnostics_router_round25.py b/backend/tests/test_acme_diagnostics_router_round25.py new file mode 100644 index 0000000..74b6c78 --- /dev/null +++ b/backend/tests/test_acme_diagnostics_router_round25.py @@ -0,0 +1,554 @@ +"""Router-level tests for the ACME diagnostic endpoints (Round-25 audit). + +These tests pin the contract introduced by Bulgu #94 / #95: + +* ``POST /api/letsencrypt/orders/{order_id}/diagnostics`` must NEVER return + HTTP 500 for an in-suite failure. Authentication / authorisation / + rate-limit / not-found errors still raise the appropriate 4xx, but any + unexpected exception during check execution is converted to HTTP 200 + with a structured failure envelope so the UI can render the cause. + +* ``GET /api/letsencrypt/orders/{order_id}/events`` must NEVER return + HTTP 500 because of schema drift in ``user_activity_logs`` (the + original 500 cause: SELECTing a non-existent ``status`` column). A + partial failure is reported via ``meta.errors[]``. + +The tests use AsyncMock-based fake connections rather than spinning up a +real Postgres so they run hermetically inside CI. +""" +from unittest.mock import AsyncMock, patch + +import pytest + +from routers import acme_diagnostics as router_mod + + +# ---------------------------------------------------------------------------- +# Helpers +# ---------------------------------------------------------------------------- + + +class _FakeUser(dict): + pass + + +def _patch_auth_and_db(monkeypatch, conn, user_id=1): + """Patch the auth / db helpers used by both endpoints so the tests + don't have to construct a real FastAPI request stack.""" + async def _fake_user(_auth): + return _FakeUser(id=user_id, username="t", email="t@x") + + async def _fake_perm(_uid, *_a, **_k): + return True + + async def _fake_get_conn(): + return conn + + async def _fake_close_conn(_c): + return None + + async def _fake_rate_limit(*_a, **_kw): + return None + + monkeypatch.setattr(router_mod, "get_current_user_from_token", _fake_user) + monkeypatch.setattr(router_mod, "check_user_permission", _fake_perm) + monkeypatch.setattr(router_mod, "get_database_connection", _fake_get_conn) + monkeypatch.setattr(router_mod, "close_database_connection", _fake_close_conn) + monkeypatch.setattr(router_mod, "_enforce_rate_limit", _fake_rate_limit) + + +# ---------------------------------------------------------------------------- +# /diagnostics endpoint +# ---------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_bulgu94_diagnostics_load_order_crash_returns_envelope_not_500(monkeypatch): + """A DB crash during load_order must surface a 200 envelope, not 500. + + Pre-fix, any RuntimeError between auth and run_checks bubbled out of + the bare ``try/finally`` block and FastAPI returned an opaque HTTP 500. + The Round-25 fix wraps load_order so the operator sees the cause + inside the diagnostic panel. + """ + conn = AsyncMock() + conn.fetchrow.side_effect = RuntimeError("simulated DB connectivity loss") + _patch_auth_and_db(monkeypatch, conn) + + out = await router_mod.run_diagnostics(order_id=42, authorization="Bearer x") + + assert out["order_id"] == 42 + assert out["status"] == "diagnostics_unavailable" + assert out["checks"][0]["status"] == "fail" + assert out["checks"][0]["id"] == "diagnostics_runner" + assert "simulated DB connectivity loss" in out["checks"][0]["message"] + assert out["meta"]["correlation_id"] + assert out["meta"]["error_stage"] == "load_order" + assert "humanized_error" in out + + +@pytest.mark.asyncio +async def test_bulgu94_diagnostics_run_checks_crash_returns_envelope(monkeypatch): + """A crash inside run_checks (after order is loaded) must also stay 200.""" + conn = AsyncMock() + conn.fetchrow.return_value = { + "id": 5, + "account_id": 1, + "status": "invalid", + "domains": '["a.example.com"]', + "cluster_ids": "[1]", + "error_detail": None, + "post_completion_actions": None, + "pending_apply_version_name": None, + "wizard_staged_until": None, + "created_by": 1, + } + _patch_auth_and_db(monkeypatch, conn) + + async def _boom(*_a, **_kw): + raise RuntimeError("simulated check orchestrator crash") + + monkeypatch.setattr(router_mod, "run_checks", _boom) + + out = await router_mod.run_diagnostics(order_id=5, authorization="Bearer x") + + assert out["order_id"] == 5 + assert out["status"] == "diagnostics_unavailable" + assert out["meta"]["error_stage"] == "run_checks" + assert out["meta"]["error_type"] == "RuntimeError" + assert "simulated check orchestrator crash" in out["meta"]["error_message"] + + +@pytest.mark.asyncio +async def test_bulgu94_diagnostics_returns_meta_summary_on_success(monkeypatch): + """Successful diagnostics responses carry a meta summary the UI uses + to surface 'N checks failed, M warnings' without recomputing.""" + conn = AsyncMock() + conn.fetchrow.return_value = { + "id": 5, + "account_id": 1, + "status": "invalid", + "domains": '["a.example.com"]', + "cluster_ids": "[1]", + "error_detail": None, + "post_completion_actions": None, + "pending_apply_version_name": None, + "wizard_staged_until": None, + "created_by": 1, + } + _patch_auth_and_db(monkeypatch, conn) + + async def _fake_checks(*_a, **_kw): + return [ + {"id": "dns", "label": "DNS", "status": "ok", "severity": "info", "message": "", "details": {}, "duration_ms": 1}, + {"id": "routing", "label": "Routing", "status": "fail", "severity": "error", "message": "", "details": {}, "duration_ms": 1}, + {"id": "port80", "label": "Port 80", "status": "warn", "severity": "warn", "message": "", "details": {}, "duration_ms": 1}, + ] + + monkeypatch.setattr(router_mod, "run_checks", _fake_checks) + + out = await router_mod.run_diagnostics(order_id=5, authorization="Bearer x") + + assert out["meta"]["checks_total"] == 3 + assert out["meta"]["checks_failed"] == 1 + assert out["meta"]["checks_warn"] == 1 + assert out["meta"]["correlation_id"] + + +# ---------------------------------------------------------------------------- +# /events endpoint +# ---------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_bulgu95_events_missing_status_column_returns_partial_envelope(monkeypatch): + """Bulgu #95 — user_activity_logs lacks a `status` column. + + Pre-fix, the SELECT pulled `status` directly and the endpoint + returned HTTP 500 for every order that had any correlated + user-activity rows. The Round-25 fix introspects the schema; here + we simulate a deployment with no `status` column AND a JOIN / + query that would otherwise crash — the endpoint must stay 200, + return whatever it could collect from acme_order_events, and + record the user_activity_logs section as degraded but recoverable. + """ + conn = AsyncMock() + + # _load_order + order_row = { + "id": 5, + "account_id": 1, + "status": "invalid", + "domains": '["a.example.com"]', + "cluster_ids": "[1]", + "error_detail": None, + "post_completion_actions": None, + "pending_apply_version_name": None, + "wizard_staged_until": None, + "created_by": 1, + } + + fetchval_calls = {"n": 0} + + async def _fetchval(sql, *args): + fetchval_calls["n"] += 1 + # First call: existence check for acme_order_events table + if "acme_order_events" in sql: + return True + return False + + async def _fetchrow(sql, *args): + return order_row + + columns_no_status = [ + {"column_name": "id"}, + {"column_name": "user_id"}, + {"column_name": "action"}, + {"column_name": "resource_type"}, + {"column_name": "resource_id"}, + {"column_name": "details"}, + {"column_name": "created_at"}, + # NOTE: no "status" column — this is the canonical schema. + ] + + async def _fetch(sql, *args): + if "information_schema.columns" in sql and "user_activity_logs" in sql: + return columns_no_status + if "FROM acme_order_events" in sql: + return [] # empty timeline is fine for this test + if "FROM user_activity_logs" in sql: + # If the projection includes `status` we will fail loudly. + assert "status" not in sql, ( + "SELECT must not include `status` when the column is absent; " + f"SQL was: {sql!r}" + ) + return [] + return [] + + conn.fetchval = _fetchval + conn.fetchrow = _fetchrow + conn.fetch = _fetch + + _patch_auth_and_db(monkeypatch, conn) + + out = await router_mod.get_order_events(order_id=5, authorization="Bearer x") + + assert out["order_id"] == 5 + assert out["count"] == 0 + assert out["meta"]["correlation_id"] + # No section errors expected — schema-aware projection silently + # adapted, the panel just got an empty event list. + assert out["meta"]["errors"] == [] + + +@pytest.mark.asyncio +async def test_bulgu95_events_acme_order_events_query_crash_returns_partial(monkeypatch): + """A crash in the acme_order_events sub-query must NOT kill the + whole endpoint — the user_activity_logs section should still run + and the failure must appear in meta.errors.""" + conn = AsyncMock() + order_row = { + "id": 5, + "account_id": 1, + "status": "invalid", + "domains": '["a.example.com"]', + "cluster_ids": "[1]", + "error_detail": None, + "post_completion_actions": None, + "pending_apply_version_name": None, + "wizard_staged_until": None, + "created_by": 1, + } + + async def _fetchval(sql, *args): + if "acme_order_events" in sql: + return True + return False + + async def _fetchrow(sql, *args): + return order_row + + async def _fetch(sql, *args): + if "information_schema.columns" in sql: + return [ + {"column_name": "id"}, {"column_name": "action"}, + {"column_name": "resource_type"}, {"column_name": "resource_id"}, + {"column_name": "details"}, {"column_name": "created_at"}, + ] + if "FROM acme_order_events" in sql: + raise RuntimeError("simulated acme_order_events index corruption") + if "FROM user_activity_logs" in sql: + return [] + return [] + + conn.fetchval = _fetchval + conn.fetchrow = _fetchrow + conn.fetch = _fetch + _patch_auth_and_db(monkeypatch, conn) + + out = await router_mod.get_order_events(order_id=5, authorization="Bearer x") + + assert out["count"] == 0 + error_sections = [e["section"] for e in out["meta"]["errors"]] + assert "acme_order_events" in error_sections + + +@pytest.mark.asyncio +async def test_bulgu95_events_load_order_404_still_raises(monkeypatch): + """The 404 HTTPException raised by `_load_order` for an unknown order + must remain a 404 — the Round-25 envelope is only for *unexpected* + failures, not for client-supplied invalid order IDs.""" + from fastapi import HTTPException + + conn = AsyncMock() + conn.fetchrow.return_value = None # no order found + _patch_auth_and_db(monkeypatch, conn) + + with pytest.raises(HTTPException) as exc_info: + await router_mod.get_order_events(order_id=9999, authorization="Bearer x") + assert exc_info.value.status_code == 404 + + +@pytest.mark.asyncio +async def test_bulgu94_rerun_setup_crash_returns_check_envelope(monkeypatch): + """The single-check re-run path must also envelope, never 500. + + Pre-fix the rerun handler used the same bare ``try/finally`` shape + as the suite POST. If `_load_order` / `_enforce_rate_limit` raised, + the operator clicking the row's "Re-run" button got an opaque + toast and the row never updated. Now the rerun handler returns a + `fail` check shaped the same way the table already renders, so + the row updates in place with the cause + correlation_id. + """ + conn = AsyncMock() + conn.fetchrow.side_effect = RuntimeError("simulated DB drop during rerun") + _patch_auth_and_db(monkeypatch, conn) + + out = await router_mod.rerun_diagnostic_check( + order_id=5, check_id="dns", authorization="Bearer x", + ) + assert out["order_id"] == 5 + assert out["check"]["status"] == "fail" + assert out["check"]["id"] == "dns" + assert "simulated DB drop during rerun" in out["check"]["message"] + assert out["meta"]["error_stage"] == "setup" + assert out["meta"]["correlation_id"] + + +@pytest.mark.asyncio +async def test_bulgu94_rerun_run_checks_crash_returns_check_envelope(monkeypatch): + """A crash inside run_checks during a re-run also stays 200.""" + conn = AsyncMock() + conn.fetchrow.return_value = { + "id": 5, "account_id": 1, "status": "invalid", + "domains": '["a.example.com"]', "cluster_ids": "[1]", + "error_detail": None, "post_completion_actions": None, + "pending_apply_version_name": None, "wizard_staged_until": None, + "created_by": 1, + } + _patch_auth_and_db(monkeypatch, conn) + + async def _boom(*_a, **_kw): + raise RuntimeError("simulated run_checks failure") + + monkeypatch.setattr(router_mod, "run_checks", _boom) + + out = await router_mod.rerun_diagnostic_check( + order_id=5, check_id="agents", authorization="Bearer x", + ) + assert out["check"]["id"] == "agents" + assert out["check"]["status"] == "fail" + assert "simulated run_checks failure" in out["check"]["message"] + assert out["meta"]["error_stage"] == "run_checks" + + +@pytest.mark.asyncio +async def test_bulgu94_rerun_invalid_check_id_still_400(monkeypatch): + """An unknown check_id must remain a 400, not an envelope. The + envelope is only for *unexpected* server-side failures, not for + client-supplied invalid identifiers.""" + from fastapi import HTTPException + + conn = AsyncMock() + _patch_auth_and_db(monkeypatch, conn) + + with pytest.raises(HTTPException) as exc_info: + await router_mod.rerun_diagnostic_check( + order_id=5, check_id="bogus", authorization="Bearer x", + ) + assert exc_info.value.status_code == 400 + + +@pytest.mark.asyncio +async def test_bulgu95_events_status_column_is_used_when_present(monkeypatch): + """If a deployment DID add a `status` column (e.g. via a private + schema extension), the projection picks it up and the resulting + severity reflects it.""" + conn = AsyncMock() + order_row = { + "id": 5, "account_id": 1, "status": "invalid", + "domains": '["a.example.com"]', "cluster_ids": "[1]", + "error_detail": None, "post_completion_actions": None, + "pending_apply_version_name": None, "wizard_staged_until": None, + "created_by": 1, + } + + async def _fetchval(sql, *args): + if "acme_order_events" in sql: + return True + return False + + async def _fetchrow(sql, *args): + return order_row + + captured_sql = {"ua": None} + from datetime import datetime, timezone + + async def _fetch(sql, *args): + if "information_schema.columns" in sql: + return [ + {"column_name": "id"}, {"column_name": "action"}, + {"column_name": "resource_type"}, {"column_name": "resource_id"}, + {"column_name": "details"}, {"column_name": "created_at"}, + {"column_name": "status"}, + ] + if "FROM acme_order_events" in sql: + return [] + if "FROM user_activity_logs" in sql: + captured_sql["ua"] = sql + return [{ + "id": 100, + "action": "letsencrypt.order.create", + "resource_type": "letsencrypt_order", + "resource_id": "5", + "details": '{"foo":"bar"}', + "created_at": datetime(2026, 5, 13, 19, 0, 0, tzinfo=timezone.utc), + "user_id": 7, + "status": "failure", + }] + return [] + + conn.fetchval = _fetchval + conn.fetchrow = _fetchrow + conn.fetch = _fetch + _patch_auth_and_db(monkeypatch, conn) + + out = await router_mod.get_order_events(order_id=5, authorization="Bearer x") + + assert "status" in captured_sql["ua"] + assert out["count"] == 1 + ev = out["events"][0] + assert ev["source"] == "user_activity_log" + assert ev["severity"] == "warn" # status="failure" → warn + assert ev["details"] == {"foo": "bar"} + + +# ---------------------------------------------------------------------------- +# Bulgu #96 (prod-canary follow-up): int4 overflow on order_id must NOT +# leak the asyncpg DataError message ("invalid input for query argument +# $1: ... value out of int32 range") into the operator-facing response +# body. Same shape as the "row not found" path: clean HTTPException(404). +# ---------------------------------------------------------------------------- + + +import asyncpg as _asyncpg # noqa: E402 — late import so the test module +# can still be collected even if asyncpg has changed its exception module. + + +def _data_error(msg: str) -> _asyncpg.exceptions.DataError: + """Construct an asyncpg DataError that mirrors what Postgres returns + when a path-param order_id overflows int4. We can't easily build the + real instance from the binary protocol, so we synthesize one with the + same class so the router's `except asyncpg.exceptions.DataError` + branch is exercised.""" + return _asyncpg.exceptions.DataError(msg) + + +@pytest.mark.asyncio +async def test_bulgu96_diagnostics_int4_overflow_returns_clean_404(monkeypatch): + """``order_id`` outside the int4 range must surface as a clean 404, + NOT as a `diagnostics_unavailable` envelope leaking the asyncpg + DataError message ("query argument $1", "int32 range"). + """ + conn = AsyncMock() + conn.fetchrow.side_effect = _data_error( + "invalid input for query argument $1: 2147483648 (value out of int32 range)" + ) + _patch_auth_and_db(monkeypatch, conn) + + with pytest.raises(router_mod.HTTPException) as excinfo: + await router_mod.run_diagnostics( + order_id=2_147_483_648, + authorization="Bearer x", + ) + + assert excinfo.value.status_code == 404 + # The operator must see the canonical "not found" detail, NOT the + # raw asyncpg error message. + assert "not found" in str(excinfo.value.detail).lower() + assert "int32" not in str(excinfo.value.detail).lower() + assert "query argument" not in str(excinfo.value.detail).lower() + + +@pytest.mark.asyncio +async def test_bulgu96_events_int4_overflow_returns_clean_404(monkeypatch): + """Same contract on the events endpoint — out-of-range order_id is a + clean 404, not a `meta.errors[]` envelope leaking SQL detail.""" + conn = AsyncMock() + conn.fetchrow.side_effect = _data_error( + "invalid input for query argument $1: 9999999999 (value out of int32 range)" + ) + _patch_auth_and_db(monkeypatch, conn) + + with pytest.raises(router_mod.HTTPException) as excinfo: + await router_mod.get_order_events( + order_id=9_999_999_999, + authorization="Bearer x", + ) + + assert excinfo.value.status_code == 404 + assert "not found" in str(excinfo.value.detail).lower() + assert "int32" not in str(excinfo.value.detail).lower() + + +@pytest.mark.asyncio +async def test_bulgu96_rerun_int4_overflow_returns_clean_404(monkeypatch): + """Same contract on the per-check rerun endpoint — out-of-range + order_id is a clean 404, not a structured ``check.fail`` envelope + leaking the asyncpg DataError message.""" + conn = AsyncMock() + conn.fetchrow.side_effect = _data_error( + "invalid input for query argument $1: 5000000000 (value out of int32 range)" + ) + _patch_auth_and_db(monkeypatch, conn) + + with pytest.raises(router_mod.HTTPException) as excinfo: + await router_mod.rerun_diagnostic_check( + order_id=5_000_000_000, + check_id="dns", + authorization="Bearer x", + ) + + assert excinfo.value.status_code == 404 + assert "not found" in str(excinfo.value.detail).lower() + assert "int32" not in str(excinfo.value.detail).lower() + + +@pytest.mark.asyncio +async def test_bulgu96_load_order_dataerror_does_not_leak_correlation_envelope(monkeypatch): + """Belt-and-braces: even when the DataError carries other kinds of + invalid-input strings (e.g. type coercion failure on an int4 + column), `_load_order` must still answer with the canonical 404 + envelope and NOT route it through `_diagnostic_failure_envelope` + (which would surface the raw SQL detail to the UI).""" + conn = AsyncMock() + conn.fetchrow.side_effect = _data_error("invalid integer literal: 'NaN'") + _patch_auth_and_db(monkeypatch, conn) + + with pytest.raises(router_mod.HTTPException) as excinfo: + await router_mod.run_diagnostics(order_id=123, authorization="Bearer x") + + assert excinfo.value.status_code == 404 + # No SQL detail leak + assert "invalid integer literal" not in str(excinfo.value.detail).lower() + assert "NaN" not in str(excinfo.value.detail) diff --git a/backend/tests/test_haproxy_validator_bulgu12.py b/backend/tests/test_haproxy_validator_bulgu12.py index 45ca917..ea661d6 100644 --- a/backend/tests/test_haproxy_validator_bulgu12.py +++ b/backend/tests/test_haproxy_validator_bulgu12.py @@ -6363,9 +6363,16 @@ def test_bulgu83_static_marker_in_fe_warning_toast(): calls these rules "legacy" and now lists each offending rule body. Static-source check so a refactor that re-introduces the misleading wording or drops the rule snippets is caught. + + Skipped automatically when the test runs inside the backend + Dockerfile build context (which only copies `backend/` and + therefore has no `frontend/` tree next to it). This mirrors + the guard already used by every other front-end static-source + pin in this file (e.g. lines 583-592, 795-810, 833-845, etc.) + — Bulgu #83's pin was missing it, which broke the corporate + CI's `RUN python -m pytest` step in the backend Docker build + immediately after Round-23 went live. """ - # The frontend tree lives next to backend/ at the workspace - # root, so walk up one extra level from _BACKEND_DIR. fm_path = ( _BACKEND_DIR.parent / "frontend" @@ -6373,6 +6380,11 @@ def test_bulgu83_static_marker_in_fe_warning_toast(): / "components" / "FrontendManagement.js" ) + if not fm_path.exists(): + pytest.skip( + f"frontend not present at {fm_path}; running in backend-only " + "container is expected — skip JS source pin" + ) fm_src = fm_path.read_text() assert "Bulgu #83 (round-23 audit)" in fm_src # No more `legacy routing/redirect rule(s)` wording. diff --git a/docs/screenshots/site-wizard.png b/docs/screenshots/site-wizard.png new file mode 100644 index 0000000..01cbc48 Binary files /dev/null and b/docs/screenshots/site-wizard.png differ diff --git a/frontend/package.json b/frontend/package.json index 0bf7693..2773838 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,8 @@ { "name": "haproxy-openmanager-frontend", - "version": "1.5.1", + "version": "1.5.2", "description": "HAProxy Load Balancer Management UI", + "license": "AGPL-3.0-or-later", "dependencies": { "react": "^18.2.0", "react-dom": "^18.2.0", diff --git a/frontend/src/components/ACMEAutomation.js b/frontend/src/components/ACMEAutomation.js index 15ba50c..0ad0350 100644 --- a/frontend/src/components/ACMEAutomation.js +++ b/frontend/src/components/ACMEAutomation.js @@ -92,7 +92,15 @@ const ACMEAutomation = () => { const [diagEvents, setDiagEvents] = useState([]); const [diagLoading, setDiagLoading] = useState(false); const [diagRunningCheckId, setDiagRunningCheckId] = useState(null); + // Bulgu #94 (Round-25 audit): the diagnostic panel's job is to make + // failures visible. We now keep dedicated error envelopes for the + // diagnostics POST and the /events GET so the modal can render them + // inline instead of silently dropping them like the v1.5.1 build did. + const [diagRunError, setDiagRunError] = useState(null); + const [diagEventsError, setDiagEventsError] = useState(null); + const [diagMeta, setDiagMeta] = useState(null); const diagPollRef = useRef(null); + const diagPollFailCountRef = useRef(0); const fetchData = useCallback(async () => { setLoading(true); @@ -231,6 +239,12 @@ const ACMEAutomation = () => { }; // v1.5.0 Issue #13: open diagnostics modal for an order + // Bulgu #94 (Round-25 audit): the modal is the operator's last line + // of defence when ACME goes sideways — it MUST render failure causes + // verbatim instead of swallowing them. We capture both the + // diagnostics POST and the events GET errors into dedicated state + // and stop the auto-tail poll after consecutive failures so the + // network tab does not get spammed with /events 500s every 5s. const handleDiagnose = async (orderId) => { setDiagOrderId(orderId); setDiagVisible(true); @@ -238,6 +252,10 @@ const ACMEAutomation = () => { setDiagChecks([]); setDiagHumanizedError(null); setDiagEvents([]); + setDiagRunError(null); + setDiagEventsError(null); + setDiagMeta(null); + diagPollFailCountRef.current = 0; try { const [orderRes, diagRes] = await Promise.allSettled([ axios.get(`/api/letsencrypt/orders/${orderId}`), @@ -245,18 +263,68 @@ const ACMEAutomation = () => { ]); if (orderRes.status === 'fulfilled') setDiagOrder(orderRes.value.data); if (diagRes.status === 'fulfilled') { - setDiagChecks(diagRes.value.data?.checks || []); - setDiagHumanizedError(diagRes.value.data?.humanized_error || null); + const data = diagRes.value.data || {}; + setDiagChecks(data.checks || []); + setDiagHumanizedError(data.humanized_error || null); + setDiagMeta(data.meta || null); + // Bulgu #94 follow-up: the Round-25 backend now returns HTTP 200 + // with `status: 'diagnostics_unavailable'` + `meta.error_stage` + // instead of HTTP 500 when the diagnostic runner itself crashes + // (e.g. `column a.last_heartbeat does not exist` on a stale + // deploy). The fulfilled branch must therefore detect the + // envelope and surface the structured failure alert — otherwise + // the operator would only see one `diagnostics_runner` row in + // the table without the prominent red banner that explains the + // crash + correlation_id. This is exactly the "panel renders + // but doesn't visibly say WHY" trap we're trying to avoid. + if (data.status === 'diagnostics_unavailable' || data?.meta?.error_stage) { + setDiagRunError({ + status: 200, + message: data?.meta?.error_message + || data?.checks?.[0]?.message + || 'Diagnostic runner failed', + correlation_id: data?.meta?.correlation_id || null, + error_stage: data?.meta?.error_stage || null, + error_type: data?.meta?.error_type || null, + }); + } } else if (diagRes.status === 'rejected') { - const detail = diagRes.reason?.response?.data?.detail || 'Diagnostics failed'; + const resp = diagRes.reason?.response; + const detail = resp?.data?.detail + || resp?.data?.error?.message + || diagRes.reason?.message + || 'Diagnostics failed'; + setDiagRunError({ + status: resp?.status || 0, + message: detail, + correlation_id: resp?.data?.error?.correlation_id || resp?.headers?.['x-correlation-id'] || null, + }); message.error(detail); } - // Best-effort merged event log fetch (404 if migration not yet run) + // Best-effort merged event log fetch — bug #95: server-side schema + // drift used to return 500; we now surface that in the modal so + // the operator sees "events unavailable because ". try { const evRes = await axios.get(`/api/letsencrypt/orders/${orderId}/events`); setDiagEvents(evRes.data?.events || []); - } catch (_evErr) { - // ignore + if (evRes.data?.meta?.errors?.length) { + setDiagEventsError({ + kind: 'partial', + errors: evRes.data.meta.errors, + correlation_id: evRes.data.meta.correlation_id, + }); + } + } catch (evErr) { + const resp = evErr?.response; + setDiagEventsError({ + kind: 'fatal', + status: resp?.status || 0, + message: resp?.data?.detail + || resp?.data?.error?.message + || evErr?.message + || 'Event log unavailable', + correlation_id: resp?.data?.error?.correlation_id || null, + }); } } finally { setDiagLoading(false); @@ -299,8 +367,38 @@ const ACMEAutomation = () => { try { const evRes = await axios.get(`/api/letsencrypt/orders/${diagOrderId}/events`); setDiagEvents(evRes.data?.events || []); - } catch (_e) { - /* ignore */ + diagPollFailCountRef.current = 0; + if (evRes.data?.meta?.errors?.length) { + setDiagEventsError({ + kind: 'partial', + errors: evRes.data.meta.errors, + correlation_id: evRes.data.meta.correlation_id, + }); + } else { + setDiagEventsError(null); + } + } catch (e) { + // Bulgu #94 (Round-25): stop the auto-tail after 3 consecutive + // failures so a broken backend doesn't drown the user's + // network tab in 500s. Operator can re-open the modal to retry. + diagPollFailCountRef.current += 1; + if (diagPollFailCountRef.current >= 3) { + if (diagPollRef.current) { + clearInterval(diagPollRef.current); + diagPollRef.current = null; + } + const resp = e?.response; + setDiagEventsError({ + kind: 'fatal', + status: resp?.status || 0, + message: resp?.data?.detail + || resp?.data?.error?.message + || e?.message + || 'Event log polling stopped after repeated failures', + correlation_id: resp?.data?.error?.correlation_id || null, + polling_stopped: true, + }); + } } }, 5000); return () => { @@ -318,6 +416,10 @@ const ACMEAutomation = () => { setDiagChecks([]); setDiagHumanizedError(null); setDiagEvents([]); + setDiagRunError(null); + setDiagEventsError(null); + setDiagMeta(null); + diagPollFailCountRef.current = 0; if (diagPollRef.current) { clearInterval(diagPollRef.current); diagPollRef.current = null; @@ -1100,8 +1202,50 @@ const ACMEAutomation = () => { label: 'Pre-flight Checks', children: (
+ {/* Bulgu #94 (Round-25): expose backend failures so the + operator can act on them — not just see a blank panel. */} + {diagRunError && ( + +
{diagRunError.message}
+ {(diagRunError.error_stage || diagRunError.error_type) && ( +
+ {diagRunError.error_stage && Stage: {diagRunError.error_stage} } + {diagRunError.error_type && · Type: {diagRunError.error_type}} +
+ )} + {diagRunError.correlation_id && ( +
+ Correlation ID: {diagRunError.correlation_id} +
+ )} +
+ Share this correlation ID with the platform team; the full traceback is in the API log. +
+
+ } + /> + )} + {diagMeta?.correlation_id && (diagMeta.checks_failed > 0 || diagMeta.checks_warn > 0) && !diagRunError && ( + 0 ? 'warning' : 'info'} + showIcon + style={{ marginBottom: 12 }} + message={`${diagMeta.checks_failed} check(s) failed, ${diagMeta.checks_warn} warning(s)`} + description={ + + Correlation ID: {diagMeta.correlation_id} + + } + /> + )} {diagChecks.length === 0 ? ( - + ) : ( { key: 'events', label: `Event Log (${diagEvents.length})`, children: ( - diagEvents.length === 0 ? ( - - ) : ( - ({ - color: - (ev.severity || '').toUpperCase() === 'ERROR' ? 'red' : - (ev.severity || '').toUpperCase() === 'WARN' ? 'orange' : 'blue', - children: ( +
+ {/* Bulgu #95 (Round-25): /events used to 500 because of + a schema-drift bug (`status` column did not exist). + Now the response carries `meta.errors[]` for partial + failures and a `kind: fatal` envelope for total + failure — both render here so the operator never + wonders why the timeline is empty. */} + {diagEventsError?.kind === 'fatal' && ( + -
{ev.created_at} · {ev.source}
-
{ev.event_type}
- {ev.message &&
{ev.message}
} +
{diagEventsError.message}
+ {diagEventsError.correlation_id && ( +
+ Correlation ID: {diagEventsError.correlation_id} +
+ )} + {diagEventsError.polling_stopped && ( +
+ Auto-refresh stopped after repeated failures. Re-open the panel to retry. +
+ )}
- ), - }))} - /> - ) + } + /> + )} + {diagEventsError?.kind === 'partial' && ( + +
    + {diagEventsError.errors.map((err, i) => ( +
  • + {err.section}: {err.exception_type} — {err.message} +
    • + ))} +
+ {diagEventsError.correlation_id && ( +
+ Correlation ID: {diagEventsError.correlation_id} +
+ )} + + } + /> + )} + {diagEvents.length === 0 ? ( + + ) : ( + ({ + color: + (ev.severity || '').toUpperCase() === 'ERROR' ? 'red' : + (ev.severity || '').toUpperCase() === 'WARN' ? 'orange' : 'blue', + children: ( +
+
{ev.created_at} · {ev.source}
+
{ev.event_type}
+ {ev.message &&
{ev.message}
} +
+ ), + }))} + /> + )} + ), }, { diff --git a/version.json b/version.json index b365943..1778d1f 100644 --- a/version.json +++ b/version.json @@ -1,5 +1,5 @@ { - "version": "1.5.1", - "releaseName": "Round-23 + Round-24 audit follow-ups", + "version": "1.5.2", + "releaseName": "ACME Diagnostics Panel Hardening", "releaseDate": "2026-05-13" }