diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index 3188b338a..394c7b548 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -33,7 +33,7 @@ jobs: # working-directory: somedir # Optional: golangci-lint command line arguments. - args: --build-tags apitests + # args: --build-tags apitests # Optional: show only new issues if it's a pull request. The default value is `false`. # only-new-issues: true diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 2bfc23781..99c752626 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -127,12 +127,6 @@ jobs: CC=arm-linux-gnueabihf-gcc $(go env GOPATH)/bin/gox -cgo -os=linux -arch=arm -output=$GOX_OUTPUT ./... CC=aarch64-linux-gnu-gcc $(go env GOPATH)/bin/gox -cgo -os=linux -arch=arm64 -output=$GOX_OUTPUT ./... - - name: Run Go Quickstart Test - timeout-minutes: 5 - shell: bash - run: | - go test -v -tags "quickstart automated" ./ziti/cmd/edge/...; - - name: Upload artifacts uses: actions/upload-artifact@v3 with: @@ -164,7 +158,7 @@ jobs: go test -v -tags "quickstart automated" ./ziti/cmd/edge/...; - name: Run Unit and Integration Tests - timeout-minutes: 5 + timeout-minutes: 10 shell: bash run: | go test ./... --tags apitests diff --git a/.golangci.yml b/.golangci.yml index 3e112cc29..e27f49a58 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -1,4 +1,6 @@ run: + build-tags: + - apitests deadline: 8m skip-files: - ziti/cmd/edge/verify_ca.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 9f9113964..b49f172b1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,11 @@ ## What's New +* `ziti edge quickstart` +* Edge SDK terminator improvements + +## `ziti edge quickstart` + * `ziti edge quickstart`](https://github.com/openziti/ziti/issues/1298). You can now download the `ziti` CLI and have a functioning network with just one command. The network it creates is ephemeral and is intended to be torn down when the process exits. @@ -33,10 +38,46 @@ --password someOtherPassword ``` -* Bugfixes +## Edge SDK Terminator Improvements + +There was a race condition in edge sdk terminator handling, where if sdk noticed a broken connection before the router did, +it would reconnect and rebind while the router still though it had the old connection and old binding. Because we were +using the session token to key terminator state in the router, the new terminator information would overwrite the old +terminator information in the router. However, in the controller, the information wouldn't get overridden, since we use a +UUID to key things in the controller. When the router noticed the old connection was gone it would try to clean things +up, but since the state had been overwritten, it couldn't and the controller would be left with an orphaned terminator. + +The router now uses a UUID as well so there shouldn't be any more orphaned terminators. ## Component Updates and Bug Fixes +* github.com/openziti/channel/v2: [v2.0.95 -> v2.0.99](https://github.com/openziti/channel/compare/v2.0.95...v2.0.99) +* github.com/openziti/edge-api: [v0.25.33 -> v0.25.37](https://github.com/openziti/edge-api/compare/v0.25.33...v0.25.37) + * [Issue #44](https://github.com/openziti/edge-api/issues/44) - create session can return 404 if service id is invalid + +* github.com/openziti/fabric: [v0.24.23 -> v0.24.36](https://github.com/openziti/fabric/compare/v0.24.23...v0.24.36) + * [Issue #799](https://github.com/openziti/fabric/issues/799) - Alll controllers to advertise capabilities to routers/other controllers + * [Issue #796](https://github.com/openziti/fabric/issues/796) - Make link heartbeats configurable, including an unresponive close timeout + * [Issue #794](https://github.com/openziti/fabric/issues/794) - Add output file to ziti agent controller snapshot-db command + * [Issue #792](https://github.com/openziti/fabric/issues/792) - include raft index in DB snapshot filename + * [Issue #791](https://github.com/openziti/fabric/issues/791) - FieldError Reason field not parsed correct when transmitted from cluster node to cluster node + * [Issue #789](https://github.com/openziti/fabric/issues/789) - Retransmitting prevents circuit from being idle checked + +* github.com/openziti/foundation/v2: [v2.0.30 -> v2.0.32](https://github.com/openziti/foundation/compare/v2.0.30...v2.0.32) +* github.com/openziti/identity: [v1.0.61 -> v1.0.63](https://github.com/openziti/identity/compare/v1.0.61...v1.0.63) +* github.com/openziti/metrics: [v1.2.33 -> v1.2.35](https://github.com/openziti/metrics/compare/v1.2.33...v1.2.35) +* github.com/openziti/runzmd: [v1.0.30 -> v1.0.32](https://github.com/openziti/runzmd/compare/v1.0.30...v1.0.32) +* github.com/openziti/sdk-golang: [v0.20.101 -> v0.20.116](https://github.com/openziti/sdk-golang/compare/v0.20.101...v0.20.116) + * [Issue #431](https://github.com/openziti/sdk-golang/issues/431) - sdk can panic when re-authenticating and rebinding at the same time + * [Issue #238](https://github.com/openziti/sdk-golang/issues/238) - service binding needs to restart if service is recreated + * [Issue #115](https://github.com/openziti/sdk-golang/issues/115) - Don't use hostname for CN in CSR + * [Issue #429](https://github.com/openziti/sdk-golang/issues/429) - error handling is broken, so we don't properly retry + +* github.com/openziti/secretstream: [v0.1.11 -> v0.1.12](https://github.com/openziti/secretstream/compare/v0.1.11...v0.1.12) +* github.com/openziti/storage: [v0.2.14 -> v0.2.18](https://github.com/openziti/storage/compare/v0.2.14...v0.2.18) + * [Issue #31](https://github.com/openziti/storage/issues/31) - Fix ANTLR int overflow error on arm 32 + +* github.com/openziti/transport/v2: [v2.0.103 -> v2.0.107](https://github.com/openziti/transport/compare/v2.0.103...v2.0.107) * github.com/openziti/ziti: [v0.30.3 -> v0.30.4](https://github.com/openziti/ziti/compare/v0.30.3...v0.30.4) * Fixed an issue causing router configs to be rewritten when docker compose was brought up with existing configs diff --git a/controller/handler_edge_ctrl/remove_terminator.go b/controller/handler_edge_ctrl/remove_terminator.go index e9c62da3c..a34c90b18 100644 --- a/controller/handler_edge_ctrl/remove_terminator.go +++ b/controller/handler_edge_ctrl/remove_terminator.go @@ -22,7 +22,6 @@ import ( "github.com/openziti/ziti/common" "github.com/openziti/ziti/common/pb/edge_ctrl_pb" "github.com/openziti/ziti/controller/env" - "github.com/openziti/ziti/controller/persistence" "github.com/sirupsen/logrus" "google.golang.org/protobuf/proto" ) @@ -84,26 +83,11 @@ func (self *removeTerminatorHandler) RemoveTerminator(ctx *RemoveTerminatorReque return } - ctx.loadSession(ctx.req.SessionToken) - if ctx.err != nil { - // if the session is invalid, we still want to delete the terminator if the session is gone, but - // the terminator matches the sessions - if terminator.Address == "hosted:"+ctx.req.SessionToken { - ctx.err = nil - } else { - self.returnError(ctx, ctx.err) - return - } - } else { - ctx.checkSessionType(persistence.SessionTypeBind) - ctx.checkSessionFingerprints(ctx.req.Fingerprints) - } - - if ctx.err != nil { - self.returnError(ctx, ctx.err) - return - } - + // SDKs can't request terminator deletes directly, they can only do so through the router. + // The router will only ask to delete terminators that belong to the SDK, so we shouldn't need + // to check the session again here. The session may already be deleted, and if it is, we don't + // currently have a way to verify that it's associated. Also, with idempotent terminators, a + // terminator may belong to a series of sessions. err := self.getNetwork().Terminators.Delete(ctx.req.TerminatorId, ctx.newChangeContext()) if err != nil { self.returnError(ctx, internalError(err)) diff --git a/go.mod b/go.mod index dc873474b..e04c92b51 100644 --- a/go.mod +++ b/go.mod @@ -11,7 +11,6 @@ require ( github.com/cenkalti/backoff/v4 v4.2.1 github.com/coreos/go-iptables v0.6.0 github.com/dgryski/dgoogauth v0.0.0-20190221195224-5a805980a5f3 - github.com/dineshappavoo/basex v0.0.0-20170425072625-481a6f6dc663 github.com/fatih/color v1.15.0 github.com/fullsailor/pkcs7 v0.0.0-20190404230743-d7302db945fa github.com/go-acme/lego/v4 v4.2.0 @@ -37,14 +36,14 @@ require ( github.com/mitchellh/mapstructure v1.5.0 github.com/openziti/agent v1.0.15 github.com/openziti/channel/v2 v2.0.99 - github.com/openziti/edge-api v0.25.36 + github.com/openziti/edge-api v0.25.37 github.com/openziti/fabric v0.24.36 github.com/openziti/foundation/v2 v2.0.32 github.com/openziti/identity v1.0.63 github.com/openziti/jwks v1.0.3 github.com/openziti/metrics v1.2.35 - github.com/openziti/runzmd v1.0.31 - github.com/openziti/sdk-golang v0.20.112 + github.com/openziti/runzmd v1.0.32 + github.com/openziti/sdk-golang v0.20.116 github.com/openziti/secretstream v0.1.12 github.com/openziti/storage v0.2.18 github.com/openziti/transport/v2 v2.0.107 @@ -93,6 +92,7 @@ require ( github.com/c-bata/go-prompt v0.2.6 // indirect github.com/creack/pty v1.1.11 // indirect github.com/davecgh/go-spew v1.1.1 // indirect + github.com/dineshappavoo/basex v0.0.0-20170425072625-481a6f6dc663 // indirect github.com/disintegration/imaging v1.6.2 // indirect github.com/dlclark/regexp2 v1.10.0 // indirect github.com/docker/go-units v0.5.0 // indirect @@ -111,7 +111,7 @@ require ( github.com/go-openapi/swag v0.22.4 // indirect github.com/go-openapi/validate v0.22.1 // indirect github.com/golang/protobuf v1.5.3 // indirect - github.com/gomarkdown/markdown v0.0.0-20230916125811-7478c230c7cd // indirect + github.com/gomarkdown/markdown v0.0.0-20230922112808-5421fefb8386 // indirect github.com/gorilla/handlers v1.5.1 // indirect github.com/gorilla/schema v1.2.0 // indirect github.com/gorilla/securecookie v1.1.1 // indirect @@ -171,9 +171,9 @@ require ( github.com/yusufpapurcu/wmi v1.2.3 // indirect go.mongodb.org/mongo-driver v1.12.1 // indirect go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352 // indirect - go.opentelemetry.io/otel v1.18.0 // indirect - go.opentelemetry.io/otel/metric v1.18.0 // indirect - go.opentelemetry.io/otel/trace v1.18.0 // indirect + go.opentelemetry.io/otel v1.19.0 // indirect + go.opentelemetry.io/otel/metric v1.19.0 // indirect + go.opentelemetry.io/otel/trace v1.19.0 // indirect golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect golang.org/x/image v0.12.0 // indirect golang.org/x/mod v0.12.0 // indirect diff --git a/go.sum b/go.sum index fba27cc5d..c63916677 100644 --- a/go.sum +++ b/go.sum @@ -381,8 +381,8 @@ github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8l github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/golangci/lint-1 v0.0.0-20181222135242-d2cdd8c08219/go.mod h1:/X8TswGSh1pIozq4ZwCfxS0WA5JGXguxk94ar/4c87Y= github.com/gomarkdown/markdown v0.0.0-20191123064959-2c17d62f5098/go.mod h1:aii0r/K0ZnHv7G0KF7xy1v0A7s2Ljrb5byB7MO5p6TU= -github.com/gomarkdown/markdown v0.0.0-20230916125811-7478c230c7cd h1:laCEzrtkKEkT2424vMTGl6N1m0xN8kq371hksD5Be+8= -github.com/gomarkdown/markdown v0.0.0-20230916125811-7478c230c7cd/go.mod h1:JDGcbDT52eL4fju3sZ4TeHGsQwhG9nbDV21aMyhwPoA= +github.com/gomarkdown/markdown v0.0.0-20230922112808-5421fefb8386 h1:EcQR3gusLHN46TAD+G+EbaaqJArt5vHhNpXAa12PQf4= +github.com/gomarkdown/markdown v0.0.0-20230922112808-5421fefb8386/go.mod h1:JDGcbDT52eL4fju3sZ4TeHGsQwhG9nbDV21aMyhwPoA= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= @@ -712,8 +712,8 @@ github.com/openziti/channel/v2 v2.0.99 h1:/Ji5fxcCqc+dhgqf1ZRHuvjUSPujZjTLOsuYYe github.com/openziti/channel/v2 v2.0.99/go.mod h1:k+pa5xOJEUl/kMw4K3Lj9FAjGUkU34RFm7lX1AnmJ8Q= github.com/openziti/dilithium v0.3.3 h1:PLgQ6PMNLSTzCFbX/h98cmudgz/cU6TmjdSv5NAPD8k= github.com/openziti/dilithium v0.3.3/go.mod h1:vsCjI2AU/hon9e+dLhUFbCNGesJDj2ASgkySOcpmvjo= -github.com/openziti/edge-api v0.25.36 h1:CjQTmXyYWjUX1Yc1FO0xkeATwd9A1KPu6CvNpstNMrY= -github.com/openziti/edge-api v0.25.36/go.mod h1:OMedQFZPoBER7E9nkyECML/Pt2C2uajdbm/BBP1wOuY= +github.com/openziti/edge-api v0.25.37 h1:VfxJJUPTWGTyfSSfIlthvp6yrb6qrBWHpZWhThoctwc= +github.com/openziti/edge-api v0.25.37/go.mod h1:OMedQFZPoBER7E9nkyECML/Pt2C2uajdbm/BBP1wOuY= github.com/openziti/fabric v0.24.36 h1:KAlMvFjeCeTfuZDKbH55M03eKpIiv/Zx4Tt+qdRydDE= github.com/openziti/fabric v0.24.36/go.mod h1:ZglDYZABDfl0rWI4eXo4AYMc9G18XSlDNombiGNzs7Q= github.com/openziti/foundation/v2 v2.0.32 h1:I4CS7UCMjxWrS2JqzQvPfgalJM9Z9PKi3wtn0b7ULmM= @@ -724,10 +724,10 @@ github.com/openziti/jwks v1.0.3 h1:hf8wkb+Cg4nH/HM0KROFd7u+C3DkRVcFZJ7tDV+4icc= github.com/openziti/jwks v1.0.3/go.mod h1:t4xxq8vlXGsPn29kiQVnZBBDDnEoOFqtJoHibkJunQQ= github.com/openziti/metrics v1.2.35 h1:pj1Ch+fzkgeOg/ukcCtFPGM7nL8TNqP53CWsY4pyPGU= github.com/openziti/metrics v1.2.35/go.mod h1:xOksS37b8sUxA3PJYWTnSoTC9udGbK38Tiq1GQp0GBc= -github.com/openziti/runzmd v1.0.31 h1:mXDBCv3jtYtoSOnvShsVmUtw+EPyAlcjI2ep7/qdtZs= -github.com/openziti/runzmd v1.0.31/go.mod h1:EKlsBLIBycT/IkUYacWn7JaFUGb6esq0EgiaMeXKmvE= -github.com/openziti/sdk-golang v0.20.112 h1:oc6fNVg9poyRMdPQK2L84RGvOk3fKs3X+dzY6kN9w7k= -github.com/openziti/sdk-golang v0.20.112/go.mod h1:/ioQ2qRfkwpSse3NCu37eXyJvQLgcK2RdenH4ajmhPw= +github.com/openziti/runzmd v1.0.32 h1:Ft3P8Gk9tSnvvIieeUd+4L5QXy1zoIZVqt3qYx0uLEQ= +github.com/openziti/runzmd v1.0.32/go.mod h1:MR8FdjwQde+OSdByg5e2sA6n8yWrVXu6PwzdzVfLiBA= +github.com/openziti/sdk-golang v0.20.116 h1:qHH5DgnqoKiciJqHis4QAo0e7vn5OZOXZd17oF2JCgE= +github.com/openziti/sdk-golang v0.20.116/go.mod h1:ZJDJFhmiBk223r+9F8ziN2xDDDfkbUSN4rBWC7qiAi4= github.com/openziti/secretstream v0.1.12 h1:N78CHxtqWzSyNFOsYtYRWNNTfX1ZDAPkFgzHobpodZU= github.com/openziti/secretstream v0.1.12/go.mod h1:gHMH1REH0r4VlmCtuWx8biU7j5ZfOivFjz9mLgwq7mk= github.com/openziti/storage v0.2.18 h1:a5G9bYJ+nMNU00WvDlWyhMlrlhPLsnxPkyStHKE6KMY= @@ -998,13 +998,13 @@ go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk= go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E= -go.opentelemetry.io/otel v1.18.0 h1:TgVozPGZ01nHyDZxK5WGPFB9QexeTMXEH7+tIClWfzs= -go.opentelemetry.io/otel v1.18.0/go.mod h1:9lWqYO0Db579XzVuCKFNPDl4s73Voa+zEck3wHaAYQI= -go.opentelemetry.io/otel/metric v1.18.0 h1:JwVzw94UYmbx3ej++CwLUQZxEODDj/pOuTCvzhtRrSQ= -go.opentelemetry.io/otel/metric v1.18.0/go.mod h1:nNSpsVDjWGfb7chbRLUNW+PBNdcSTHD4Uu5pfFMOI0k= +go.opentelemetry.io/otel v1.19.0 h1:MuS/TNf4/j4IXsZuJegVzI1cwut7Qc00344rgH7p8bs= +go.opentelemetry.io/otel v1.19.0/go.mod h1:i0QyjOq3UPoTzff0PJB2N66fb4S0+rSbSB15/oyH9fY= +go.opentelemetry.io/otel/metric v1.19.0 h1:aTzpGtV0ar9wlV4Sna9sdJyII5jTVJEvKETPiOKwvpE= +go.opentelemetry.io/otel/metric v1.19.0/go.mod h1:L5rUsV9kM1IxCj1MmSdS+JQAcVm319EUrDVLrt7jqt8= go.opentelemetry.io/otel/sdk v1.14.0 h1:PDCppFRDq8A1jL9v6KMI6dYesaq+DFcDZvjsoGvxGzY= -go.opentelemetry.io/otel/trace v1.18.0 h1:NY+czwbHbmndxojTEKiSMHkG2ClNH2PwmcHrdo0JY10= -go.opentelemetry.io/otel/trace v1.18.0/go.mod h1:T2+SGJGuYZY3bjj5rgh/hN7KIrlpWC5nS8Mjvzckz+0= +go.opentelemetry.io/otel/trace v1.19.0 h1:DFVQmlVbfVeOuBRrwdtaehRrWiL1JoVs9CPIQ1Dzxpg= +go.opentelemetry.io/otel/trace v1.19.0/go.mod h1:mfaSyvGyEJEI0nyV2I4qhNQnbBOUUmYZpYojqMnX2vo= go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= diff --git a/router/xgress_edge/hosted.go b/router/xgress_edge/hosted.go index 8fa1dd395..26187677d 100644 --- a/router/xgress_edge/hosted.go +++ b/router/xgress_edge/hosted.go @@ -62,6 +62,23 @@ func (registry *hostedServiceRegistry) cleanupServices(proxy *edgeClientConn) { }) } +func (registry *hostedServiceRegistry) cleanupDuplicates(newest *edgeTerminator) { + registry.services.Range(func(key, value interface{}) bool { + terminator := value.(*edgeTerminator) + if terminator != newest && newest.token == terminator.token && newest.instance == terminator.instance { + terminator.close(true, "duplicate terminator") // don't notify, channel is already closed, we can't send messages + registry.services.Delete(key) + pfxlog.Logger().WithField("routerId", terminator.edgeClientConn.listener.id.Token). + WithField("sessionToken", terminator.token). + WithField("instance", terminator.instance). + WithField("terminatorId", terminator.terminatorId.Load()). + WithField("duplicateOf", newest.terminatorId.Load()). + Info("duplicate removed") + } + return true + }) +} + func (registry *hostedServiceRegistry) unbindSession(sessionToken string, proxy *edgeClientConn) bool { atLeastOneRemoved := false registry.services.Range(func(key, value interface{}) bool { diff --git a/router/xgress_edge/listener.go b/router/xgress_edge/listener.go index 0ead94308..67b0d6e2d 100644 --- a/router/xgress_edge/listener.go +++ b/router/xgress_edge/listener.go @@ -19,7 +19,6 @@ package xgress_edge import ( "encoding/binary" "fmt" - "github.com/dineshappavoo/basex" "github.com/openziti/fabric/common/capabilities" fabricMetrics "github.com/openziti/fabric/common/metrics" "github.com/openziti/fabric/common/pb/ctrl_pb" @@ -27,7 +26,6 @@ import ( "github.com/openziti/ziti/common/cert" "github.com/openziti/ziti/common/pb/edge_ctrl_pb" "github.com/pkg/errors" - "golang.org/x/crypto/sha3" "google.golang.org/protobuf/proto" "math/big" "time" @@ -314,23 +312,6 @@ func (self *edgeClientConn) processBindV1(req *channel.Message, ch channel.Chann log.Info("created terminator") } -func (self *edgeClientConn) getTerminatorId(sessionToken, terminatorInstance string) string { - hasher := sha3.NewShake256() - // Shake.Write is documented to only fail if written to after being read - _, _ = hasher.Write([]byte(self.listener.id.Token)) - _, _ = hasher.Write([]byte(sessionToken)) - _, _ = hasher.Write([]byte(terminatorInstance)) - - buf := make([]byte, 16) - // Shake.Read is documented not to fail - _, _ = hasher.Read(buf) - - v := &big.Int{} - v.SetBytes(buf) - result, _ := basex.EncodeInt(v) - return result -} - func (self *edgeClientConn) processBindV2(req *channel.Message, ch channel.Channel, ctrlCh channel.Channel) { token := string(req.Body) @@ -350,9 +331,9 @@ func (self *edgeClientConn) processBindV2(req *channel.Message, ch channel.Chann assignIds, _ := req.GetBoolHeader(edge.RouterProvidedConnId) log.Debugf("client requested router provided connection ids: %v", assignIds) - terminatorId := self.getTerminatorId(token, terminatorInstance) + terminatorId := idgen.NewUUIDString() - messageSink := &edgeTerminator{ + terminator := &edgeTerminator{ MsgChannel: *edge.NewEdgeMsgChannel(self.ch, connId), edgeClientConn: self, token: token, @@ -361,30 +342,8 @@ func (self *edgeClientConn) processBindV2(req *channel.Message, ch channel.Chann v2: true, } - log = log.WithField("bindConnId", messageSink.MsgChannel.Id()) - - createTerminatorOnCtrl := true - - terminator, found := self.listener.factory.hostedServices.Get(terminatorId) - // we unfortunately can't re-use existing edgeTerminators, as a new bind request will have a new connId. - if found { - // if we somehow got a conflict, just use a new terminator id - if terminator.token != token || terminator.instance != terminatorInstance { - terminatorId = idgen.NewUUIDString() - } else if terminator.terminatorId.CompareAndSwap(terminatorId, "") { - // there was an existing terminator, and we got to it before it was torn down. We can just replace it - // and carry on - self.listener.factory.hostedServices.Put(terminatorId, messageSink) - createTerminatorOnCtrl = false - log.WithField("terminatorId", terminatorId). - WithField("oldBindConnId", terminator.MsgChannel.Id()). - Info("duplicate create terminator request detected, taking over terminator") - } else { - terminatorId = idgen.NewUUIDString() - } - } - - messageSink.terminatorId.Store(terminatorId) + log = log.WithField("bindConnId", terminator.MsgChannel.Id()) + terminator.terminatorId.Store(terminatorId) log = log.WithField("terminatorId", terminatorId) log.Debug("binding service") @@ -413,11 +372,11 @@ func (self *edgeClientConn) processBindV2(req *channel.Message, ch channel.Chann log.Debug("establishing listener") // need to remove session remove listener on close - messageSink.onClose = self.listener.factory.stateManager.AddEdgeSessionRemovedListener(token, func(token string) { - messageSink.close(true, "session ended") + terminator.onClose = self.listener.factory.stateManager.AddEdgeSessionRemovedListener(token, func(token string) { + terminator.close(true, "session ended") }) - self.listener.factory.hostedServices.Put(terminatorId, messageSink) + self.listener.factory.hostedServices.Put(terminatorId, terminator) var terminatorInstanceSecret []byte if terminatorInstance != "" { @@ -435,26 +394,24 @@ func (self *edgeClientConn) processBindV2(req *channel.Message, ch channel.Chann InstanceSecret: terminatorInstanceSecret, } - if createTerminatorOnCtrl { - timeout := self.listener.factory.ctrls.DefaultRequestTimeout() - responseMsg, err := protobufs.MarshalTyped(request).WithTimeout(timeout).SendForReply(ctrlCh) - resp := &edge_ctrl_pb.CreateTerminatorV2Response{} - err = protobufs.TypedResponse(resp).Unmarshall(responseMsg, err) - if err == nil && resp.Result != edge_ctrl_pb.CreateTerminatorResult_Success { - err = errors.Errorf("terminator create failed: %s", resp.Msg) - } - - if err != nil { - log.WithError(err).Warn("error creating terminator") - messageSink.close(false, "") // don't notify here, as we're notifying next line with a response - self.sendStateClosedReply(err.Error(), req) - return - } + timeout := self.listener.factory.ctrls.DefaultRequestTimeout() + responseMsg, err := protobufs.MarshalTyped(request).WithTimeout(timeout).SendForReply(ctrlCh) + resp := &edge_ctrl_pb.CreateTerminatorV2Response{} + err = protobufs.TypedResponse(resp).Unmarshall(responseMsg, err) + if err == nil && resp.Result != edge_ctrl_pb.CreateTerminatorResult_Success { + err = errors.Errorf("terminator create failed: %s", resp.Msg) } - if messageSink.MsgChannel.IsClosed() { + if err != nil { + log.WithError(err).Warn("error creating terminator") + terminator.close(false, "") // don't notify here, as we're notifying next line with a response + self.sendStateClosedReply(err.Error(), req) + return + } + + if terminator.MsgChannel.IsClosed() { log.Warn("edge channel closed while setting up terminator. cleaning up terminator now") - messageSink.close(false, "edge channel closed") + terminator.close(false, "edge channel closed") return } @@ -462,6 +419,8 @@ func (self *edgeClientConn) processBindV2(req *channel.Message, ch channel.Chann log.Debug("returning connection state CONNECTED to client") self.sendStateConnectedReply(req, nil) + self.listener.factory.hostedServices.cleanupDuplicates(terminator) + log.Info("created terminator") } diff --git a/tests/sdk_event_test.go b/tests/sdk_event_test.go index 2cf413456..899cf5f9a 100644 --- a/tests/sdk_event_test.go +++ b/tests/sdk_event_test.go @@ -205,7 +205,7 @@ func Test_SDK_Events(t *testing.T) { defer removeUnauthedListener() implZtx := ztxPostMfa.(*ziti.ContextImpl) - apiSessionId := *implZtx.CtrlClt.CurrentAPISessionDetail.ID + apiSessionId := *implZtx.CtrlClt.CurrentAPISessionDetail.Load().ID deleteParams := api_session.NewDeleteAPISessionsParams() deleteParams.ID = apiSessionId diff --git a/zititest/go.mod b/zititest/go.mod index fb9606663..beba1a96d 100644 --- a/zititest/go.mod +++ b/zititest/go.mod @@ -16,7 +16,7 @@ require ( github.com/openziti/fabric v0.24.36 github.com/openziti/foundation/v2 v2.0.32 github.com/openziti/identity v1.0.63 - github.com/openziti/sdk-golang v0.20.112 + github.com/openziti/sdk-golang v0.20.116 github.com/openziti/storage v0.2.18 github.com/openziti/transport/v2 v2.0.107 github.com/openziti/ziti v0.28.3 @@ -78,7 +78,7 @@ require ( github.com/go-resty/resty/v2 v2.8.0 // indirect github.com/golang-jwt/jwt/v5 v5.0.0 // indirect github.com/golang/protobuf v1.5.3 // indirect - github.com/gomarkdown/markdown v0.0.0-20230916125811-7478c230c7cd // indirect + github.com/gomarkdown/markdown v0.0.0-20230922112808-5421fefb8386 // indirect github.com/gorilla/handlers v1.5.1 // indirect github.com/gorilla/mux v1.8.0 // indirect github.com/gorilla/schema v1.2.0 // indirect @@ -129,10 +129,10 @@ require ( github.com/oliveagle/jsonpath v0.0.0-20180606110733-2e52cf6e6852 // indirect github.com/opentracing/opentracing-go v1.2.0 // indirect github.com/openziti/dilithium v0.3.3 // indirect - github.com/openziti/edge-api v0.25.36 // indirect + github.com/openziti/edge-api v0.25.37 // indirect github.com/openziti/jwks v1.0.3 // indirect github.com/openziti/metrics v1.2.35 // indirect - github.com/openziti/runzmd v1.0.31 // indirect + github.com/openziti/runzmd v1.0.32 // indirect github.com/openziti/secretstream v0.1.12 // indirect github.com/openziti/x509-claims v1.0.3 // indirect github.com/openziti/xweb/v2 v2.1.0 // indirect @@ -171,9 +171,9 @@ require ( github.com/zitadel/oidc/v2 v2.7.0 // indirect go.mongodb.org/mongo-driver v1.12.1 // indirect go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352 // indirect - go.opentelemetry.io/otel v1.18.0 // indirect - go.opentelemetry.io/otel/metric v1.18.0 // indirect - go.opentelemetry.io/otel/trace v1.18.0 // indirect + go.opentelemetry.io/otel v1.19.0 // indirect + go.opentelemetry.io/otel/metric v1.19.0 // indirect + go.opentelemetry.io/otel/trace v1.19.0 // indirect golang.org/x/crypto v0.13.0 // indirect golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect golang.org/x/image v0.12.0 // indirect diff --git a/zititest/go.sum b/zititest/go.sum index 78fdee374..26f346835 100644 --- a/zititest/go.sum +++ b/zititest/go.sum @@ -381,8 +381,8 @@ github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8l github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/golangci/lint-1 v0.0.0-20181222135242-d2cdd8c08219/go.mod h1:/X8TswGSh1pIozq4ZwCfxS0WA5JGXguxk94ar/4c87Y= github.com/gomarkdown/markdown v0.0.0-20191123064959-2c17d62f5098/go.mod h1:aii0r/K0ZnHv7G0KF7xy1v0A7s2Ljrb5byB7MO5p6TU= -github.com/gomarkdown/markdown v0.0.0-20230916125811-7478c230c7cd h1:laCEzrtkKEkT2424vMTGl6N1m0xN8kq371hksD5Be+8= -github.com/gomarkdown/markdown v0.0.0-20230916125811-7478c230c7cd/go.mod h1:JDGcbDT52eL4fju3sZ4TeHGsQwhG9nbDV21aMyhwPoA= +github.com/gomarkdown/markdown v0.0.0-20230922112808-5421fefb8386 h1:EcQR3gusLHN46TAD+G+EbaaqJArt5vHhNpXAa12PQf4= +github.com/gomarkdown/markdown v0.0.0-20230922112808-5421fefb8386/go.mod h1:JDGcbDT52eL4fju3sZ4TeHGsQwhG9nbDV21aMyhwPoA= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= @@ -722,8 +722,8 @@ github.com/openziti/channel/v2 v2.0.99 h1:/Ji5fxcCqc+dhgqf1ZRHuvjUSPujZjTLOsuYYe github.com/openziti/channel/v2 v2.0.99/go.mod h1:k+pa5xOJEUl/kMw4K3Lj9FAjGUkU34RFm7lX1AnmJ8Q= github.com/openziti/dilithium v0.3.3 h1:PLgQ6PMNLSTzCFbX/h98cmudgz/cU6TmjdSv5NAPD8k= github.com/openziti/dilithium v0.3.3/go.mod h1:vsCjI2AU/hon9e+dLhUFbCNGesJDj2ASgkySOcpmvjo= -github.com/openziti/edge-api v0.25.36 h1:CjQTmXyYWjUX1Yc1FO0xkeATwd9A1KPu6CvNpstNMrY= -github.com/openziti/edge-api v0.25.36/go.mod h1:OMedQFZPoBER7E9nkyECML/Pt2C2uajdbm/BBP1wOuY= +github.com/openziti/edge-api v0.25.37 h1:VfxJJUPTWGTyfSSfIlthvp6yrb6qrBWHpZWhThoctwc= +github.com/openziti/edge-api v0.25.37/go.mod h1:OMedQFZPoBER7E9nkyECML/Pt2C2uajdbm/BBP1wOuY= github.com/openziti/fablab v0.5.16 h1:+tqqDR5Sl4qLT3fSC+Dv1AGZuOeeX87h9LNKa1KJS5Y= github.com/openziti/fablab v0.5.16/go.mod h1:KiU/NxThNtH9U803Ep0WcSrbov3moZso2oRgIpQq0S0= github.com/openziti/fabric v0.24.36 h1:KAlMvFjeCeTfuZDKbH55M03eKpIiv/Zx4Tt+qdRydDE= @@ -736,10 +736,10 @@ github.com/openziti/jwks v1.0.3 h1:hf8wkb+Cg4nH/HM0KROFd7u+C3DkRVcFZJ7tDV+4icc= github.com/openziti/jwks v1.0.3/go.mod h1:t4xxq8vlXGsPn29kiQVnZBBDDnEoOFqtJoHibkJunQQ= github.com/openziti/metrics v1.2.35 h1:pj1Ch+fzkgeOg/ukcCtFPGM7nL8TNqP53CWsY4pyPGU= github.com/openziti/metrics v1.2.35/go.mod h1:xOksS37b8sUxA3PJYWTnSoTC9udGbK38Tiq1GQp0GBc= -github.com/openziti/runzmd v1.0.31 h1:mXDBCv3jtYtoSOnvShsVmUtw+EPyAlcjI2ep7/qdtZs= -github.com/openziti/runzmd v1.0.31/go.mod h1:EKlsBLIBycT/IkUYacWn7JaFUGb6esq0EgiaMeXKmvE= -github.com/openziti/sdk-golang v0.20.112 h1:oc6fNVg9poyRMdPQK2L84RGvOk3fKs3X+dzY6kN9w7k= -github.com/openziti/sdk-golang v0.20.112/go.mod h1:/ioQ2qRfkwpSse3NCu37eXyJvQLgcK2RdenH4ajmhPw= +github.com/openziti/runzmd v1.0.32 h1:Ft3P8Gk9tSnvvIieeUd+4L5QXy1zoIZVqt3qYx0uLEQ= +github.com/openziti/runzmd v1.0.32/go.mod h1:MR8FdjwQde+OSdByg5e2sA6n8yWrVXu6PwzdzVfLiBA= +github.com/openziti/sdk-golang v0.20.116 h1:qHH5DgnqoKiciJqHis4QAo0e7vn5OZOXZd17oF2JCgE= +github.com/openziti/sdk-golang v0.20.116/go.mod h1:ZJDJFhmiBk223r+9F8ziN2xDDDfkbUSN4rBWC7qiAi4= github.com/openziti/secretstream v0.1.12 h1:N78CHxtqWzSyNFOsYtYRWNNTfX1ZDAPkFgzHobpodZU= github.com/openziti/secretstream v0.1.12/go.mod h1:gHMH1REH0r4VlmCtuWx8biU7j5ZfOivFjz9mLgwq7mk= github.com/openziti/storage v0.2.18 h1:a5G9bYJ+nMNU00WvDlWyhMlrlhPLsnxPkyStHKE6KMY= @@ -1012,13 +1012,13 @@ go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk= go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E= -go.opentelemetry.io/otel v1.18.0 h1:TgVozPGZ01nHyDZxK5WGPFB9QexeTMXEH7+tIClWfzs= -go.opentelemetry.io/otel v1.18.0/go.mod h1:9lWqYO0Db579XzVuCKFNPDl4s73Voa+zEck3wHaAYQI= -go.opentelemetry.io/otel/metric v1.18.0 h1:JwVzw94UYmbx3ej++CwLUQZxEODDj/pOuTCvzhtRrSQ= -go.opentelemetry.io/otel/metric v1.18.0/go.mod h1:nNSpsVDjWGfb7chbRLUNW+PBNdcSTHD4Uu5pfFMOI0k= +go.opentelemetry.io/otel v1.19.0 h1:MuS/TNf4/j4IXsZuJegVzI1cwut7Qc00344rgH7p8bs= +go.opentelemetry.io/otel v1.19.0/go.mod h1:i0QyjOq3UPoTzff0PJB2N66fb4S0+rSbSB15/oyH9fY= +go.opentelemetry.io/otel/metric v1.19.0 h1:aTzpGtV0ar9wlV4Sna9sdJyII5jTVJEvKETPiOKwvpE= +go.opentelemetry.io/otel/metric v1.19.0/go.mod h1:L5rUsV9kM1IxCj1MmSdS+JQAcVm319EUrDVLrt7jqt8= go.opentelemetry.io/otel/sdk v1.14.0 h1:PDCppFRDq8A1jL9v6KMI6dYesaq+DFcDZvjsoGvxGzY= -go.opentelemetry.io/otel/trace v1.18.0 h1:NY+czwbHbmndxojTEKiSMHkG2ClNH2PwmcHrdo0JY10= -go.opentelemetry.io/otel/trace v1.18.0/go.mod h1:T2+SGJGuYZY3bjj5rgh/hN7KIrlpWC5nS8Mjvzckz+0= +go.opentelemetry.io/otel/trace v1.19.0 h1:DFVQmlVbfVeOuBRrwdtaehRrWiL1JoVs9CPIQ1Dzxpg= +go.opentelemetry.io/otel/trace v1.19.0/go.mod h1:mfaSyvGyEJEI0nyV2I4qhNQnbBOUUmYZpYojqMnX2vo= go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=