[DISCUSSION] New Quorum vote pluggable implementation

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Re: [DISCUSSION] New Quorum vote pluggable implementation

michael.andre.pearce
I was more referring to supporting the plugin.Concerns raised were:Other systems client dependencies e.g. kafka client lib or Prometheus lib in core broker, getting out of dateExposure of cves in core broker that every plugin with its dependencies could bring.Plugin code becoming un-maintained e.g. the older vertex ones that were removed. People seemed to be against having plugins being in broker and then lapsing in maintenance.All I care for is we have a solution that we agree on and stop having to revisit this, and its a simply process to contribute new ones. I disagree strongly that it should be case by case, we should have an agreed place and process.Both the above can be sorted by either being simply accepted risk, or by being separately maintained in a seperate plugin repo.My personal is take the risk as will be also easier to deliver.But I understand the concerns of others, it might be easier to go the new git repo for plugins (i see this alot in other projects)Sent from my Samsung Galaxy smartphone.
-------- Original message --------From: Clebert Suconic <[hidden email]> Date: 18/03/2020  01:25  (GMT+00:00) To: [hidden email] Subject: Re: Re: [DISCUSSION] New Quorum vote pluggable implementation >> no longer no one supports it  <<I think this is just apache license here.As long as you have community around it.. code is maintained.. it'sall fine. I mean... in apache terms you guarantee things as in apachelicense. I try to do good work on the open source community, fixingbugs.. everything... but as you know the limits of the apache licenseterms.The term support here I think applies to third parties.. companieslisted here, (Including the company I work for... Red Hat / IBM).:https://activemq.apache.org/support.htmlIt will be up to them what they consider supported or not.. and thatgoes both for plugins or core functionalities. Company X may decide tonot support certain functionality as in providing you patches, help..etc. and you would be on your own on such functionality like any otherapache license.that's pretty standard... not just for ActiveMQ.. I think that'scommon for any other project.On Tue, Mar 17, 2020 at 9:05 PM michael.andre.pearce<[hidden email]> wrote:>> I totally get it.As said im for the camp that we have plugin modules in the broker. And if someone wants to commit one, we allow it without need to votes etc and all this discussion every time.(e.g. Justin and I shouldnt need to make discussion thread, it should just be accepted and added)If something gets un-maintained (some peoples concerns) we simply remove it / attic it. The fact its un-maintained means no longer no one supports it But its for everyone to agree to this more open way of working.....not just me and you.An alternative approach if people are concerned these are in broker is that we have a separate plugin repo that we stick them all, and release...but this then makes it harder for end users to enable or use...Sent from my Samsung Galaxy smartphone.> -------- Original message --------From: Clebert Suconic <[hidden email]> Date: 18/03/2020  00:57  (GMT+00:00) To: [hidden email] Subject: Re: Re: [DISCUSSION] New Quorum vote pluggable implementation Just like anything else in apache... committers/community can ask orvote features out. it doesn't even need to be a plugin.I don't mean to hijack this discussion away from Quorum.. although itwould affect the implementation.. .we can keep a plugin's area in thebroker... and activate them based on the CLI / config.I just want to make sure we can unlock this going forward.On Tue, Mar 17, 2020 at 8:53 PM michael.andre.pearce<[hidden email]> wrote:>> Well the point is without us having sorted the issue, then this will keep occuring.I am for one in favour we relax a little and if someone wants to add a plugin we simply allow it. Why should there be any filter.Sent from my Samsung Galaxy smartphone.> -------- Original message --------From: Clebert Suconic <[hidden email]> Date: 18/03/2020  00:38  (GMT+00:00) To: [hidden email] Subject: Re: Re: [DISCUSSION] New Quorum vote pluggable implementation It's on a case by case basis... What we can't do is transform everynew core functionality (such as monitoring. .it's pretty standard /core functionality) into a plug-in, and then block its implementationbased on a past discussion.I say, if there's a plugin, there should be at least oneimplementation on the broker.. that goes for Monitoring... we cansupport more.. but this has to have an implementation.You are a committer, if you want to have your kafka plugin again, justraise a discussion.. but we can't block every new functionality basedon that baggage.On Tue, Mar 17, 2020 at 7:35 PM michael.andre.pearce<[hidden email]> wrote:>> I think the issue with plugins is that whats the rule to include one or not.You know exactly where im coming from there... re kafka plugin.Im totally for a number of plugins being added. Including the kafka one.But we have to have a clear rule of what will be allowed or not and a place to maintain.Sent from my Samsung Galaxy smartphone.> -------- Original message --------From: Clebert Suconic <[hidden email]> Date: 16/03/2020  01:20  (GMT+00:00) To: [hidden email] Subject: Re: Re: [DISCUSSION] New Quorum vote pluggable implementation We can make pluggable as much as you want, but going forward I wouldlike us to change how we have been treating plugins.Because of other discussions, plugins are not really implemented aspart of the community.We should elect one implementation we can make it and implement itwell.... and bring it as part of the community. People can add moreimplementations as part of the open source project.This is probably the same with Micrometer. Justin implementedMicrometer and the implementation is not part of activemq. I think weshould change that going forward.This will be a 3.0, so we should definitely improve how we do pluginsgoing forward.On Fri, Mar 13, 2020 at 3:42 AM michael.andre.pearce<[hidden email]> wrote:>> I think a main thing here is a pluggable api.If someone already has etcd, zookeeper, consul etc. In their infrastructure they should be able to reuse.Like wise if you have multiple bulkheaded broker sets it be useful to be able to reuse the same quorum compontent e.g. in ZK that would be a different base root on the tree path per broker set.Likewise that said we should have some out the box support plugins from the community. Obviously what ever we do whilst it maybe a break change e.g. a 3.0 release there must be a migration solution (accepted that it may need outage) for existing brokers that works via the pluggable api not to a specific solution.Also anything from a management api pov should also be visible in the web management console. Sent from my Samsung Galaxy smartphone.> -------- Original message --------From: Clebert Suconic <[hidden email]> Date: 09/03/2020  14:55  (GMT+00:00) To: [hidden email] Subject: Re: Re: [DISCUSSION] New Quorum vote pluggable implementation I think we should have a management component, that runs outside ofthe broker and would manage quorum.That way you can have the quorum running outside of the broker itself.which would improve the need of multiple brokers to manage the quorum.You just need Quorum Managers in distinct places.I had recently worked with a software called Ceph... And Ceph has theconcept of managers working away from their "broker" (it's not abroker.. it's a DB, but in a sense it's the same concept here). Ithink we should do the same.I have talked to Franz and other guys in private.. and it seemseverybody these days mention raft consensus algorithm.  Perhaps weshould look into it.. and make it pluggable. I believe there's JRaftworking on top of JGroups already.If we make this pluggable and make it a manager a separate process,this will be a big win IMO.On Tue, Mar 3, 2020 at 9:55 AM Andy Taylor <[hidden email]> wrote:>> Personally I wouldn't use Zookeeper, I think there are better options. Also> looks like Kafka are replacing it as well. Saying that, it doesn't really> matter what is used, the main thing is we need to remove the burden of> providing consensus away from the broker.>> It would make sense to make it pluggable.>> A>> On Mon, 2 Mar 2020 at 19:19, KimmKing <[hidden email]> wrote:>> > Can't agree any more.> > The HA&Replication is more and more important for a modern messaging> > system.> > Other apache opensource mq, kafka/rocketmq/pulsar maybe referred to.> >> >> > And In my experience this also bring some extra-complexity about> > performance/brainsplitting issues when rebalance data.> >> > At 2020-03-02 20:33:31, "Martyn Taylor" <[hidden email]> wrote:> > >I think this is a great idea Franz.  The HA and replication components> > have> > >been a source of issues over the years.  Two problems I see are that 1)> > >there isn't a clean separation between the consensus mechanism and the> > >replication, and 2) the consensus algorithm used in Artemis isn't based on> > >any standard algorithm or a research paper.  Hence, all the issues that> > >were caught over the years due to various edge cases.  Integration with> > >ZooKeeper seems like the obvious solution (i.e. push the consensus logic> > >off to a third party lib).  I suspect though, this will be a considerable> > >amount of work and is likely to introduce new issues, so I'd proceed with> > >caution.> > >> > >Cheers> > >> > >> > >> > >On Mon, Mar 2, 2020 at 8:28 AM nigro_franz <[hidden email]> wrote:> > >> > >> Hi folks,> > >>> > >> especially due to the requirements of the current Artemis quorum vote> > >> algorithm, we've thought to re-implementing it with a different focus in> > >> mind:> > >> 1) to make it pluggable (eg by using the many Raft implementations,> > >> ZooKeeper or others)> > >> 2) to cleanly separate the election phase and cluster member states (ie> > it> > >> should be the Topology shared between them)> > >> 3) to simplify most common setups in both amount of configuration and> > >> requirements (eg "witness" nodes could be implemented to get a minimum> > 2*n> > >> +1 quorum of nodes instead of forcing 2*n + 1 master-backup pairs)> > >> 4) [OPTIONALLY] to reduce/eliminate implicit "good practices" in term of> > >> order of actions to be performed on nodes in "special states" eg proper> > >> restart sequence after failover or similar cases> > >> 5) [OPTIONALLY] to make shared-store and replication behaviour more> > >> similar:> > >> journal's presence should be the only difference between the 2s> > >>> > >> A proposal of steps to be followed to get this:> > >> 1) abstract away the current quorum vote: it requires extra-care because> > >> the> > >> logic is melted together with the replication/clustering behaviour> > >> 2) refactor it in order to separate election phase and cluster member> > >> states> > >> 3) implement a RI version of such APIs> > >>> > >> Post-actions to help people adopt it, but need to be thought upfront:> > >> 1) a clean upgrade path for current HA replication users> > >> 2) deprecate or integrate the current HA replication into the new> > version> > >>> > >> I've opened this here because many of the HA replication users are devs> > too> > >> and given that this isn't yet implemented: we're stull in the> > >> design/proposal phase, so anyone that want to express their> > >> ideas/opinions/POC on this, is invited to talk here ;)> > >>> > >> Cheers,> > >> Franz> > >>> > >>> > >>> > >>> > >>> > >> --> > >> Sent from:> > >> http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html> > >>> >-- Clebert Suconic-- Clebert Suconic-- Clebert Suconic-- Clebert Suconic-- Clebert Suconic
Reply | Threaded
Open this post in threaded view
|

Re: Re: [DISCUSSION] New Quorum vote pluggable implementation

nigro_franz
This post was updated on .
Thanks for all the feedbacks: I see this is an hot topic for sure :)

In the next week I will start putting some idea in a shared doc and open a
discussion on slack to collect some ideas & soft/hard requirements in order
to decide a roadmap: ATM I'm very conflicted how much we should leverage on
some of the mechanisms/abstractions we currently have (Activation(s), Topology
propagation etc etc) abstracting it away into a more generic API or just
designing a new thing from scratch and then trying to make the current
behaviour to fit into it to help migration or to let existing users to not
be forced to adopt the new one until it wil meet their needs.

Regardless which direction to take one of my "best to have" is the
convergence between shared-store and replication ha into a sigle solution
with 2 "flavours": for what HA has been until now I see key to it the concept of
keeping integrity and ownership of the journal (considering the fact
that with Ceph/GlusterFS the sync replication is provided at file-system
level and with the current "software replication" is up to the broker to
perform it).
It looks to me that things like FileLockNodeManager or the JdbcNodeManager
are just 2 different building blocks that allow to detect a loss/acquire the journal ownership
and could be used to implement (the shared store flavour of) the new API to manage the
live-backup(s) states.

Just thinking loud; another thing that could be a nice thing to have
(looking at kafka at this, although they allow it with a finer granularity)
is the multiple-backups synchronization to increase availability of the
service: if anyone has specific concern or any idea why it shouldn't be a
good idea, feel free to share it.

Happy easter and enjoy the longer week-end



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
Reply | Threaded
Open this post in threaded view
|

Re: Re: [DISCUSSION] New Quorum vote pluggable implementation

michael.andre.pearce
I would be against us bringing in something that makes a nasty barrier for entry for existing users.Think about the current situation with classic and artemis where one of the biggest issues has been that a user cannot simply take his/her configuration and just update the broker to the new version, theres quite some minefield to migrate. And probably alot of the reason why still a large part of community is still on classic.If we ended up coming up with some system that starts from scratch and makes it that maybe not all is implemented we will basically be making that chasm worse.We should be making this as easy as possible for existing users / admins to just upgrade. User Community needs to come first here.Sent from my Samsung Galaxy smartphone.
-------- Original message --------From: nigro_franz <[hidden email]> Date: 11/04/2020  18:42  (GMT+00:00) To: [hidden email] Subject: Re: Re: [DISCUSSION] New Quorum vote pluggable implementation Thanks for all the feedbacks: I see this is an hot topic for sure :)In the next week I will start putting some idea in a shared doc and open adiscussion on slack to collect some ideas & soft/hard requirements in orderto decide a roadmap: ATM I'm very conflicted how much we should leverage onsome of the mechanisms what we currently have (Activation(s), Topologypropagation etc etc) abstracting it away into a more generic API or justdesigning a new thing from scratch and then trying to make the currentbehaviour to fit into it to help migration or to let existing users to notbe forced to adopt the new one until it wil meet their needs.Regardless which direction to take one of my "best to have" is theconvergence between shared-store and replication ha into a sigle solutionwith 2 "flavours": for what HA has been done until now is just matter ofkeep integrity and the right ownership of the journal (considering the factthat with Ceph/GlusterFS the sync replication is provided at file-systemlevel and with the current "software replication" is up to the broker toperform it).It looks to me that things like FileLockNodeManager or the JdbcNodeManagerare just 2 possible building blocks to provide the functionality that allowto detect a loss in the journal ownership and could be used as buildingblocks to implement (the shared store flavour of) the new API to manage thelive-backup(s) states.Just thinking loud; another thing that could be a nice thing to have(looking at kafka at this, although they allow it with a finer granularity)is the multiple-backups synchronization to increase availability of theservice: if anyone has specific concern or any idea why it shouldn't be agood idea, feel free to share it.Happy easter and enjoy the longer week-end--Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] New Quorum vote pluggable implementation

michaelpearce
In reply to this post by nigro_franz
(Try again using my other mail account so formatting of my mail doesnt muck
up)


I would be against us bringing in something that makes a nasty barrier for
entry for existing users.

Think about the current situation with classic and artemis where one of the
biggest issues has been that a user cannot simply take his/her
configuration and just update the broker to the new version, theres quite
some minefield to migrate. And probably alot of the reason why still a
large part of community is still on classic.

If we ended up coming up with some system that starts from scratch and
makes it that maybe not all is implemented we will basically be making that
chasm worse.


We should be making this as easy as possible for existing users / admins to
just upgrade.

User Community needs to come first here.

On Mon, 2 Mar 2020, 08:28 nigro_franz, <[hidden email]> wrote:

> Hi folks,
>
> especially due to the requirements of the current Artemis quorum vote
> algorithm, we've thought to re-implementing it with a different focus in
> mind:
> 1) to make it pluggable (eg by using the many Raft implementations,
> ZooKeeper or others)
> 2) to cleanly separate the election phase and cluster member states (ie it
> should be the Topology shared between them)
> 3) to simplify most common setups in both amount of configuration and
> requirements (eg "witness" nodes could be implemented to get a minimum 2*n
> +1 quorum of nodes instead of forcing 2*n + 1 master-backup pairs)
> 4) [OPTIONALLY] to reduce/eliminate implicit "good practices" in term of
> order of actions to be performed on nodes in "special states" eg proper
> restart sequence after failover or similar cases
> 5) [OPTIONALLY] to make shared-store and replication behaviour more
> similar:
> journal's presence should be the only difference between the 2s
>
> A proposal of steps to be followed to get this:
> 1) abstract away the current quorum vote: it requires extra-care because
> the
> logic is melted together with the replication/clustering behaviour
> 2) refactor it in order to separate election phase and cluster member
> states
> 3) implement a RI version of such APIs
>
> Post-actions to help people adopt it, but need to be thought upfront:
> 1) a clean upgrade path for current HA replication users
> 2) deprecate or integrate the current HA replication into the new version
>
> I've opened this here because many of the HA replication users are devs too
> and given that this isn't yet implemented: we're stull in the
> design/proposal phase, so anyone that want to express their
> ideas/opinions/POC on this, is invited to talk here ;)
>
> Cheers,
> Franz
>
>
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] New Quorum vote pluggable implementation

clebertsuconic
Sorry if I’m being naive.  I am trying to understand why your brought the
configuration and upgrade topic.

Is that a subject you are bringing now or you already see a problem ?

Quórum implementation is orthogonal to the config I think.    We have been
keeping any confirmation options compatible With previous versions so far.



On Sat, Apr 11, 2020 at 4:39 PM Michael Pearce <[hidden email]>
wrote:

> (Try again using my other mail account so formatting of my mail doesnt muck
> up)
>
>
> I would be against us bringing in something that makes a nasty barrier for
> entry for existing users.
>
> Think about the current situation with classic and artemis where one of the
> biggest issues has been that a user cannot simply take his/her
> configuration and just update the broker to the new version, theres quite
> some minefield to migrate. And probably alot of the reason why still a
> large part of community is still on classic.
>
> If we ended up coming up with some system that starts from scratch and
> makes it that maybe not all is implemented we will basically be making that
> chasm worse.
>
>
> We should be making this as easy as possible for existing users / admins to
> just upgrade.
>
> User Community needs to come first here.
>
> On Mon, 2 Mar 2020, 08:28 nigro_franz, <[hidden email]> wrote:
>
> > Hi folks,
> >
> > especially due to the requirements of the current Artemis quorum vote
> > algorithm, we've thought to re-implementing it with a different focus in
> > mind:
> > 1) to make it pluggable (eg by using the many Raft implementations,
> > ZooKeeper or others)
> > 2) to cleanly separate the election phase and cluster member states (ie
> it
> > should be the Topology shared between them)
> > 3) to simplify most common setups in both amount of configuration and
> > requirements (eg "witness" nodes could be implemented to get a minimum
> 2*n
> > +1 quorum of nodes instead of forcing 2*n + 1 master-backup pairs)
> > 4) [OPTIONALLY] to reduce/eliminate implicit "good practices" in term of
> > order of actions to be performed on nodes in "special states" eg proper
> > restart sequence after failover or similar cases
> > 5) [OPTIONALLY] to make shared-store and replication behaviour more
> > similar:
> > journal's presence should be the only difference between the 2s
> >
> > A proposal of steps to be followed to get this:
> > 1) abstract away the current quorum vote: it requires extra-care because
> > the
> > logic is melted together with the replication/clustering behaviour
> > 2) refactor it in order to separate election phase and cluster member
> > states
> > 3) implement a RI version of such APIs
> >
> > Post-actions to help people adopt it, but need to be thought upfront:
> > 1) a clean upgrade path for current HA replication users
> > 2) deprecate or integrate the current HA replication into the new version
> >
> > I've opened this here because many of the HA replication users are devs
> too
> > and given that this isn't yet implemented: we're stull in the
> > design/proposal phase, so anyone that want to express their
> > ideas/opinions/POC on this, is invited to talk here ;)
> >
> > Cheers,
> > Franz
> >
> >
> >
> >
> >
> > --
> > Sent from:
> > http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
> >
>
--
Clebert Suconic
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] New Quorum vote pluggable implementation

michaelpearce
It was in reference to Franzs last mail


"designing a new thing from scratch and then trying to make the current
behaviour to fit into it to help migration or to let existing users to not
be forced to adopt the new one until it wil meet their needs."


I think the migration should be seemless to current users. It should
support current behaviours.

If we dont we cause the same issue there is in migrating from classic to
Artemis. Just now in upgrading artemis.

On Sat, 11 Apr 2020, 22:35 Clebert Suconic, <[hidden email]>
wrote:

> Sorry if I’m being naive.  I am trying to understand why your brought the
> configuration and upgrade topic.
>
> Is that a subject you are bringing now or you already see a problem ?
>
> Quórum implementation is orthogonal to the config I think.    We have been
> keeping any confirmation options compatible With previous versions so far.
>
>
>
> On Sat, Apr 11, 2020 at 4:39 PM Michael Pearce <[hidden email]>
> wrote:
>
> > (Try again using my other mail account so formatting of my mail doesnt
> muck
> > up)
> >
> >
> > I would be against us bringing in something that makes a nasty barrier
> for
> > entry for existing users.
> >
> > Think about the current situation with classic and artemis where one of
> the
> > biggest issues has been that a user cannot simply take his/her
> > configuration and just update the broker to the new version, theres quite
> > some minefield to migrate. And probably alot of the reason why still a
> > large part of community is still on classic.
> >
> > If we ended up coming up with some system that starts from scratch and
> > makes it that maybe not all is implemented we will basically be making
> that
> > chasm worse.
> >
> >
> > We should be making this as easy as possible for existing users / admins
> to
> > just upgrade.
> >
> > User Community needs to come first here.
> >
> > On Mon, 2 Mar 2020, 08:28 nigro_franz, <[hidden email]> wrote:
> >
> > > Hi folks,
> > >
> > > especially due to the requirements of the current Artemis quorum vote
> > > algorithm, we've thought to re-implementing it with a different focus
> in
> > > mind:
> > > 1) to make it pluggable (eg by using the many Raft implementations,
> > > ZooKeeper or others)
> > > 2) to cleanly separate the election phase and cluster member states (ie
> > it
> > > should be the Topology shared between them)
> > > 3) to simplify most common setups in both amount of configuration and
> > > requirements (eg "witness" nodes could be implemented to get a minimum
> > 2*n
> > > +1 quorum of nodes instead of forcing 2*n + 1 master-backup pairs)
> > > 4) [OPTIONALLY] to reduce/eliminate implicit "good practices" in term
> of
> > > order of actions to be performed on nodes in "special states" eg proper
> > > restart sequence after failover or similar cases
> > > 5) [OPTIONALLY] to make shared-store and replication behaviour more
> > > similar:
> > > journal's presence should be the only difference between the 2s
> > >
> > > A proposal of steps to be followed to get this:
> > > 1) abstract away the current quorum vote: it requires extra-care
> because
> > > the
> > > logic is melted together with the replication/clustering behaviour
> > > 2) refactor it in order to separate election phase and cluster member
> > > states
> > > 3) implement a RI version of such APIs
> > >
> > > Post-actions to help people adopt it, but need to be thought upfront:
> > > 1) a clean upgrade path for current HA replication users
> > > 2) deprecate or integrate the current HA replication into the new
> version
> > >
> > > I've opened this here because many of the HA replication users are devs
> > too
> > > and given that this isn't yet implemented: we're stull in the
> > > design/proposal phase, so anyone that want to express their
> > > ideas/opinions/POC on this, is invited to talk here ;)
> > >
> > > Cheers,
> > > Franz
> > >
> > >
> > >
> > >
> > >
> > > --
> > > Sent from:
> > > http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
> > >
> >
> --
> Clebert Suconic
>
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] New Quorum vote pluggable implementation

clebertsuconic
I think configuration and implementation are orthogonal here.

With the new thing it should be possible to have a single replication
pair.  And having a new feature to control extra managers.

It’s a new feature I think.

But you made a valid point on the config.

On Sat, Apr 11, 2020 at 5:56 PM Michael Pearce <[hidden email]>
wrote:

> It was in reference to Franzs last mail
>
>
> "designing a new thing from scratch and then trying to make the current
> behaviour to fit into it to help migration or to let existing users to not
> be forced to adopt the new one until it wil meet their needs."
>
>
> I think the migration should be seemless to current users. It should
> support current behaviours.
>
> If we dont we cause the same issue there is in migrating from classic to
> Artemis. Just now in upgrading artemis.
>
> On Sat, 11 Apr 2020, 22:35 Clebert Suconic, <[hidden email]>
> wrote:
>
> > Sorry if I’m being naive.  I am trying to understand why your brought the
> > configuration and upgrade topic.
> >
> > Is that a subject you are bringing now or you already see a problem ?
> >
> > Quórum implementation is orthogonal to the config I think.    We have
> been
> > keeping any confirmation options compatible With previous versions so
> far.
> >
> >
> >
> > On Sat, Apr 11, 2020 at 4:39 PM Michael Pearce <[hidden email]
> >
> > wrote:
> >
> > > (Try again using my other mail account so formatting of my mail doesnt
> > muck
> > > up)
> > >
> > >
> > > I would be against us bringing in something that makes a nasty barrier
> > for
> > > entry for existing users.
> > >
> > > Think about the current situation with classic and artemis where one of
> > the
> > > biggest issues has been that a user cannot simply take his/her
> > > configuration and just update the broker to the new version, theres
> quite
> > > some minefield to migrate. And probably alot of the reason why still a
> > > large part of community is still on classic.
> > >
> > > If we ended up coming up with some system that starts from scratch and
> > > makes it that maybe not all is implemented we will basically be making
> > that
> > > chasm worse.
> > >
> > >
> > > We should be making this as easy as possible for existing users /
> admins
> > to
> > > just upgrade.
> > >
> > > User Community needs to come first here.
> > >
> > > On Mon, 2 Mar 2020, 08:28 nigro_franz, <[hidden email]> wrote:
> > >
> > > > Hi folks,
> > > >
> > > > especially due to the requirements of the current Artemis quorum vote
> > > > algorithm, we've thought to re-implementing it with a different focus
> > in
> > > > mind:
> > > > 1) to make it pluggable (eg by using the many Raft implementations,
> > > > ZooKeeper or others)
> > > > 2) to cleanly separate the election phase and cluster member states
> (ie
> > > it
> > > > should be the Topology shared between them)
> > > > 3) to simplify most common setups in both amount of configuration and
> > > > requirements (eg "witness" nodes could be implemented to get a
> minimum
> > > 2*n
> > > > +1 quorum of nodes instead of forcing 2*n + 1 master-backup pairs)
> > > > 4) [OPTIONALLY] to reduce/eliminate implicit "good practices" in term
> > of
> > > > order of actions to be performed on nodes in "special states" eg
> proper
> > > > restart sequence after failover or similar cases
> > > > 5) [OPTIONALLY] to make shared-store and replication behaviour more
> > > > similar:
> > > > journal's presence should be the only difference between the 2s
> > > >
> > > > A proposal of steps to be followed to get this:
> > > > 1) abstract away the current quorum vote: it requires extra-care
> > because
> > > > the
> > > > logic is melted together with the replication/clustering behaviour
> > > > 2) refactor it in order to separate election phase and cluster member
> > > > states
> > > > 3) implement a RI version of such APIs
> > > >
> > > > Post-actions to help people adopt it, but need to be thought upfront:
> > > > 1) a clean upgrade path for current HA replication users
> > > > 2) deprecate or integrate the current HA replication into the new
> > version
> > > >
> > > > I've opened this here because many of the HA replication users are
> devs
> > > too
> > > > and given that this isn't yet implemented: we're stull in the
> > > > design/proposal phase, so anyone that want to express their
> > > > ideas/opinions/POC on this, is invited to talk here ;)
> > > >
> > > > Cheers,
> > > > Franz
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Sent from:
> > > > http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
> > > >
> > >
> > --
> > Clebert Suconic
> >
>
--
Clebert Suconic
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] New Quorum vote pluggable implementation

nigro_franz
@clebert

> I think configuration and implementation are orthogonal here.
Totally agree!


@michael

> I think the migration should be seemless to current users. It should
> support current behaviours.

Agree as well: probably I've not been able to explain it clearly in the
previous emails: I was talking about the implementation indeed :)



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] New Quorum vote pluggable implementation

michaelpearce
"I was talking about the implementation indeed"

In that case disregard my email :)  this then should be invisible to
current / end users.

On Tue, 14 Apr 2020 at 07:33, nigro_franz <[hidden email]> wrote:

> @clebert
>
> > I think configuration and implementation are orthogonal here.
> Totally agree!
>
>
> @michael
>
> > I think the migration should be seemless to current users. It should
> > support current behaviours.
>
> Agree as well: probably I've not been able to explain it clearly in the
> previous emails: I was talking about the implementation indeed :)
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
>
12