[GitHub] activemq-artemis pull request #1775: ARTEMIS-1587 Add setting to control the...

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
40 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis pull request #1775: ARTEMIS-1587 Add setting to control the...

jbertram-2
GitHub user stanlyDoge opened a pull request:

    https://github.com/apache/activemq-artemis/pull/1775

    ARTEMIS-1587 Add setting to control the queue durable property for au…

    …to-created addresses

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/stanlyDoge/activemq-artemis ARTEMIS-1587

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/activemq-artemis/pull/1775.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1775
   
----
commit 5873e5b8d7d506c50f978b846f4f11520431ae99
Author: Stanislav Knot <sknot@...>
Date:   2018-01-09T10:47:10Z

    ARTEMIS-1587 Add setting to control the queue durable property for auto-created addresses

----


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
Github user michaelandrepearce commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    Why is this needed? If an address is allowed for queues to be auto created, the client describes if the queue should be durable or not. e.g. See JMS Subscription (which maps onto queue) you create a non durable or durable via the client, and it is reflected in the queue it creates.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user jostbg commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    I created the ARTEMIS issue as a followup to this https://issues.apache.org/jira/browse/ARTEMIS-1582. We actually have the use-case that for certain namespaces we want to dictate all properties of auto-created addresses/queues on the server-side. So the effective settings should not be guessed based on how a particular client tries to subscribes to a queue or sends a message to an address but based on a server side specification. The same way you can pre-create a queue and define it as non-durable. We need auto-created queues under certain namespaces to be non-durable no matter what one specific client tries to do. The reason is we only run the broker and do not control all the clients used by the different teams.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user michaelandrepearce commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    This is what security settings are for e.g. you can set for address A users can only create non-durable queues. If a client try to create a durable queue they will get security error. This feature is there today, already.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user jostbg commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    The problem is, that at least when using AMQP the client cannot control if a durable or non-durable queue is created. The broker always auto-creates durable queues. We discussed that at https://issues.apache.org/jira/browse/ARTEMIS-1582 where the conclusion was it may make sense to - instead of hard-code the durable property - extend the address settings.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user jostbg commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    I also don't see much of a difference if a client tries to send a message to a pre-created non-durable queue and but expects the queue to be durable over he a message sends to an auto-create address that then creates a non-durable queue on the fly while the client expects a durable queue. In both cases the client should know in advance what routing/durability properties he can use/expect for given address.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user michaelandrepearce commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    @jostbg it can, look at the AMQP JMS client, when via JMS  you make a shared subscription, or a shared durable subscription. if the
   
    On the Artemis side look at ProtonServerSenderContext, for how it maps this. But it looks at the source's TerminusDurability.
   



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user michaelandrepearce commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    @jostbg there is implications especially for JMS users, basically if a client requests a shared subscription, it should not marry up to a durable shared subscription or vice versa.
   
    "There is no restriction on durable subscriptions and shared non-durable subscriptions having the same name and clientId (which may be unset). Such subscriptions would be completely separate."  
   
    to avoid this due a prefix on the queue to separate the two, so on core this is prefixed "non-durable", see code here
   
    ```
       public static SimpleString createQueueNameForSubscription(final boolean isDurable,
                                                           final String clientID,
                                                           final String subscriptionName) {
          final String queueName;
          if (clientID != null) {
             if (isDurable) {
                queueName = ActiveMQDestination.escape(clientID) + SEPARATOR +
                   ActiveMQDestination.escape(subscriptionName);
             } else {
                queueName = "nonDurable" + SEPARATOR +
                   ActiveMQDestination.escape(clientID) + SEPARATOR +
                   ActiveMQDestination.escape(subscriptionName);
             }
          } else {
             if (isDurable) {
                queueName = ActiveMQDestination.escape(subscriptionName);
             } else {
                queueName = "nonDurable" + SEPARATOR +
                   ActiveMQDestination.escape(subscriptionName);
             }
          }
          return SimpleString.toSimpleString(queueName);
       }
   
    ```


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user michaelandrepearce commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    @jostbg from the sounds of that ticket, what you hit was a bug thats been fixed, the AMQP client should be able to request a durable or non durable queue, and the AMQP JMS client should make this cleanly exposed to end users without needing to get into AMQP detail.
   



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user jostbg commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    @michaelandrepearce What if the first client does not subscribe to any address but only sends an AMQP message to an address (either a queue or topic via QPid JMS). Based on what criterias should Artemis decide what to create? Currently even if the producer sends a non-durable message, durable queues are created.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user michaelandrepearce commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    @stanlyDoge as said earlier for this PR, if it is to go forwards id like to see
   
    - setting values should be enums, to cater for alternative way round so people can make them non-durable or durable - default should be null (which keeps todays behaviour)
   
    - it needs really good documentation with some big clear warnings about it. Including that setting these would break JMS specs.
   
    - needs cross protocol / client compatibility testing.



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user jostbg commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    I am not trying to annoying or anything, but how is a non-durable queue - that is auto created the moment the first client tries to use it (in whatever way) - less JMS compliant than a pre-created non-durable queue under the same name?


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user michaelandrepearce commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    JMS Queue is durable by spec.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user jostbg commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    So when we pre-define non-durable anycast queues in Artemis and a JMS client publishes to that queue or subscribes to it, we are actually already violating the JMS spec. Therefore it would no be different if the non-durable queue is pre-defined or auto-created.
   
    In our case we do not really care about the JMS spec. We only use the Qpid AMQP JMS client because it seems to be the only actually working and easy to use AMQP 1.0 client for Java. We tried IBM's mqlight which we didn't get to work in our scenario and also it seems to be dead (no development since 2016). SwiftMQ has an odd non-OSS license/distribution model. And proton-j is just too bare bone without proper documentation.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user michaelandrepearce commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    @jostbg as I said I’m not a fan, but if you do want to implement could you address the points I raised.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user jostbg commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    I am too new to Artemis to implement that properly but I hope @stanlyDoge can address your points.
    Thanks for your feedback.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user mtaylor commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    A lot of discussion here :).  This should not override what the client requests.  There are a couple of ways addresses and queues can be set up in Artemis.  
   
    The first way is to do what @michaelandrepearce has described.  The client is explicit about what it wants.  Typical example is JMS where the spec is very specific.  In this case the client passes a description of what it wants to the broker.  If the client requests a durable subscription, or similar then the broker will try to serve that request.  Providing the client has the correct security permissions and address settings it's all good.
   
    The second way is for the broker to decide how to handle things.  This can either be configured by being very specific about what resources can exist on the broker, for example, locking down all the addresses and pre-creating all queues and turning off any create permissions or auto-create settings or it can be configured to be open and auto-create things that aren't specified.  This setting is for this 2nd case.  With something like AMQP or STOMP the clients don't need to be aware of what type of resource they're sending or receiving from.  They only require an address to subscribe from or send to.  If the address the client asks for does not yet exist, the broker can create it automatically.  This is what these auto-create settings are used for.  They should not conflict with what clients ask for.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user jostbg commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    @mtaylor What is the difference between saying, "Artemis, create a non-durable queue named foo during server start" vs "Artemis create a non-durable queue named foo the moment it is first used."
   
    Why does it not seem to be an issue if I pre-create the non-durable anycast queue "foo" to which a JMS client can send a message but when that non-durable queue is auto-created the moment a message is send to it you suddenly talk about JMS spec violations. In both cases the result is the same, the JMS client sends a message to a non-durable queue. The only difference is the point in time when the queue is actually created.
   
    Following your argumentation, Artemis should generally prevent JMS clients trying to send a message to a non-persistent queue - no matter if it is pre-created or created on first use.



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user mtaylor commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    @jostbg The point is, if you want the broker to create something for you, then the broker needs to know what to create.  I used JMS clients as an example as they are usually specific about what they want, the same thing can happen in other protocols.  In the JMS case the client sends the broker the information it needs to create the appropriate queue with some additional bits of info.  The JMS client doesn't say, hey I want a durable queue with these properties (except the CORE client as it's tied to the Artemis implementation).  Instead, say the QPID AMQP client says, hey I want a JMS Durable Subscription.  It does this by setting some special JMS AMQP properties.  The AMQP protocol handler reads these properties and decides how to interpret them.  In some cases these get translated to durable queues, in others they are translated to non-durable queues.  Artemis has a underlying model that is protocol agnostic, so things are translated from API/protocol specifics down to the und
 erlying model.
   
    Coming back to your comments on https://issues.apache.org/jira/browse/ARTEMIS-1582.  
   
    There are two meanings of durable.  Durable in the context of a subscription (which is tied to the consumer connection life cycle), and durable in the context persistence (tied to the broker life cycle).  The reason you are seeing some queues be durable and others not, is because this is how the JMS spec is implemented using the underlying Artemis model.  An non-durable subscription means that the client only gets messages while it's connected.  If the server crashes, the client isn't connected anymore.
     In this case, the spec says, its fine to lose all the messages.  For this reason there's no point creating a durable queue.  In the JMS Queue (ANYCAST) scenario we need a durable queue because the messages outlive the subscriber connection life cycle.  This is also true for durable subscriptions.
   
    Regarding your comment on misconfiguration, yes you can do that, but you're specifically overriding broker behaviour to do something else.  You're welcome to do this, but you have to be aware that you are breaking protocol/API contracts by doing so.


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] activemq-artemis issue #1775: ARTEMIS-1587 Add setting to control the queue ...

jbertram-2
In reply to this post by jbertram-2
Github user jostbg commented on the issue:

    https://github.com/apache/activemq-artemis/pull/1775
 
    @mtaylor When I talked about durability in the context of queues I meant persistence (writing messages to disk) and not durable subscriptions.
    We want to allow clients to create their own queues under a  given namespace but we do not want to persist these messages (so they will not survive a broker restart). How else if not by declaring these auto-created queues as non-durable can we achieve that? Maybe there is some other setting I am simply missing here. Then we don't need a flag on auto-created queues to specify their durability mode.


---
12