Why is this needed? If an address is allowed for queues to be auto created, the client describes if the queue should be durable or not. e.g. See JMS Subscription (which maps onto queue) you create a non durable or durable via the client, and it is reflected in the queue it creates.
I created the ARTEMIS issue as a followup to this https://issues.apache.org/jira/browse/ARTEMIS-1582. We actually have the use-case that for certain namespaces we want to dictate all properties of auto-created addresses/queues on the server-side. So the effective settings should not be guessed based on how a particular client tries to subscribes to a queue or sends a message to an address but based on a server side specification. The same way you can pre-create a queue and define it as non-durable. We need auto-created queues under certain namespaces to be non-durable no matter what one specific client tries to do. The reason is we only run the broker and do not control all the clients used by the different teams.
This is what security settings are for e.g. you can set for address A users can only create non-durable queues. If a client try to create a durable queue they will get security error. This feature is there today, already.
The problem is, that at least when using AMQP the client cannot control if a durable or non-durable queue is created. The broker always auto-creates durable queues. We discussed that at https://issues.apache.org/jira/browse/ARTEMIS-1582 where the conclusion was it may make sense to - instead of hard-code the durable property - extend the address settings.
I also don't see much of a difference if a client tries to send a message to a pre-created non-durable queue and but expects the queue to be durable over he a message sends to an auto-create address that then creates a non-durable queue on the fly while the client expects a durable queue. In both cases the client should know in advance what routing/durability properties he can use/expect for given address.
@jostbg from the sounds of that ticket, what you hit was a bug thats been fixed, the AMQP client should be able to request a durable or non durable queue, and the AMQP JMS client should make this cleanly exposed to end users without needing to get into AMQP detail.
@michaelandrepearce What if the first client does not subscribe to any address but only sends an AMQP message to an address (either a queue or topic via QPid JMS). Based on what criterias should Artemis decide what to create? Currently even if the producer sends a non-durable message, durable queues are created.
I am not trying to annoying or anything, but how is a non-durable queue - that is auto created the moment the first client tries to use it (in whatever way) - less JMS compliant than a pre-created non-durable queue under the same name?
So when we pre-define non-durable anycast queues in Artemis and a JMS client publishes to that queue or subscribes to it, we are actually already violating the JMS spec. Therefore it would no be different if the non-durable queue is pre-defined or auto-created.
In our case we do not really care about the JMS spec. We only use the Qpid AMQP JMS client because it seems to be the only actually working and easy to use AMQP 1.0 client for Java. We tried IBM's mqlight which we didn't get to work in our scenario and also it seems to be dead (no development since 2016). SwiftMQ has an odd non-OSS license/distribution model. And proton-j is just too bare bone without proper documentation.
A lot of discussion here :). This should not override what the client requests. There are a couple of ways addresses and queues can be set up in Artemis.
The first way is to do what @michaelandrepearce has described. The client is explicit about what it wants. Typical example is JMS where the spec is very specific. In this case the client passes a description of what it wants to the broker. If the client requests a durable subscription, or similar then the broker will try to serve that request. Providing the client has the correct security permissions and address settings it's all good.
The second way is for the broker to decide how to handle things. This can either be configured by being very specific about what resources can exist on the broker, for example, locking down all the addresses and pre-creating all queues and turning off any create permissions or auto-create settings or it can be configured to be open and auto-create things that aren't specified. This setting is for this 2nd case. With something like AMQP or STOMP the clients don't need to be aware of what type of resource they're sending or receiving from. They only require an address to subscribe from or send to. If the address the client asks for does not yet exist, the broker can create it automatically. This is what these auto-create settings are used for. They should not conflict with what clients ask for.
@mtaylor What is the difference between saying, "Artemis, create a non-durable queue named foo during server start" vs "Artemis create a non-durable queue named foo the moment it is first used."
Why does it not seem to be an issue if I pre-create the non-durable anycast queue "foo" to which a JMS client can send a message but when that non-durable queue is auto-created the moment a message is send to it you suddenly talk about JMS spec violations. In both cases the result is the same, the JMS client sends a message to a non-durable queue. The only difference is the point in time when the queue is actually created.
Following your argumentation, Artemis should generally prevent JMS clients trying to send a message to a non-persistent queue - no matter if it is pre-created or created on first use.
@jostbg The point is, if you want the broker to create something for you, then the broker needs to know what to create. I used JMS clients as an example as they are usually specific about what they want, the same thing can happen in other protocols. In the JMS case the client sends the broker the information it needs to create the appropriate queue with some additional bits of info. The JMS client doesn't say, hey I want a durable queue with these properties (except the CORE client as it's tied to the Artemis implementation). Instead, say the QPID AMQP client says, hey I want a JMS Durable Subscription. It does this by setting some special JMS AMQP properties. The AMQP protocol handler reads these properties and decides how to interpret them. In some cases these get translated to durable queues, in others they are translated to non-durable queues. Artemis has a underlying model that is protocol agnostic, so things are translated from API/protocol specifics down to the und
There are two meanings of durable. Durable in the context of a subscription (which is tied to the consumer connection life cycle), and durable in the context persistence (tied to the broker life cycle). The reason you are seeing some queues be durable and others not, is because this is how the JMS spec is implemented using the underlying Artemis model. An non-durable subscription means that the client only gets messages while it's connected. If the server crashes, the client isn't connected anymore.
In this case, the spec says, its fine to lose all the messages. For this reason there's no point creating a durable queue. In the JMS Queue (ANYCAST) scenario we need a durable queue because the messages outlive the subscriber connection life cycle. This is also true for durable subscriptions.
Regarding your comment on misconfiguration, yes you can do that, but you're specifically overriding broker behaviour to do something else. You're welcome to do this, but you have to be aware that you are breaking protocol/API contracts by doing so.
@mtaylor When I talked about durability in the context of queues I meant persistence (writing messages to disk) and not durable subscriptions.
We want to allow clients to create their own queues under a given namespace but we do not want to persist these messages (so they will not survive a broker restart). How else if not by declaring these auto-created queues as non-durable can we achieve that? Maybe there is some other setting I am simply missing here. Then we don't need a flag on auto-created queues to specify their durability mode.