Could ActiveMQ be the right solution?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Could ActiveMQ be the right solution?

JavaEsse
Hello,

I'm doing some research on possible solutions to a process that is currently being completed semi manually and semi automated.  The process itself consists of feeding several thousand ID numbers to a legacy C++ application, which in turn, after some processing of its own, sends more data to an external system.  It is automated in that there is a batch file that kicks off the C++ application.  The C++ application opens up a text file, reads an ID number and then processes it.  It repeats this for as many IDs as there are in the text file.  However, it is manual in that there are several text files, broken up into 500 IDs per file (which we create), that are sent over several weeks.  The reason for this is a limitation in the external system that we have no control over.  Its 500 IDs per day, no questions asked.  Because of this limitation, we have to go in and run the batch file for each of those text files.  In addition, we have to monitor as each text file is processed in case of a failure.  If a failure occurs, we have to go into the text file, delete all of the IDs that were successfully processed, and then run the batch process again.

At any rate, as much as we would like to, we are unable to rip the current system out and replace it all together.  The legacy C++ component must remain intact.  I am looking at possibly modifying the C++ application to pick up the ID numbers from an external application--instead of a text file--that keeps count of the number of IDs processed in a given day.  I'd like to continue logging error messages, but instead of crapping out the entire process, I'd like for them to be ignored and for the process to continue until the 500 ID threshhold is reached for the day.  I'm not too familiar with JMS technology, but from what I have read a JMS based external application may be a good candidate, but I'm thinking it may be overkill.  I do like the fact that it is reliable, loosely coupled, and it's asynchronous.  So I guess my question is if ActiveMQ is the right solution for this problem?  Or is it overkill?

Thanks in advance for the assistance.
Reply | Threaded
Open this post in threaded view
|

Re: Could ActiveMQ be the right solution?

Bruce Snyder
On Fri, Jun 12, 2009 at 1:18 PM, JavaEsse<[hidden email]> wrote:

>
> Hello,
>
> I'm doing some research on possible solutions to a process that is currently
> being completed semi manually and semi automated.  The process itself
> consists of feeding several thousand ID numbers to a legacy C++ application,
> which in turn, after some processing of its own, sends more data to an
> external system.  It is automated in that there is a batch file that kicks
> off the C++ application.  The C++ application opens up a text file, reads an
> ID number and then processes it.  It repeats this for as many IDs as there
> are in the text file.  However, it is manual in that there are several text
> files, broken up into 500 IDs per file (which we create), that are sent over
> several weeks.  The reason for this is a limitation in the external system
> that we have no control over.  Its 500 IDs per day, no questions asked.
> Because of this limitation, we have to go in and run the batch file for each
> of those text files.  In addition, we have to monitor as each text file is
> processed in case of a failure.  If a failure occurs, we have to go into the
> text file, delete all of the IDs that were successfully processed, and then
> run the batch process again.
>
> At any rate, as much as we would like to, we are unable to rip the current
> system out and replace it all together.  The legacy C++ component must
> remain intact.  I am looking at possibly modifying the C++ application to
> pick up the ID numbers from an external application--instead of a text
> file--that keeps count of the number of IDs processed in a given day.  I'd
> like to continue logging error messages, but instead of crapping out the
> entire process, I'd like for them to be ignored and for the process to
> continue until the 500 ID threshhold is reached for the day.  I'm not too
> familiar with JMS technology, but from what I have read a JMS based external
> application may be a good candidate, but I'm thinking it may be overkill.  I
> do like the fact that it is reliable, loosely coupled, and it's
> asynchronous.  So I guess my question is if ActiveMQ is the right solution
> for this problem?  Or is it overkill?

ActiveMQ could certainly help you here to be the backbone for the
messaging. But you may want to look at Apache Camel for reading the
file and handling that as well. See the camel-file component:

http://camel.apache.org/file.html

Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'

ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder
Reply | Threaded
Open this post in threaded view
|

Re: Could ActiveMQ be the right solution?

Kenny Stone
He is using the file as middleware.  ActiveMQ fits in by replacing the file
or else it's pretty useless in my opinion.

On Fri, Jun 12, 2009 at 3:37 PM, Bruce Snyder <[hidden email]>wrote:

> On Fri, Jun 12, 2009 at 1:18 PM, JavaEsse<[hidden email]> wrote:
> >
> > Hello,
> >
> > I'm doing some research on possible solutions to a process that is
> currently
> > being completed semi manually and semi automated.  The process itself
> > consists of feeding several thousand ID numbers to a legacy C++
> application,
> > which in turn, after some processing of its own, sends more data to an
> > external system.  It is automated in that there is a batch file that
> kicks
> > off the C++ application.  The C++ application opens up a text file, reads
> an
> > ID number and then processes it.  It repeats this for as many IDs as
> there
> > are in the text file.  However, it is manual in that there are several
> text
> > files, broken up into 500 IDs per file (which we create), that are sent
> over
> > several weeks.  The reason for this is a limitation in the external
> system
> > that we have no control over.  Its 500 IDs per day, no questions asked.
> > Because of this limitation, we have to go in and run the batch file for
> each
> > of those text files.  In addition, we have to monitor as each text file
> is
> > processed in case of a failure.  If a failure occurs, we have to go into
> the
> > text file, delete all of the IDs that were successfully processed, and
> then
> > run the batch process again.
> >
> > At any rate, as much as we would like to, we are unable to rip the
> current
> > system out and replace it all together.  The legacy C++ component must
> > remain intact.  I am looking at possibly modifying the C++ application to
> > pick up the ID numbers from an external application--instead of a text
> > file--that keeps count of the number of IDs processed in a given day.
>  I'd
> > like to continue logging error messages, but instead of crapping out the
> > entire process, I'd like for them to be ignored and for the process to
> > continue until the 500 ID threshhold is reached for the day.  I'm not too
> > familiar with JMS technology, but from what I have read a JMS based
> external
> > application may be a good candidate, but I'm thinking it may be overkill.
>  I
> > do like the fact that it is reliable, loosely coupled, and it's
> > asynchronous.  So I guess my question is if ActiveMQ is the right
> solution
> > for this problem?  Or is it overkill?
>
> ActiveMQ could certainly help you here to be the backbone for the
> messaging. But you may want to look at Apache Camel for reading the
> file and handling that as well. See the camel-file component:
>
> http://camel.apache.org/file.html
>
> Bruce
> --
> perl -e 'print
> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> );'
>
> ActiveMQ in Action: http://bit.ly/2je6cQ
> Blog: http://bruceblog.org/
> Twitter: http://twitter.com/brucesnyder
>



--
Kenny Stone
Connamara Systems, LLC
Reply | Threaded
Open this post in threaded view
|

Re: Could ActiveMQ be the right solution?

Bruce Snyder
On Fri, Jun 12, 2009 at 2:46 PM, Kenny Stone<[hidden email]> wrote:
> He is using the file as middleware.  ActiveMQ fits in by replacing the file
> or else it's pretty useless in my opinion.

He indicated that the external system cannot be changed, so I figured
that this system would continue to provide the IDs via the files.

Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'

ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder