Added array of sObjects and run at interval support

[ppf2]

The original author of this plugin intended to provide support to fetch an array of sObjects (#4) and an option to run on an interval (continuously). Some of this code got lost and was never merged into this repository. This is an attempt to add them back.

@acchen97 @suyograo

[mPanasiewicz]

@ppf2 Are you going to fix this build and merge PR? I need salesforce data refreshing.

[fransf-wtax]

Hi all, any reason this hasn't been merged yet? Would be a lot more convenient if it was part of the official plugin that you install with bin/logstash-plugin install. Happy to help in any way I can.

I'm thinking it probably makes sense to split this into 2 PRs? One to add the interval parameter, and the other for supporting multiple sObjects. The interval change seems very simple and non-breaking, so perhaps it can be merged more easily as a separate PR?

[fransf-wtax]

@ppf2 @gtback Could you assist with getting this into the next official release of this plugin? What is the process for getting something into an official Logstash plugin release? Thanks.

[gtback]

Hi @fransf-wtax . I'm not directly involved with managing Logstash plugins but @jsvd @flexitrev should be able to help.

[qury]

Hi All,
Who is responsible for this plugin?
I would very much like to see the interval functionality in the next release.

[fransf-wtax]

Hi All, Who is responsible for this plugin? I would very much like to see the interval functionality in the next release.

Hi @qury , @jsvd gave me access a while back as the project had become more or less owner-less and I had the intention to try and get some of the changes like this one out in an official release, however I got a bit stuck and then other things took over.

I'll have another look at this in the coming weeks.

[qury]

@fransf-wtax
I do not want to rush you, but is there any progress on your part?

[fransf-wtax]

@fransf-wtax I do not want to rush you, but is there any progress on your part?

Hey @qury , sorry, no. Thanks for the reminder :)
I'm going to work on it now. There's two things I want to do:

• split out the interval stuff into a separate feature branch
• add some kind of test that demonstrates it actually works

Hope to get a 3.3.0 of this plugin out by the end of this week. I'll comment the link to the new pull request once I have it and then close this one.

[fransf-wtax]

@qury I've added the functionality to a new branch and opened a PR for it: #37. Do you want to have a look at it and see if the change would work for you like this? Otherwise I'll release later this week. I'll also include #38 in that release which adds incremental data loading, which will be useful for objects with lots of records of which most don't change a lot, and for preventing duplicates when running the plugin multiple times or on a schedule.

[fransflippo]

I've released version 3.3.0 of the plugin which now contains the interval support from #37. I don't see the value of the multiple sObjects part of this PR: you can easily have two instances of the plugin in your Logstash pipeline, one for each object, which as far as I can tell achieves the exact same thing, albeit with slightly more configuration.