Work towards automated cleaning of JcrJobRepository

[sagarsane]

No description provided.

[sagarsane]

@jazeren1 @jbornemann @jdigger just WIP. Need to still properly test this and implement unit tests

[sagarsane]

Opening this up for some initial "approach" review.

Currently I have a servlet that's triggering the cleaning. I'll be updating this to have a Cron like we discussed. Wanted to get some feedback on the approach I have taken.

After I almost finished the job repp cleanup functionality .. But then I realized that there are way too many private methods in the service I have for cleanup. (this commit : 1e51ff8)

Because of that, I thought it'd be better "host" the functionality on the DAO objects themselves.. Felt like that looked and organized better. ( this commit: 7d855e2)

Does the latter approach look better? In order to make that happen, I had to extend Spring Batch's DAO interfaces to add the additional APIs we need for clean-up.

@jdigger @jazeren1 @jbornemann @masroormohammed can you please take a look?

[sagarsane]

Ugh hold on. Tests are failing. Taking a look.

[sagarsane]

I'll change the logging to debug, etc. when this is good to go. I figured the logging will help right now during testing.

Also made the change in grabbit version right now for easier installation for testing.

To test,

gw clean build install -x uninstallBundles -x uninstall -x remove

I've had to ignore uninstall. Seems to be having issues with other bundles in AEM :(

Run some content syncs for any number of paths .. or if you already have some data under

/var/grabbit/job/repository/jobExecutions then you probably have some data you can use to test the clean up.

Current testing is to do a POST to http://localhost:4502/grabbit/jobrepository/clean and add a hours parameter that should be an integer.

This number can be any "integer" .. I have tested that it works fine for > 24 hours as well. More-over, I am also logging everything that's going on .. along with the Date its using to compare and everything. So your logs will look something like this :

07.04.2016 16:20:01.267 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.JcrJobExecutionDao JobExecutions: [/var/grabbit/job/repository/jobExecutions/1628554003093086836, /var/grabbit/job/repository/jobExecutions/5944394638596509927], size: 2
07.04.2016 16:20:01.269 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.JcrJobExecutionDao Current time: Thu Apr 07 16:20:01 UTC 2016
07.04.2016 16:20:01.269 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.JcrJobExecutionDao Hours 24 .. OlderThanHours Time: Wed Apr 06 16:20:01 UTC 2016
07.04.2016 16:20:01.271 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.JcrJobExecutionDao JobExecutionsOlder than 24 hours: [] , length: 0
07.04.2016 16:20:01.277 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.services.impl.DefaultCleanJobRepository jobInstancesToRemove: [], size: 0
07.04.2016 16:20:01.278 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.services.impl.DefaultCleanJobRepository stepExecutionsToRemove: [], size: 0
07.04.2016 16:20:01.278 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.services.impl.DefaultCleanJobRepository jobExecutionContextsToRemove: [], size: 0
07.04.2016 16:20:01.278 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.services.impl.DefaultCleanJobRepository stepExecutionContextsToResource: [], size: 0
07.04.2016 16:20:01.278 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.services.impl.DefaultCleanJobRepository Removing 0 JobInstances
07.04.2016 16:20:01.278 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.services.impl.DefaultCleanJobRepository Removing 0 JobExecutions
07.04.2016 16:20:01.278 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.services.impl.DefaultCleanJobRepository Removing 0 StepExecutions
07.04.2016 16:20:01.278 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.services.impl.DefaultCleanJobRepository Removing 0 JobExecutionContexts
07.04.2016 16:20:01.278 *INFO* [10.0.2.2 [1460046001202] GET /bin/cleanRepository HTTP/1.1] com.twcable.grabbit.spring.batch.repository.services.impl.DefaultCleanJobRepository Removing 0 StepExecutionContexts

You can play around with the "hours" param for the servlet and see how it performs .. deletions will be made based on Comparison between "Date.now() - hours" and "All COMPLETED and FAILED JobExecutions" ... it will also delete all the corresponding "StepExecutions", "JobInstances" and "ExecutionContexts" ...

[sagarsane]

Other "BatchStatus" options are

STARTING, STARTED, STOPPING, STOPPED, ABANDONED, UNKNOWN;

So, should we query for everything except "STARTING" and "STARTED" and "STOPPING" ?

[jazeren1]

@sagarsane whats the difference between STARTING and STARTED?

[sagarsane]

From what I understand .. its similar to OSGi's STARTING and ACTIVE bundle states (not 1-1 comparison tho :) )

https://blog.codecentric.de/en/2014/04/spring-batch-batchstatus-state-transitions/ has a bit more info.

[sagarsane]

!fixed

[sagarsane]

Done with some main CR comments. Still not done anything to suggestions like #107 (comment) .. I like that idea but wanted to discuss that first.

@jazeren1 @jbornemann @jdigger @masroormohammed please take a look .

[Dnjoroge]

👍 Testing

[jbornemann]

s/s/resource/ ?