Feature #112
closed
conduct POC using Spring Batch framework
Added by chin-yeh over 13 years ago.
Updated over 10 years ago.
Description
To demonstrate the concept, I've used one of the existing batch programs, MonthlyGstSalesReport as an example.
The batch program will:
- query record from store database and transform data in certain columns
- query record from my_store database and transform data in certain columns
- output the transformed data to a double-comma-quoted CSV file
- run every month (tweaked to run daily)
List of component used (in Spring-Batch):
- ItemReader - to query record from database
- Processor - to transform data
- ItemWriter - to output the transformed data to file
To test run the program:
- via Spring Batch Admin
- go to http://192.168.2.13:8080/batch-console/jobs
- click on the gstSalesReport job
- in the JobParameter input box, enter:
format:
report.date=yyyy-MM-dd,run.id=<<any integer value>>
example:
report.date=2010-01-01,run.id=123
- click the Launch button to start the job
- via Command Line
- login to 192.168.2.13 as support user
- execute the following command:
/opt/batch-poc/scripts/gstSalesReport.sh
How to monitor/query the job status:
- go to http://192.168.2.13:8080/batch-console/jobs
- click on the gstSalesReport job
You could refer to wiki for more information about this POC.
Success case tested. Can simulate a failed case and retry scenario?
To simulate a failed job execution, you either quiesce store or my_store database. By doing this, it will make one of the steps in the job failed. Once quiesced, go to to the particular failed job execution and press the restart button.
How to quiesce db: (need to restart the respective application server after done)
- connect to the db, e.g. store
$ db2 connect to store user db2inst1 using xxxxx
- execute the command:
$ db2 QUIESCE db immediate
Added a new batch program, OfflineVip. This program processes those offline transaction which contains new registration, renewal, and cancellation.
input:
- filename (e.g. file=VIP0101.DAT)
processor:
- check if the shopper id exists in memberid_audit table
* use the id in table
- check if the sponsor's shopper id exists in memberid_audit table
* use the id in the table
- if creator id is FOC_VIP, set freevip to F and freevipsponsor to sponsorid else both is null
- set the rp and rp1 earned to 0 if empty
- set the dob to null if empty
- if new vip, set trxcode to 1240 and trxdesc to offline vip reg; if renewal, set trxcode to 1320 and trxdesc to offline vip ren.
- encode the following based on filename:
* sanick1
* sanick2
* sanick3
* address1
* addr2
* addr3
* city
* state
- check if the shopper id exists
- generate password from handphone if ic is empty and country is usa
- set shopper group to 7 and premiumvip to n for vip; for premium vip, group to 9 and premium vip to y
output:
- new VIP registration
- new Premium VIP registration
- cancellation (not implemented)
- renewal (not implemented)
- send welcome email (not implemented)
How To Run:
Input file are stored at: (contains files from April to 16-June)
/opt/batch-poc/data
All data will be saved into TSTORE database with ECOS schema.
To clear up test data, you may execute the following SQL:
delete from ECOS.SHOPPER;
delete from ECOS.SHADDR;
delete from ECOS.SHOPDET;
delete from ECOS.MEMBER_ACCOUNT;
delete from ECOS.TRANSACTION_032011;
delete from ECOS.TRANSACTION_042011;
delete from ECOS.TRANSACTION_052011;
delete from ECOS.TRANSACTION_062011;
- Status changed from Feedback to Closed
Also available in: Atom
PDF