Feature #112: conduct POC using Spring Batch framework - POC - Collaboration Platform for eCosway

Actions

Copy link

Feature #112

closed

conduct POC using Spring Batch framework

Added by chin-yeh over 13 years ago. Updated over 10 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Chooi-Mey

Start date:

06/07/2011

Due date:

% Done:

100%

Estimated time:

Resolution:

Description

To demonstrate the concept, I've used one of the existing batch programs, MonthlyGstSalesReport as an example.

The batch program will:

query record from store database and transform data in certain columns
query record from my_store database and transform data in certain columns
output the transformed data to a double-comma-quoted CSV file
run every month (tweaked to run daily)

List of component used (in Spring-Batch):

ItemReader - to query record from database
Processor - to transform data
ItemWriter - to output the transformed data to file

To test run the program:

via Spring Batch Admin
1. go to http://192.168.2.13:8080/batch-console/jobs
2. click on the gstSalesReport job
3. in the JobParameter input box, enter:
  format:
```
report.date=yyyy-MM-dd,run.id=<<any integer value>>
```
  example:
```
report.date=2010-01-01,run.id=123
```
4. click the Launch button to start the job
via Command Line
1. login to 192.168.2.13 as support user
2. execute the following command:
```
/opt/batch-poc/scripts/gstSalesReport.sh
```

How to monitor/query the job status:

go to http://192.168.2.13:8080/batch-console/jobs
click on the gstSalesReport job

You could refer to wiki for more information about this POC.

Actions

Copy link

Updated by Chooi-Mey over 13 years ago

Success case tested. Can simulate a failed case and retry scenario?

Actions

Copy link

Updated by chin-yeh over 13 years ago

To simulate a failed job execution, you either quiesce store or my_store database. By doing this, it will make one of the steps in the job failed. Once quiesced, go to to the particular failed job execution and press the restart button.

How to quiesce db: (need to restart the respective application server after done)

connect to the db, e.g. store

$ db2 connect to store user db2inst1 using xxxxx

execute the command:
```
$ db2 QUIESCE db immediate
```

Actions

Copy link

Updated by chin-yeh over 13 years ago

Added a new batch program, OfflineVip. This program processes those offline transaction which contains new registration, renewal, and cancellation.

input:

filename (e.g. file=VIP0101.DAT)

processor:

check if the shopper id exists in memberid_audit table * use the id in table
check if the sponsor's shopper id exists in memberid_audit table * use the id in the table
if creator id is FOC_VIP, set freevip to F and freevipsponsor to sponsorid else both is null
set the rp and rp1 earned to 0 if empty
set the dob to null if empty
if new vip, set trxcode to 1240 and trxdesc to offline vip reg; if renewal, set trxcode to 1320 and trxdesc to offline vip ren.
encode the following based on filename: * sanick1 * sanick2 * sanick3 * address1 * addr2 * addr3 * city * state
check if the shopper id exists
generate password from handphone if ic is empty and country is usa
set shopper group to 7 and premiumvip to n for vip; for premium vip, group to 9 and premium vip to y

output:

new VIP registration
new Premium VIP registration
cancellation (not implemented)
renewal (not implemented)
send welcome email (not implemented)

How To Run:

Input file are stored at: (contains files from April to 16-June)

/opt/batch-poc/data

All data will be saved into TSTORE database with ECOS schema.

Actions

Copy link

Updated by chin-yeh over 13 years ago

To clear up test data, you may execute the following SQL:

delete from ECOS.SHOPPER;
delete from ECOS.SHADDR;
delete from ECOS.SHOPDET;
delete from ECOS.MEMBER_ACCOUNT;
delete from ECOS.TRANSACTION_032011;
delete from ECOS.TRANSACTION_042011;
delete from ECOS.TRANSACTION_052011;
delete from ECOS.TRANSACTION_062011;

Actions

Copy link