Spring Batch continuation.

To start with for Spring Batch, there are three basic components,

1)  A reading step.

2) A Processor step.

3) A writer step.

The assumptions that Spring Batch API's are built out are, Each operation in the batch is going to be performed as different steps. All the steps are collected to form a job. The steps involved are further broken down into a read step, where you will read a flat file or comma separated files or a database. Once the read operation is performed, the idea is application will undergo a transformation.  Upon the transformation there will be write step, where data will be persisted or written into a medium where it will be an input for the next step of the Job.

Keeping our Flickr program in mind, we are going to categorize our flow to one simple Job, the job will have one step. The step will be split into three steps, a read step where data will be read from local hard disk, second step will be the upload process to Flickr, last step is to write the success info into a database to run some reports.

Step 1) Read data file; here is a code snippet for Reading information.
First Turning our attention to the spring config file:

Spring-Config File

We would want to account for couple of more spring config files-

common-context.xml
flickrbatchwithdb.xml

Secondly the input stream reader. In this code we are first examining the input/reader step.


package com.skminfotek.flickr.springbatch;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStreamReader;
import java.nio.charset.Charset;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;

import org.apache.commons.io.FileUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.batch.item.ItemStream;
import org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.util.ClassUtils;

import com.aetrion.flickr.uploader.Uploader;

public class UploadReader extends
AbstractItemCountingItemStreamItemReader implements ItemStream,
InitializingBean {
public static final String DEFAULT_CHARSET = Charset.defaultCharset()
.name();
private static final Log log = LogFactory.getLog(Uploader.class);
private boolean noInput = false;

public UploadReader() {
setName(ClassUtils.getShortName(UploadReader.class));
}

@Override
protected void doClose() throws Exception {
//We are not Closing anything here.

}

public void afterPropertiesSet() throws Exception {
// Assert.isTrue(noInput);
}

@Override
@SuppressWarnings("unchecked")
protected T doRead() throws Exception {
log.debug("Do Read UploadReader Method Start");
if (!noInput) {
Map iReader = new HashMap();
String[] extns = { "jpg" };
Iterator fileIteator = FileUtils.iterateFiles(new File(
"/Users/sats/projects/bin"),
extns, true);
while (fileIteator.hasNext()) {
File file = (File) fileIteator.next();
BufferedReader reader = new BufferedReader(
new InputStreamReader(new FileInputStream(file)));
iReader.put(file.getName(), reader);
}
noInput = true;
log.debug("Do Read UploadReader Method End Returns Data");
return (T) iReader;
}
log.debug("Do Read UploadReader Method End No Data");
return null;
}

@Override
protected void doOpen() throws Exception {
//There is nothing to open.
}

}

Important pointers to note are the inheritance hierarchy i.e. extends clause of AbstractItemCountingItemStreamItemReader, ItemStream. Other important note was the constructor definition

public UploadReader() {
setName(ClassUtils.getShortName(UploadReader.class));
}


As we discussed earlier the read step is reading a jpg file and converting it into a bufferedreader.

Now we got the reader, next would be the writer. Here is the code-piece for writer. Spring batch has prebuilt api's that can integrate with Hibernate.

package com.skminfotek.flickr.springbatch;

import java.io.BufferedReader;
import java.util.List;
import java.util.Map;

import org.hibernate.SessionFactory;
import org.springframework.batch.item.ItemWriter;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.orm.hibernate3.HibernateOperations;
import org.springframework.orm.hibernate3.HibernateTemplate;
import org.springframework.util.Assert;

import com.skminfotek.flickr.dao.PhotosDataDAO;
import com.skminfotek.flickr.dataobjects.PhotosData;

public class UploadWriter implements ItemWriter, InitializingBean {
private PhotosDataDAO photoDao;
static final long FLICKR_UPLOADER = 001;
private HibernateOperations hibernateTemplate;

@SuppressWarnings("unchecked")
public void write(List items) throws Exception {
for (Object fileReadList : items) {
for (String bufr : ((Map) fileReadList)
.keySet()) {
PhotosData phData = new PhotosData();
phData.setPicName(bufr);
phData.setAppLoaded(FLICKR_UPLOADER);
photoDao.save(phData);
}
}
try {
hibernateTemplate.flush();
} finally {
// this should happen automatically on commit, but to be on the safe
// side...
hibernateTemplate.clear();
}

}

public void setSessionFactory(SessionFactory sessionFactory) {
this.hibernateTemplate = new HibernateTemplate(sessionFactory);
}

public void afterPropertiesSet() throws Exception {
Assert.notNull(hibernateTemplate,
"Hibernate session factory must be set");
Assert.notNull(photoDao, "Delegate DAO must be set");
}

public PhotosDataDAO getPhotoDao() {
return photoDao;
}

public void setPhotoDao(PhotosDataDAO photoDao) {
this.photoDao = photoDao;
}
}

Finally the hibernate config that is required for this module

hibernate-config file
Hbm File

Finally the code base for testing all these-


package com.skminfotek.flickr;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = { "flickrbatchconfig.xml" })
public class JobFunctionalTest implements ApplicationContextAware {

/** Logger */
protected final Log logger = LogFactory.getLog(getClass());

protected ApplicationContext applicationContext;

private JobLauncher launcher;

private Job job;

private JobParameters jobParameters = new JobParameters();

public void setApplicationContext(ApplicationContext applicationContext) {
this.applicationContext = applicationContext;
}

@Autowired
public void setLauncher(JobLauncher bootstrap) {
this.launcher = bootstrap;
}

@Autowired
public void setJob(Job job) {
this.job = job;
}

public Job getJob() {
return job;
}

protected String getJobName() {
return job.getName();
}

public void setJobParameters(JobParameters jobParameters) {
this.jobParameters = jobParameters;
}

@Test
public void testLaunchJob() throws Exception {
getLauncher().run(job, jobParameters);
}

/**
* Public getter for the launcher.
*
* @return the launcher
*/
protected JobLauncher getLauncher() {
return launcher;
}
}


Hope this overview gives some insights of using the spring batch.