Batch Framework‏

Java Batch Job Framework - http://jbjf.sourceforge.net/documentation.html#

TERASOLUNA Batch解惑 http://www.offshore-jp.com/html/49/t-549.html

Enterprise Batch Server  http://batchserver.sourceforge.net/

CA 7® Workload Automation 参考





Batch Application Frameworks

 

How do I do bulk processing? For example, how do I get my ETL done to load the warehouse?

 

Here are some batch application frameworks, in no particular order.

Vestigo JavaBatch - a zOS (IBM Mainframe) framework for running batch applications written in Java.
CleverSoft Sodiax includes some batch control capability.
Xebia J2EE Batch Framework - depends on WebLogic and Flux for job scheduling.
BMC Control-M - a sophisticated batch control product.
Microsoft Job Scheduling SMF - A windows framework for running batch applications.
Taricon Xi-Batch - job scheduling and control.
CA Unicenter CA-7 - very, very complete scheduling solution.
OpenPBS.ORG Portable Batch System - Looks reasonably complete, and well-established.
Digipede http://www.digipede.net/
http://www.digipede.net/" target="NewWindow">Digipede Network
- which raises the question, do grid controllers count?

What do these frameworks require? Almost nothing... and that's a problem.

What Makes Batch Processing Complex?

In the web framework world, the directory structure for web applications is tightly constrained by the web server and the security considerations around what files can be served. In the desktop GUI world -- Windows mostly -- your application's location is defined by convention, and your data's location is often left to the user to make wise choices.

In the batch control applications listed above, however, directory structures are left unspecified. In my experience, this leaves people foundering, unaware of precisely what they must specify at a broad, architectural level. These products leave too many degrees of freedom.

This foundering can be compounded by enterprise's shift to midrange processing. Some places are growing out of low-end Windows boxes. Other places are not expanding their zOS mainframes and are adding midrange servers instead. Whether growing or down-sizing, they find themselves lost in the UNIX directory tree, unable to determine where files should be located.

The batch-design foundering is further compounded by the implementation of a data warehouse, which is typified by long-running, complex batch processing. In many cases, programmers have more experience with interactive applications and small, simple batch interfaces. This kind of long-running, complex batch job is new to some programmers.

Filling in the Solution.

To prevent foundering, and get the project afloat, we must manage the complexity of batch processing. Data warehouse loads, for example, are already complex; too much freedom in the batch environment adds complexity without creating real value.

This is a four-step process:
1. Document the audit and control requirements. These include non-functional features of batch processing, and functional requirements outside the business purpose of the application software. These are the worst kind of requirements to capture because there's so little user involvement -- it's all IT navel-gazing.
2. Develop a Framework. The important thing is to pare down the degrees of freedom to arrive at the simplest possible batch processing environment that fills audit and control objectives. This is actually hard to do because IT folks tend to over-write their internal requirements.
3. Catalog your basic Design Patterns. Batch jobs are all very similar, and a few key superclasses will assure that every job fits the framework and supports the essential design patterns.
4. Supplement with Programming in the Large patterns. Avoid the shell; it's trouble waiting to happen. These additional design patterns add considerable flexibility and avoid the performance and maintenance costs of doing too much shell programming.

The issue is that the framework is elusive. Many things can work, but what's the simplest approach that assures compliance with the control objectives?
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值