【原创】大数据基础之Chronos

最新推荐文章于 2024-09-27 10:47:12 发布

weixin_30466039

最新推荐文章于 2024-09-27 10:47:12 发布

阅读量189

点赞数

文章标签：大数据运维 shell

原文链接：http://www.cnblogs.com/barneywill/p/11051254.html

版权

官方：https://mesos.github.io/chronos/

mesos集群中替换crontab

Chronos

A fault tolerant job scheduler for Mesos which handles dependencies and ISO8601 based schedules

简介

Chronos is a replacement for cron. It is a distributed and fault-tolerant scheduler that runs on top of Apache Mesos that can be used for job orchestration. It supports custom Mesos executors as well as the default command executor. Thus by default, Chronos executes sh (on most systems bash) scripts.

Chronos can be used to interact with systems such as Hadoop (incl. EMR), even if the Mesos slaves on which execution happens do not have Hadoop installed. Chronos is also natively able to schedule jobs that run inside Docker containers.

Chronos has a number of advantages over regular cron. It allows you to schedule your jobs using ISO8601 repeating interval notation, which enables more flexibility in job scheduling. Chronos also supports the definition of jobs triggered by the completion of other jobs. It supports arbitrarily long dependency chains.

工作过程

Internally, the Chronos scheduler main loop is quite simple. The pattern is as follows:

Chronos reads all job state from the state store (ZooKeeper)
Jobs are registered within the scheduler and loaded into the job graph for tracking dependencies.
Jobs are separated into a list of those which should be run at the current time (based on the clock of the host machine), and those which should not.
Jobs in the list of jobs to run are queued, and will be launched as soon as a sufficient offer becomes available.
Chronos will sleep until the next job is scheduled to run, and begin again from step 1.

chronos通过zk来维护任务状态，任务会随机分配到一个mesos slave节点运行