Home

Exelixi is a distributed framework for running genetic algorithms at scale. The framework is based on Apache Mesos and the code is mostly implemented in Python.

Table of Contents

• Quick Start
• System Dependencies
• Background
• Implementation
• Current Status
• Notes about Distributed Systems
• Acknowledgements

Rationale

Why build yet another framework for this purpose? Apache Hadoop would be quite a poor fit, due to requirements for in-memory iteration. Apache Spark could fit the problem more closely, in terms of iterative tasks. However, task overhead can become high in proportion to tasks being performed ("small file problem"), plus there is a considerable amount of configuration required at scale. Server-side operations and coprocessors in Apache Cassandra or Apache HBase might also provide a good fit for GA processing, but both of those also require lots of configuration. Moreover, many of the features for these more heavyweight frameworks are not needed, plus it helps to have some lightweight messaging available among the shards -- which these existing frameworks tend to lack.

On the one hand, Exelixi provides the basis for a tutorial for building distributed frameworks in Apache Mesos. On the other hand, it provides a general-purpose GA platform that emphasizes scalability and fault tolerance, while leveraging the wealth of available Python analytics packages.

Project Name

Where does the name come from? Exelixi comes from the Greek word ekseliksi meaning progress, or a similar connotation for the more contemporary notion of evolution. It is pronounced as: e'kseliksi