Directory Structure

Django Apps

EvalAI is a Django based application, hence it leverages the concept of Django apps to properly namespace the functionalities. One can find all the apps in the apps directory situated in the root folder.

Some important apps along with their main use are:

• Challenges

This app handles all the workflow related to a challenge like creating, modifying and deleting a challenge etc.

• Hosts

This app is responsible for providing functionalities to the challenge hosts/organizers.

• Participants

This app serves users who want to take part in any challenge. It contains code for creating a Participant Team, through which they can participate in any challenge.

• Jobs

One of the most important apps, it is responsible for processing and evaluating the submission made by a participant. It contains code for creating a submission, changing the visibility of the submission and populating the leaderboard for any challenge.

• Web

This app serves some basic functionalities like providing support for contact us or adding a new contributor to the team etc.

• Accounts

As the name indicates, this app deals with storing and managing data related to user accounts.

• Base

A placeholder app which contains the code that is used across various other apps.

Settings

Settings are used across the backend codebase by Django to provide config values on an environment basis. Presently following setting a re present

• dev

Used in development environment

• testing

Used whenever test cases are run

• staging

Used on staging server

• production

Used on production server

URLs

The base URLs for the project are present in evalai/urls.py. This URL includes URLs of various application also which are namespaced by app name. So URLs for challenges app will have its app namespace in the URL as challenges. This actually helps us in separating our API on the basis of apps.

Frontend

The whole code base for the frontend resides in a folder named frontend in the root directory

Scripts

Scripts contain various helper scripts and utilities along with python workers. It contains the following folders

• migration

Contains some of the scripts which are used for one-time migration or formatting of data.

• tools

A folder for storing helper scripts like script to fetch pull request

• workers

One of the main directory, which contains the code for submission worker. Submission worker is a normal python worker which is responsible for processing and evaluating submission of a user. The command to start a worker is:

python scripts/workers/submission_worker.py

Test Suite

All of the code base related to testing resides in tests folder present in the root directory. In this directory, tests are namespace according to the name of the app like tests for challenges app lives in a folder named challenges.

Management Commands

To perform certain actions like seeding the database, we use Django management commands. Since the management commands are common throughout the project, hence they are present in base application directory. At the moment, only one management command seed is present, which is used to populate the database with some random values. The command can be invoked by calling

python manage.py seed --settings=settings.dev