Performance analyzer

This application exemplifies how InPUT simplified setup for benchmark tasks.

The question is: Given a computer and a customizable job (e.g. a web server request). What is the thread pool size that maximizes throughput?

In order to test the throughput, we have to decide several system parameters for the simulation of our simplified scenario:

• The task.
• Amount of tasks to be executed.
• The different pool sizes we test.
• Repetitions of experiment (statistical confidence).

We illustrate how InPUT can externalize all these decisions and make the code reusable as a generic framework without recompilation.

Five versions of the application are available:

1. Plain Java

The least flexible option. All parameters are fixed, and no changes can be made after compilation. Example:

amountTasks = 1000;
task = new SomeJob(100000000, 4000000, new SomeOption());
executions = 100;
poolSize = new int[]{1,2,8,9,10};

2. Java with Properties file import

Properties files allow for external configuration. They restrict the user to the options given at compilation time. They require boilerplate parsing for primitive types, and switch or if-else structures for the string - class mapping. For instance, given a properties file with the following content:

amountTasks=1000
executions=100
poolSize=1,2,8,9,10
task=someJob
someJob.firstValue=100000000
someJob.secondValue=4000000
someJob.option=someOption

this is how the amount of tasks is retrieved:

		amountTasks = Integer.parseInt(poolInvestigation
				.getProperty("amountTasks"));

The Job retrieval requires the creation of a tailored function

task = initTask(properties);

and the function has to consider the options and make error handling explicit, as in:

private Runnable initTask(Properties poolInvestigation) {
		String taskString = poolInvestigation.getProperty("task");
		if (taskString.equals("someJob")) {
			int firstValue = Integer.parseInt(poolInvestigation.getProperty("someJob.firstValue"));
			int secondValue = Integer.parseInt(poolInvestigation.getProperty("someJob.secondValue"));

			String optionString = poolInvestigation.getProperty("someJob.option");
			if (optionString.equals("someOption")) {
				Option option = new SomeOption();
				return new SomeJob(firstValue, secondValue, option);
			}
			throw new IllegalArgumentException(
					"The defined option type is not known by the source code :(");
		}
		throw new IllegalArgumentException(
				"The defined task type is not known by the source code :(");
	}

A rather tedious task to create. Job type or sub-parameters of Job cannot be changed or be extended once the simulation code has been compiled.

3. Java with InPUT

Using InPUT, all decisions can be externalized. The required code

amountTasks = poolInvestigation.getValue("amountTasks");
task = poolInvestigation.getValue("task");
executions = poolInvestigation.getValue("executions");
poolSize = poolInvestigation.getValue("poolSize");

can easily be called in a batch, and decisions can be adjusted in the respective design file:

<NValue id="amountTasks" value="10" />
<NValue id="executions" value="3" />
<NValue id="poolSize">
	<NValue id="1" value="1" />
	<NValue id="2" value="2" />
	<NValue id="3" value="8" />
	<NValue id="4" value="9" />
	<NValue id="5" value="10" />
</NValue>
<SValue id="task" value="someJob">
	<NValue id="firstValue" value="100000000"/>
	<NValue id="secondValue" value="4000000"/>
<SValue id="option" value="some"/>

New Job types can be defined in the respective design space. For instance, a new job type MyJob could be added as in:

<SParam id="task">
	<SChoice id="someJob">
	...
	</SChoice>
	<SChoice id="myJob">
		<NParam type="decimal" id="whateverValue" inclMin="0.43" inclMax="4.5" />
	</SChoice>
</SParam>

and pointing the respective code mapping to the implementation class:

<Mapping id="task.myJob"
		type="you.name.it.MyNewJob" constructor="firstValue secondValue option" />

This allows even for the simulator to mix heterogeneous jobs. Now, by adjusting the design file to consider myJob instead of someJob, the simulator should run the experiments as before, just with another task type. As an alternative, the someJob implementation could simply be substituted in the code mapping.

4. Java with InPUT and result export

By defining the output space, InPUT can export the results, running the following code:

IDesign results = new DesignSpace("performanceSpace.xml").nextEmptyDesign("someId");
results.setValue("performance", runtime);
results.export(new XMLArchiveExporter("performance.xml"));
results.export(new LaTeXFileExporter("performance.tex"));

We are working on a database input/output.

5. Java with InPUT and injection based result export

An extension that does the exact same as the former example, but using the injection based design and parameter references using annotations (and AspectJ).