options.md

Command-Line Options

EvoMaster has several options that can be configured. Those can be set on the command-line when EvoMaster is started.

There are 3 types of options:

• Important: those are the main options that most users will need to set and know about.

• Internal: these are low-level tuning options, which most users do not need to modify. These were mainly introduced when experimenting with different configurations to maximize the performance of EvoMaster.

• Experimental: these are work-in-progress options, for features still under development and testing.

The list of available options can also be displayed by using --help, e.g.:

java -jar evomaster.jar --help

Options might also have constraints, e.g., a numeric value within a defined range, or a string being an URL. In some cases, strings might only be chosen within a specific set of possible values (i.e., an Enum). If any constraint is not satisfied, EvoMaster will fail with an error message.

When used, all options need to be prefixed with a --, e.g., --maxTime.

Important Command-Line Options

Options	Description
maxTime	String. Maximum amount of time allowed for the search. The time is expressed with a string where hours (h), minutes (m) and seconds (s) can be specified, e.g., 1h10m120s and 72m are both valid and equivalent. Each component (i.e., h, m and s) is optional, but at least one must be specified. In other words, if you need to run the search for just 30 seconds, you can write 30s instead of 0h0m30s. The more time is allowed, the better results one can expect. But then of course the test generation will take longer. For how long should EvoMaster be left run? The default 1 minute is just for demonstration. We recommend to run it between 1 and 24 hours, depending on the size and complexity of the tested application. Constraints: regex (\s*)((?=(\S+))(\d+h)?(\d+m)?(\d+s)?)(\s*). Default value: 60s.
outputFolder	String. The path directory of where the generated test classes should be saved to. Default value: src/em.
outputFormat	Enum. Specify in which format the tests should be outputted. If left on DEFAULT, then the value specified in the EvoMaster Driver will be used. But a different value must be chosen if doing Black-Box testing. Valid values: DEFAULT, JAVA_JUNIT_5, JAVA_JUNIT_4, KOTLIN_JUNIT_4, KOTLIN_JUNIT_5, JS_JEST. Default value: DEFAULT.
testSuiteFileName	String. The name of generated file with the test cases, without file type extension. In JVM languages, if the name contains '.', folders will be created to represent the given package structure. Also, in JVM languages, should not use '-' in the file name, as not valid symbol for class identifiers. Constraints: regex [-a-zA-Z$_][-0-9a-zA-Z$_]*(.[-a-zA-Z$_][-0-9a-zA-Z$_]*)*. Default value: EvoMasterTest.
blackBox	Boolean. Use EvoMaster in black-box mode. This does not require an EvoMaster Driver up and running. However, you will need to provide further option to specify how to connect to the SUT. Default value: false.
bbSwaggerUrl	String. When in black-box mode for REST APIs, specify where the Swagger schema can be downloaded from. Constraints: URL. Default value: "".
bbTargetUrl	String. When in black-box mode, specify the URL of where the SUT can be reached. If this is missing, the URL will be inferred from Swagger. Constraints: URL. Default value: "".

Internal Command-Line Options

Options	Description
adaptiveGeneSelectionMethod	Enum. Specify a strategy to select genes for mutation adaptively. Valid values: NONE, AWAY_NOIMPACT, APPROACH_IMPACT, APPROACH_LATEST_IMPACT, APPROACH_LATEST_IMPROVEMENT, BALANCE_IMPACT_NOIMPACT, BALANCE_IMPACT_NOIMPACT_WITH_E, ALL_FIXED_RAND. Default value: APPROACH_IMPACT.
algorithm	Enum. The algorithm used to generate test cases. Valid values: MIO, RANDOM, WTS, MOSA. Default value: MIO.
appendToStatisticsFile	Boolean. Whether should add to an existing statistics file, instead of replacing it. Default value: false.
archiveGeneMutation	Enum. Whether to enable archive-based gene mutation. Valid values: NONE, SPECIFIED, SPECIFIED_WITH_TARGETS, SPECIFIED_WITH_SPECIFIC_TARGETS, SPECIFIED_WITH_TARGETS_DIRECTION, SPECIFIED_WITH_SPECIFIC_TARGETS_DIRECTION, ADAPTIVE. Default value: SPECIFIED_WITH_SPECIFIC_TARGETS.
archiveTargetLimit	Int. Limit of number of individuals per target to keep in the archive. Constraints: min=1.0. Default value: 10.
avoidNonDeterministicLogs	Boolean. At times, we need to run EvoMaster with printed logs that are deterministic. For example, this means avoiding printing out time-stamps. Default value: false.
baseTaintAnalysisProbability	Double. Probability to use input tracking (i.e., a simple base form of taint-analysis) to determine how inputs are used in the SUT. Constraints: probability 0.0-1.0. Default value: 0.9.
bbExperiments	Boolean. Only used when running experiments for black-box mode, where an EvoMaster Driver would be present, and can reset state after each experiment. Default value: false.
bloatControlForSecondaryObjective	Boolean. Whether secondary objectives are less important than test bloat control. Default value: false.
coveredTargetFile	String. Specify a file which saves covered targets info regarding generated test suite. Default value: coveredTargets.txt.
createTests	Boolean. Specify if test classes should be created as output of the tool. Usually, you would put it to 'false' only when debugging EvoMaster itself. Default value: true.
customNaming	Boolean. Enable custom naming and sorting criteria. Default value: true.
d	Double. When weight-based mutation rate is enabled, specify a percentage of calculating mutation rate based on a number of candidate genes to mutate. For instance, d = 1.0 means that the mutation rate fully depends on a number of candidate genes to mutate, and d = 0.0 means that the mutation rate fully depends on weights of candidates genes to mutate. Constraints: probability 0.0-1.0. Default value: 0.8.
e_u1f984	Boolean. QWN0aXZhdGUgdGhlIFVuaWNvcm4gTW9kZQ==. Default value: false.
enableBasicAssertions	Boolean. Generate basic assertions. Basic assertions (comparing the returned object to itself) are added to the code. NOTE: this should not cause any tests to fail. Default value: true.
enableTrackEvaluatedIndividual	Boolean. Whether to enable tracking the history of modifications of the individuals with its fitness values (i.e., evaluated individual) during the search. Note that we enforced that set enableTrackIndividual false when enableTrackEvaluatedIndividual is true since information of individual is part of evaluated individual. Default value: true.
enableWeightBasedMutationRateSelectionForGene	Boolean. Specify whether to enable weight-based mutation selection for selecting genes to mutate for a gene. Default value: true.
endNumberOfMutations	Int. Number of applied mutations on sampled individuals, by the end of the search. Constraints: min=0.0. Default value: 10.
endpointFocus	String. Only for debugging. Concentrate search on only one single REST endpoint. Default value: null.
expandRestIndividuals	Boolean. Enable to expand the genotype of REST individuals based on runtime information missing from Swagger. Default value: true.
exportCoveredTarget	Boolean. Specify whether to export covered targets info. Default value: false.
extraHeuristicsFile	String. Where the extra heuristics file (if any) is going to be written (in CSV format). Default value: extra_heuristics.csv.
extractSqlExecutionInfo	Boolean. Enable extracting SQL execution info. Default value: true.
feedbackDirectedSampling	Enum. Specify whether when we sample from archive we do look at the most promising targets for which we have had a recent improvement. Valid values: NONE, LAST, FOCUSED_QUICKEST. Default value: LAST.
focusedSearchActivationTime	Double. The percentage of passed search before starting a more focused, less exploratory one. Constraints: probability 0.0-1.0. Default value: 0.5.
geneMutationStrategy	Enum. Strategy used to define the mutation probability. Valid values: ONE_OVER_N, ONE_OVER_N_BIASED_SQL. Default value: ONE_OVER_N_BIASED_SQL.
geneWeightBasedOnImpactsBy	Enum. Specify a strategy to calculate a weight of a gene based on impacts. Valid values: SORT_COUNTER, SORT_RATIO, COUNTER, RATIO. Default value: RATIO.
generateSqlDataWithSearch	Boolean. Enable EvoMaster to generate SQL data with direct accesses to the database. Use a search algorithm. Default value: true.
heuristicsForSQL	Boolean. Tracking of SQL commands to improve test generation. Default value: true.
jsControllerPath	String. When generating tests in JavaScript, there is the need to know where the driver is located in respect to the generated tests. Default value: ./app-driver.js.
maxActionEvaluations	Int. Maximum number of action evaluations for the search. A fitness evaluation can be composed of 1 or more actions, like for example REST calls or SQL setups. The more actions are allowed, the better results one can expect. But then of course the test generation will take longer. Only applicable depending on the stopping criterion. Constraints: min=1.0. Default value: 1000.
maxResponseByteSize	Int. Maximum size (in bytes) that EM handles response payloads in the HTTP responses. If larger than that, a response will not be stored internally in EM during the test generation. This is needed to avoid running out of memory. Default value: 1000000.
maxSearchSuiteSize	Int. Define the maximum number of tests in a suite in the search algorithms that evolve whole suites, e.g. WTS. Constraints: min=1.0. Default value: 50.
maxSqlInitActionsPerMissingData	Int. When generating SQL data, how many new rows (max) to generate for each specific SQL Select. Constraints: min=1.0. Default value: 5.
maxTestSize	Int. Max number of 'actions' (e.g., RESTful calls or SQL commands) that can be done in a single test. Constraints: min=1.0. Default value: 10.
maxTimeInSeconds	Int. Maximum number of seconds allowed for the search. The more time is allowed, the better results one can expect. But then of course the test generation will take longer. Only applicable depending on the stopping criterion. If this value is 0, the setting 'maxTime' will be used instead. Constraints: min=0.0. Default value: 0.
maxlengthOfHistoryForAGM	Int. Specify a maximum length of history when applying archive-based gene mutation. Default value: 10.
minimumSizeControl	Int. Specify minimum size when bloatControlForSecondaryObjective. Constraints: min=0.0. Default value: 2.
populationSize	Int. Define the population size in the search algorithms that use populations (e.g., Genetic Algorithms, but not MIO). Constraints: min=1.0. Default value: 30.
probOfArchiveMutation	Double. Specify a probability to enable archive-based mutation. Constraints: probability 0.0-1.0. Default value: 0.5.
probOfRandomSampling	Double. Probability of sampling a new individual at random. Constraints: probability 0.0-1.0. Default value: 0.5.
probOfSmartSampling	Double. When sampling new test cases to evaluate, probability of using some smart strategy instead of plain random. Constraints: probability 0.0-1.0. Default value: 0.5.
problemType	Enum. The type of SUT we want to generate tests for, e.g., a RESTful API. Valid values: REST. Experimental values: WEB. Default value: REST.
secondaryObjectiveStrategy	Enum. Strategy used to handle the extra heuristics in the secondary objectives. Valid values: AVG_DISTANCE, AVG_DISTANCE_SAME_N_ACTIONS, BEST_MIN. Default value: AVG_DISTANCE_SAME_N_ACTIONS.
seed	Long. The seed for the random generator used during the search. A negative value means the CPU clock time will be rather used as seed. Default value: -1.
showProgress	Boolean. Whether to print how much search done so far. Default value: true.
snapshotInterval	Double. If positive, check how often, in percentage % of the budget, to collect statistics snapshots. For example, every 5% of the time. Constraints: max=50.0. Default value: -1.0.
snapshotStatisticsFile	String. Where the snapshot file (if any) is going to be written (in CSV format). Default value: snapshot.csv.
specializeSQLGeneSelection	Boolean. Whether to specialize sql gene selection to mutation. Default value: true.
startNumberOfMutations	Int. Number of applied mutations on sampled individuals, at the start of the search. Constraints: min=0.0. Default value: 1.
statisticsColumnId	String. An id that will be part as a column of the statistics file (if any is generated). Default value: -.
statisticsFile	String. Where the statistics file (if any) is going to be written (in CSV format). Default value: statistics.csv.
stoppingCriterion	Enum. Stopping criterion for the search. Valid values: TIME, FITNESS_EVALUATIONS. Default value: TIME.
structureMutationProbability	Double. Probability of applying a mutation that can change the structure of a test. Constraints: probability 0.0-1.0. Default value: 0.5.
sutControllerHost	String. Host name or IP address of where the SUT REST controller is listening on. Default value: localhost.
sutControllerPort	Int. TCP port of where the SUT REST controller is listening on. Constraints: min=0.0, max=65535.0. Default value: 40100.
tournamentSize	Int. Number of elements to consider in a Tournament Selection (if any is used in the search algorithm). Constraints: min=1.0. Default value: 10.
useMethodReplacement	Boolean. Apply method replacement heuristics to smooth the search landscape. Default value: true.
useNonIntegerReplacement	Boolean. Apply non-integer numeric comparison heuristics to smooth the search landscape. Default value: true.
useTimeInFeedbackSampling	Boolean. Whether to use timestamp info on the execution time of the tests for sampling (e.g., to reward the quickest ones). Default value: true.
weightBasedMutationRate	Boolean. Whether to enable a weight-based mutation rate. Default value: true.
writeExtraHeuristicsFile	Boolean. Whether we should collect data on the extra heuristics. Only needed for experiments. Default value: false.
writeStatistics	Boolean. Whether or not writing statistics of the search process. This is only needed when running experiments with different parameter settings. Default value: false.
xoverProbability	Double. Probability of applying crossover operation (if any is used in the search algorithm). Constraints: probability 0.0-1.0. Default value: 0.7.

Experimental Command-Line Options

Options	Description
S1dR	Double. Specify a probability to apply S1dR when resource sampling strategy is 'Customized'. Constraints: probability 0.0-1.0. Default value: 0.25.
S1iR	Double. Specify a probability to apply S1iR when resource sampling strategy is 'Customized'. Constraints: probability 0.0-1.0. Default value: 0.25.
S2dR	Double. Specify a probability to apply S2dR when resource sampling strategy is 'Customized'. Constraints: probability 0.0-1.0. Default value: 0.25.
SMdR	Double. Specify a probability to apply SMdR when resource sampling strategy is 'Customized'. Constraints: probability 0.0-1.0. Default value: 0.25.
abstractInitializationGeneToMutate	Boolean. During mutation, whether to abstract genes for repeated SQL actions. Default value: false.
archiveAfterMutationFile	String. Specify a path to save archive after each mutation during search, only useful for debugging. Default value: archive.csv.
coveredTargetSortedBy	Enum. Specify a format to organize the covered targets by the search. Valid values: NAME, TEST. Default value: NAME.
dependencyFile	String. Specify a file that saves derived dependencies. Default value: dependencies.csv.
disableStructureMutationDuringFocusSearch	Boolean. Specify whether to disable structure mutation during focus search. Default value: false.
doCollectImpact	Boolean. Specify whether to collect impact info that provides an option to enable of collecting impact info when archive-based gene selection is disable. Default value: false.
doesApplyNameMatching	Boolean. Whether to apply text/name analysis with natural language parser to derive relationships between name entities, e.g., a resource identifier with a name of table. Default value: false.
enableProcessMonitor	Boolean. Whether or not enable a search process monitor for archiving evaluated individuals and Archive regarding an evaluation of search. This is only needed when running experiments with different parameter settings. Default value: false.
enableTrackIndividual	Boolean. Whether to enable tracking the history of modifications of the individuals during the search. Default value: false.
errorTextEpsilon	Double. The Distance Metric Error Text may use several values for epsilon.During experimentation, it may be useful to adjust these values. Epsilon describes the size of the neighbourhood used for clustering, so may result in different clustering results.Epsilon should be between 0.0 and 1.0. If the value is outside of that range, epsilon will use the default of 0.8. Constraints: min=0.0, max=1.0. Default value: 0.8.
exceedTargetsFile	String. Specify a path to save all not covered targets when the number is more than 100. Default value: exceedTargets.txt.
executiveSummary	Boolean. Generate an executive summary, containing an example of each category of potential fault found. Default value: false.
expectationsActive	Boolean. Enable Expectation Generation. If enabled, expectations will be generated. A variable called expectationsMasterSwitch is added to the test suite, with a default value of false. If set to true, an expectation that fails will cause the test case containing it to fail. Default value: false.
exportDependencies	Boolean. Specify whether to export derived dependencies among resources. Default value: false.
exportImpacts	Boolean. Specify whether to export derived impacts among genes. Default value: false.
generateSqlDataWithDSE	Boolean. Enable EvoMaster to generate SQL data with direct accesses to the database. Use Dynamic Symbolic Execution. Default value: false.
impactAfterMutationFile	String. Specify a path to save collected impact info after each mutation during search, only useful for debugging. Default value: impactSnapshot.csv.
impactFile	String. Specify a path to save derived genes. Default value: impact.csv.
lastLineEpsilon	Double. The Distance Metric Last Line may use several values for epsilon.During experimentation, it may be useful to adjust these values. Epsilon describes the size of the neighbourhood used for clustering, so may result in different clustering results.Epsilon should be between 0.0 and 1.0. If the value is outside of that range, epsilon will use the default of 0.8. Constraints: min=0.0, max=1.0. Default value: 0.8.
maxLengthOfTraces	Int. Specify a maxLength of tracking when enableTrackIndividual or enableTrackEvaluatedIndividual is true. Note that the value should be specified with a non-negative number or -1 (for tracking all history). Constraints: min=-1.0. Default value: 10.
maxMutationRate	Double. Specify a maximum mutation rate when enabling 'adaptiveMutationRate'. Constraints: probability 0.0-1.0. Default value: 0.9.
minRowOfTable	Int. Specify a minimal number of rows in a table that enables selection (i.e., SELECT sql) to prepare resources for REST Action. In other word, if the number is less than the specified, insertion is always applied. Constraints: min=0.0. Default value: 10.
mutatedGeneFile	String. Specify a path to save mutation details which is useful for debugging mutation. Default value: mutatedGeneInfo.csv.
mutationTargetsSelectionStrategy	Enum. Specify a strategy to select targets for evaluating mutation. Valid values: FIRST_NOT_COVERED_TARGET, EXPANDED_UPDATED_NOT_COVERED_TARGET, UPDATED_NOT_COVERED_TARGET. Default value: FIRST_NOT_COVERED_TARGET.
probOfApplySQLActionToCreateResources	Double. Specify a probability to apply SQL actions for preparing resources for REST Action. Constraints: probability 0.0-1.0. Default value: 0.0.
probOfEnablingResourceDependencyHeuristics	Double. Specify whether to enable resource dependency heuristics, i.e, probOfEnablingResourceDependencyHeuristics > 0.0. Note that the option is available to be enabled only if resource-based smart sampling is enable. This option has an effect on sampling multiple resources and mutating a structure of an individual. Constraints: probability 0.0-1.0. Default value: 0.0.
probOfSelectFromDatabase	Double. Specify a probability that enables selection (i.e., SELECT sql) of data from database instead of insertion (i.e., INSERT sql) for preparing resources for REST actions. Constraints: probability 0.0-1.0. Default value: 0.1.
processFiles	String. Specify a folder to save results when a search monitor is enabled. Default value: process_data.
processInterval	Int. Specify how often to save results when a search monitor is enabled. Default value: 100.
recordExceededTargets	Boolean. Whether to record targets when the number is more than 100. Default value: false.
resourceSampleStrategy	Enum. Specify whether to enable resource-based strategy to sample an individual during search. Note that resource-based sampling is only applicable for REST problem with MIO algorithm. Valid values: NONE, Customized, EqualProbability, Actions, TimeBudgets, Archive, ConArchive. Default value: NONE.
saveArchiveAfterMutation	Boolean. Whether to save archive info after each of mutation, which is typically useful for debugging mutation and archive. Default value: false.
saveImpactAfterMutation	Boolean. Whether to save impact info after each of mutation, which is typically useful debugging impact driven solutions and mutation. Default value: false.
saveMutationInfo	Boolean. Whether to save mutated gene info, which is typically used for debugging mutation. Default value: false.
seedTestCases	Boolean. Whether to seed EvoMaster with some initial test cases. These test cases will be used and evolved throughout the search process. Default value: false.
seedTestCasesFormat	Enum. Format of the test cases seeded to EvoMaster. Valid values: POSTMAN. Default value: POSTMAN.
seedTestCasesPath	String. File path where the seeded test cases are located. Default value: postman.postman_collection.json.
startingPerOfGenesToMutate	Double. Specify a starting percentage of genes of an individual to mutate. Constraints: probability 0.0-1.0. Default value: 0.5.
testSuiteSplitType	Enum. Instead of generating a single test file, it could be split in several files, according to different strategies. Valid values: NONE, CLUSTER, SUMMARY_ONLY, CODE. Default value: NONE.