Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more metadata about a CLP and its arguments. #19

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add more metadata about a CLP and its arguments. #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 55 additions & 12 deletions src/main/scala/com/fulcrumgenomics/sopt/Sopt.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
package com.fulcrumgenomics.sopt

import com.fulcrumgenomics.sopt.cmdline.{ClpArgumentDefinitionPrinting, ClpGroup, CommandLineParser, CommandLineProgramParser}
import com.fulcrumgenomics.commons.reflect.ReflectionUtil
import com.fulcrumgenomics.sopt.cmdline.{CommandLineParser, _}
import com.fulcrumgenomics.sopt.util.{MarkDownProcessor, ParsingUtil}
import com.sun.org.apache.xpath.internal.Arg

import scala.collection.mutable.ListBuffer
import scala.reflect.runtime.universe.TypeTag
import scala.reflect.ClassTag

Expand Down Expand Up @@ -56,7 +58,25 @@ object Sopt {
hidden: Boolean,
description: String,
args: Seq[Arg]
) extends MarkDownDescription
) extends MarkDownDescription {
/** Returns a command line string representation of the arguments. If user-specified values are set on the args,
* it will use those, otherwise the defaults if present, otherwise it the type of the argument.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

last part after the comma doesn't read well. Also, why would it output the type if there is no user set value or default value? I would say they should be omitted if no value is present.

*
* @param withDefaults true to include args with defaults, otherwise those with user-set values.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if the doc is wrong or I'm misunderstanding. I think you always use the user set values, and if withDefaults is true, you also include options that have default values but no user set value?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're right to be confused. If I rename this includeArgsWithDefaults:

  • If false, then only include arguments who have values set by the user.
  • If true, also include arguments with default values.

* @return
*/
def commandLine(withDefaults: Boolean = true): String = {
// groupBy.toSeq.sortBy is used to ensure that args with user-defined values come first, then args with default
// values, then non-special args, and finally special args.
val argStrings = args
.filter(_.userValue.nonEmpty || withDefaults)
.groupBy(arg => (arg.userValue.isEmpty, arg.isSpecial))
.toSeq
.sortBy(_._1)
.flatMap(_._2.map(_.toCommandLineString))
if (argStrings.isEmpty) this.name else this.name + " " + argStrings.mkString(" ")
}
}

/**
* Represents information about an argument to a command line program.
Expand All @@ -70,6 +90,9 @@ object Sopt {
* @param defaultValues the seq of default values, as strings
* @param sensitive if true the argument is sensitive and values should not be re-displayed
* @param description the description of the argument
* @param isSpecial true if the argument is "special" (of type [[SpecialArgumentsCollection]], typically help or
* version), otherwise false.
* @param userValue the value(s) set by the user, [[None]] otherwise
*/
case class Arg(name: String,
group: Option[String],
Expand All @@ -79,8 +102,17 @@ object Sopt {
maxValues: Int,
defaultValues: Seq[String],
sensitive: Boolean,
description: String
) extends MarkDownDescription
description: String,
isSpecial: Boolean,
userValue: Option[Seq[String]]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like this. These classes are supposed to be metadata about the class/args. Stuffing values into them is weird and makes then seem more like the internal implementation classes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I don't think userValue is correct, as it could also be the default value.
  2. The use case I have is inspecting the class built from command line parsing, so I want to know the value. For example, I have built a tool from the arguments on the command line, now I want to know for a given argument for that tool, does it have a value and if so, what is it?

) extends MarkDownDescription {
/** Gets the command line string for this argument. Will use the user specified values if present, otherwise the
* defaults if present, and otherwise prints the type.*/
def toCommandLineString: String = userValue match {
case Some(value) => s"--$name ${value.mkString(" ")}"
case None => if (defaultValues.isEmpty) s"--$name <$kind>" else defaultValues.map(v => s"--$name $v").mkString(" ")
}
}

/** Finds classes that extend the given type within the specified packages.
*
Expand All @@ -100,10 +132,20 @@ object Sopt {
* @tparam A the type of the class
* @return a metadata object containing information about the command and it's arguments
*/
def inspect[A](clp: Class[A]): ClpMetadata = {
val parser = new CommandLineProgramParser(clp, includeSpecialArgs=false)
val clpAnn = ParsingUtil.findClpAnnotation(clp).getOrElse(throw new IllegalStateException("No @clp on " + clp.getName))
val args = parser.argumentLookup.ordered.filter(_.annotation.isDefined).map ( a => Arg(
@deprecated(message="Use inspect()", since="0.6.0")
def inspect[A](clp: Class[A])(implicit typeTag: TypeTag[A]): ClpMetadata = inspect[A](parser = None)

/**
* Inspect a command class that is annotated with [[clp]] and [[arg]] annotations.
*
* @param parser optionally a parser to use to lookup the arguments.
* @tparam A the type of the CLP class
* @return a metadata object containing information about the command and it's arguments
*/
def inspect[A](parser: Option[CommandLineProgramParser[A]] = None)(implicit typeTag: TypeTag[A]): ClpMetadata = {
val clp = ReflectionUtil.typeTagToClass[A]
val _parser = parser.getOrElse(new CommandLineProgramParser(clp, includeSpecialArgs=false))
val args = _parser.argumentLookup.ordered.filter(_.annotation.isDefined).map ( a => Arg(
name = a.longName,
group = a.groupName,
flag = a.shortName,
Expand All @@ -112,11 +154,12 @@ object Sopt {
maxValues = if (a.isCollection) a.maxElements else 1,
defaultValues = ClpArgumentDefinitionPrinting.defaultValuesAsSeq(a.defaultValue),
sensitive = a.isSensitive,
description = a.doc
description = a.doc,
isSpecial = a.isSpecial,
userValue = if (a.hasValue && a.isSetByUser) Some(a.toArgs) else None
))

val group = clpAnn.group().newInstance()

val clpAnn = ParsingUtil.findClpAnnotation(clp).getOrElse(throw new IllegalStateException("No @clp on " + clp.getName))
val group = clpAnn.group().newInstance()
ClpMetadata(
name = clp.getSimpleName,
group = Group(name=group.name, description=group.description, rank=group.rank),
Expand Down
54 changes: 40 additions & 14 deletions src/main/scala/com/fulcrumgenomics/sopt/cmdline/ClpReflectiveBuilder.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ private[sopt] class ClpArgument(declaringClass: Class[_],
val optional: Boolean = hidden || isFlag || hasValue || (isCollection && annotation.get.minElements() == 0) || argumentType == classOf[Option[_]]

/** true if the field was set by the user */
private[cmdline] var isSetByUser: Boolean = false // NB: only true when [[setArgument]] is called, vs. this.value =
private[sopt] var isSetByUser: Boolean = false // NB: only true when [[setArgument]] is called, vs. this.value =

lazy val isSpecial: Boolean = annotation.exists(_.special())
lazy val isSensitive: Boolean = annotation.exists(_.sensitive())
Expand Down Expand Up @@ -243,36 +243,62 @@ private[sopt] class ClpArgument(declaringClass: Class[_],
}

/**
* Helper for pretty printing this option.
* Helper for pretty printing the option name and value.
*
* @param value A value this argument was given
* @return a string
*
*/
private def prettyNameValue(value: Any): String = {
private def prettyNameAndValue(value: Any): String = {
Option(value) match {
case Some(_) if isSensitive => f"--$longName ***********"
case Some(Some(optionValue)) => prettyNameValue(optionValue)
case Some(None) => f"--$longName ${ReflectionUtil.SpecialEmptyOrNoneToken}"
case Some(v) => f"--$longName $v"
case _ => ""
case Some(v) => f"--$longName ${prettyValue(v)}"
case _ => ""
}
}

/**
* Helper for pretty printing the option value (no name).
*
* @param value A value this argument was given
* @return a string
*
*/
private def prettyValue(value: Any): String = {
Option(value) match {
case Some(_) if isSensitive => "***********"
case Some(Some(optionValue)) => prettyValue(optionValue)
case Some(None) => s"${ReflectionUtil.SpecialEmptyOrNoneToken}"
case Some(v) => s"$v"
case _ => ""
}
}

/**
* Returns a string representation of this argument and it's value(s) which would be valid if copied and pasted
* back as a command line argument. Will throw an exception if called on an ArgumentDefinition that has not been
* back as a command line argument. Will throw an exception if called on an [[ClpArgument]] that has not been
* set.
*/
def toCommandLineString: String = this.value match {
case Some(v) =>
if (this.isCollection) {
val someCollection = new SomeCollection(v)
if (someCollection.isEmpty) prettyNameValue(None)
else prettyNameValue(someCollection.values.mkString(" "))
if (someCollection.isEmpty) prettyNameAndValue(None)
else prettyNameAndValue(someCollection.values.mkString(" "))
}
else prettyNameAndValue(v)
case None => throw new IllegalStateException("toCommandLineString not allowed on unset argument.")
}

/** Returns the sequence of values for this argument. Will throw an exception if called on an [[ClpArgument]] that has
* not been set.*/
def toArgs: Seq[String] = this.value match {
case Some(v) =>
if (this.isCollection) {
val someCollection = new SomeCollection(v)
if (someCollection.isEmpty) Seq(prettyValue(None))
else someCollection.values.map(prettyValue).toSeq
}
else prettyNameValue(v)
case None =>
throw new IllegalStateException("toCommandLineString not allowed on unset argument.")
else Seq(prettyValue(v))
case None => throw new IllegalStateException("toCommandLineString not allowed on unset argument.")
}
}
32 changes: 22 additions & 10 deletions src/main/scala/com/fulcrumgenomics/sopt/cmdline/CommandLineParser.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ package com.fulcrumgenomics.sopt.cmdline

import com.fulcrumgenomics.commons.CommonsDef._
import com.fulcrumgenomics.commons.reflect.ReflectionUtil
import com.fulcrumgenomics.sopt.Sopt.{CommandSuccess, Failure, Result, SubcommandSuccess}
import com.fulcrumgenomics.sopt.Sopt._
import com.fulcrumgenomics.sopt.{Sopt, clp}
import com.fulcrumgenomics.sopt.util.ParsingUtil._
import com.fulcrumgenomics.sopt.util._
Expand Down Expand Up @@ -147,14 +147,28 @@ class CommandLineParser[SubCommand](val commandLineName: String)
private type SubCommandClass = Class[_ <: SubCommand]
private type SubCommandGroupClass = Class[_ <: ClpGroup]

private var _commandLine: Option[String] = None
private val usagePrinter = new StringBuilder()

private var _commandParser: Option[CommandLineProgramParser[_]] = None

private var _subcommandParser: Option[CommandLineProgramParser[SubCommand]] = None

private def print(s: String) = usagePrinter.append(s).append("\n")

/** The command line with all arguments and values stored after the [[parseSubCommand()]] or
* [[parseCommandAndSubCommand()]] was successful, otherwise [[None]]. */
def commandLine: Option[String] = _commandLine
def commandLine: Option[String] = (_commandParser, _subcommandParser) match {
case (Some(cmdParser), Some(subParser)) => Some(cmdParser.commandLine() + " " + subParser.commandLine())
case (Some(cmdParser), None) => Some(cmdParser.commandLine())
case (None, Some(subParser)) => Some(subParser.commandLine())
case (None, None) => None
}

/** The metadata about the command parser from [[parseCommandAndSubCommand()]], [[None]] otherwise. */
def commandMetadata: Option[ClpMetadata] = _commandParser.map(parser => Sopt.inspect(parser=Some(parser)))

/** The metadata about the sub-command parser from [[parseSubCommand()]] or [[parseCommandAndSubCommand()]], [[None]] otherwise. */
def subcommandMetadata: Option[ClpMetadata] = _subcommandParser.map(parser => Sopt.inspect(parser=Some(parser)))

/** The error message for an unknown sub-command. */
private[cmdline] def unknownSubCommandErrorMessage(command: String, classes: Traversable[SubCommandClass] = Set.empty): String = {
Expand Down Expand Up @@ -299,16 +313,14 @@ class CommandLineParser[SubCommand](val commandLineName: String)
/////////////////////////////////////////////////////////////////////////
// Parse the arguments for the Command Line Program class
/////////////////////////////////////////////////////////////////////////
// FIXME: could not get this to work as an anonymous subclass, so I just extended it.
class ClpParser[SubCommand](targetClass: Class[SubCommand]) extends CommandLineProgramParser[SubCommand](targetClass, withSpecialArgs) {
val clpParser = new CommandLineProgramParser[SubCommand](clazz.asInstanceOf[Class[SubCommand]], withSpecialArgs) {
override protected def standardUsagePreamble: String = {
extraUsage match {
case Some(_) => s"${KBLDRED(targetName)}"
case None => standardSubCommandUsagePreamble(Some(clazz))
}
}
}
val clpParser = new ClpParser(clazz)
clpParser.parseAndBuild(args.drop(1)) match {
case ParseFailure(ex, _) => // Case 5
printExtraUsage(clazz=Some(clazz))
Expand All @@ -325,7 +337,7 @@ class CommandLineParser[SubCommand](val commandLineName: String)
case ParseSuccess() => // Case 8
val clp: SubCommand = clpParser.instance.get
try {
_commandLine = Some(_commandLine.map(_ + " ").getOrElse("") + clpParser.commandLine())
_subcommandParser = Some(clpParser)
CommandSuccess(clp)
}
catch {
Expand Down Expand Up @@ -391,7 +403,7 @@ class CommandLineParser[SubCommand](val commandLineName: String)
// Try parsing and building CommandClass and handle the outcomes
/////////////////////////////////////////////////////////////////////////
mainClassParser.parseAndBuild(args=mainClassArgs) match {
case ParseFailure(ex, remaining) => // Case (1)
case ParseFailure(ex, _) => // Case (1)
print(mainClassParser.usage())
print(wrapError(ex.getMessage))
Failure(() => usagePrinter.toString())
Expand All @@ -407,7 +419,7 @@ class CommandLineParser[SubCommand](val commandLineName: String)
// Get setup, and attempt to ID and load the clp class
/////////////////////////////////////////////////////////////////////////
val mainInstance = mainClassParser.instance.get
this._commandLine = Some(mainClassParser.commandLine())
_commandParser = Some(mainClassParser)

this.parseSubCommand(args=clpArgs, subcommands=subcommands, extraUsage = Some(mainClassParser.usage()), withVersion=false) match {
case f: Failure => f
Expand Down Expand Up @@ -438,7 +450,7 @@ class CommandLineParser[SubCommand](val commandLineName: String)
// if we found one that was similar, then split the args at that point
distances.zipWithIndex.min match {
case (distance, idx) if distance < Integer.MAX_VALUE => args.splitAt(idx)
case (distance, idx) => (args, Seq.empty)
case (_, _) => (args, Seq.empty)
}
}
case idx =>
Expand Down
15 changes: 8 additions & 7 deletions src/main/scala/com/fulcrumgenomics/sopt/cmdline/CommandLineProgramParser.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -295,11 +295,12 @@ class CommandLineProgramParser[T](val targetClass: Class[T], val includeSpecialA
}

/** Gets the command line assuming `parseTasks` has been called */
def commandLine(): String = {
def commandLine(withDefaults: Boolean = true): String = {
val argumentList = this.argumentLookup.ordered.filterNot(_.hidden)
val toolName: String = targetName
val commandLineString = argumentList
.filterNot(_.isSpecial)
.filter(_.isSetByUser || withDefaults)
.groupBy(!_.hasValue) // so that args with values come first
.flatMap { case (_, args) => args.map(_.toCommandLineString) }
.mkString(" ")
Expand All @@ -317,11 +318,11 @@ class CommandLineProgramParser[T](val targetClass: Class[T], val includeSpecialA
try {
args.view.foreach { argumentDefinition =>
val fullName: String = argumentDefinition.longName
val mutextArgumentNames: StringBuilder = new StringBuilder
val mutexArgumentNames: StringBuilder = new StringBuilder

// Validate mutex's
//NB: Make sure to remove duplicates
mutextArgumentNames.append(
mutexArgumentNames.append(
argumentDefinition.mutuallyExclusive
.toList.flatMap(args.forField(_) match {
case Some(m) if m.isSetByUser => Some(m)
Expand All @@ -331,15 +332,15 @@ class CommandLineProgramParser[T](val targetClass: Class[T], val includeSpecialA
.map { _.longName}
.mkString(", ")
)
if (argumentDefinition.isSetByUser && mutextArgumentNames.nonEmpty) {
throw new UserException(s"Argument '$fullName' cannot be used in conjunction with argument(s): ${mutextArgumentNames.toString}")
if (argumentDefinition.isSetByUser && mutexArgumentNames.nonEmpty) {
throw UserException(s"Argument '$fullName' cannot be used in conjunction with argument(s): ${mutexArgumentNames.toString}")
}

if (argumentDefinition.isCollection && !argumentDefinition.optional) {
argumentDefinition.validateCollection()
}
else if (!argumentDefinition.optional && !argumentDefinition.hasValue && mutextArgumentNames.isEmpty) {
throw new UserException(requiredArgumentWithMutexErrorMessage(fullName, argumentDefinition))
else if (!argumentDefinition.optional && !argumentDefinition.hasValue && mutexArgumentNames.isEmpty) {
throw UserException(requiredArgumentWithMutexErrorMessage(fullName, argumentDefinition))
}
}
}
Expand Down
Loading