| layout | permalink | challenge-id | status | sidenav | card-image | agency-logo | challenge-title | tagline | agency | partner-agencies-federal | partners-non-federal | external-url | total-prize-offered-cash | type-of-challenge | submission-start | submission-end | submission-link | prize | fiscal-year | legal-authority | challenge-manager | challenge-manager-email | description | prizes | rules | judging | how-to-enter |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
front-matter-data |
/challenge/statistical-software-for-healthcare-oversight/ |
857 |
closed |
true |
/assets/images/cards/seal-clear-600.png |
hhs.png |
Statistical Software for Healthcare Oversight |
Building a simple statistical software tool to aid in effective healthcare oversight. |
Department of Health and Human Services |
$34,000 |
Software and apps |
09/28/2016 12:00 AM |
05/15/2017 05:00 PM |
true |
FY16 |
America COMPETES Act |
Jared Smith |
<p><strong>Results:</strong> We are pleased to announce the winners of the Statistical Software for Healthcare Oversight Challenge. The grand prize winner and the two finalists were able to create new versions of RAT-STATS that were 508 compliant, replicated the four required RAT-STATS modules, and met all other competition requirements. The Grand Prize winner was selected from the three initial finalists by a panel of 12 users. We have also included additional acknowledgements for the teams that provided the best and the most unique graphical user interfaces (these results have also been posted on the OIG website at <a href="https://oig.hhs.gov/compliance/rat-stats/prize/">https://oig.hhs.gov/compliance/rat-stats/prize/</a>).</p> <p><strong>Grand Prize Winner and Finalist: Doug Brown (Team Catalyst) </strong></p> <p>Overall, Team Catalyst's entry delivered the most intuitive user experience. The entry maintained the familiarity of the original RAT-STATS design while still finding room to add several new and useful features. For example, the entry included custom confidence intervals which allowed users to calculate confidence intervals at other than the standard 80, 90, or 95 percent confidence levels. The entry went beyond the 4 modules required by the contest and included functions for stratified attribute samples and for generating sets of random numbers. As with the other finalists, the Team Catalyst entry was able to replicate RAT-STATS on 60 test cases and met the key requirements for 508 compliance.</p> <p><strong>Finalist: Murray Miron </strong></p> <p>Among a number of notable new features, Murray's entry included an integrated help system that connected relevant help information with each input element within the software. The accessibility of the information greatly reduced the need for users to refer back to a separate help screen or manual. Murray's entry also included the fastest algorithm for calculating exact confidence intervals for large attribute samples. In addition, the entry went beyond the four required modules to also include a function for determining the sample size of a design given an estimated error rate. As with the other finalists, Murray's entry was able to replicate RAT-STATS on 60 test cases and met the key requirements for 508 compliance.</p> <p><strong>Finalist: Dave Kaniss and Tim Kaniss (Team ENGRdynamics) </strong></p> <p>Team ENGRdyanmics' simplified the original RAT-STATS Design by combining the stratified and unrestricted variable appraisal functions into a single module. The combined module had an elegant design and was still very effective in accounting for the differences between the stratified and unrestricted analyses. The team also included a number of additional useful new features such as the ability to import files by dragging them into the target input field. As with the other finalists, the Team ENGRdynamics' entry was able to replicate RAT-STATS on 60 test cases and met the key requirements for 508 compliance.</p> <p><strong>Best Graphical User Interface - Corey Berry and Aja Berry (Team CBTek)</strong></p> <p>The graphical user interface for CBTek's entry looked great, met 508 compliance standards, and allowed for easy access to all key program functions and operations. The interface had the added advantage that it was customizable to fit the needs of the individual user. Of final note, the team's splash screen and program icon were the best of all the entries submitted.</p> <p><strong>Graphic User Interface Honorable Mention – Sudeep Shouche and Premraj Narkhede (Team Dolcera)</strong></p> <p>Team Dolcera provided a unique entry which split several of the RAT-STATS modules into a series of steps. The component nature of the design simplified the input and made the function requirements easier to follow for new users.</p> <p><strong>Contest Rules and Description</strong></p> <p><strong>Important Note:</strong> Once five finalists have been identified, no further entries from teams not previously identified as finalists will be reviewed even if they are submitted prior to the May 15, 2017, 5:00 p.m. EST deadline. Moreover, the contest will end on May 15, 2017, at 5:00 p.m. EST even if fewer than five finalists have been selected.</p> <p>Finalists may update their entries without losing their finalist designation. Updates from the finalists will be accepted until 5:00 p.m. EST on the fourteenth day after the fifth finalist has been identified or May 15, 2017, 5:00 p.m. EST, whichever comes first. The newest entry from each team will be used for all judging purposes unless otherwise requested by the team.</p> <p><strong>Current Status:</strong> Status as of 3:00 PM EST May 31 2017: Three finalists.</p> <p><strong>Registration:</strong> To register for the competition, create an account on the challenge.gov website and send an email with the name(s) of your team member(s) to [email protected].</p> <p><strong>Background:</strong> Each year HHS handles hundreds of millions of Medicare and Medicaid claims valued at over a trillion dollars. Due to the high volume of claims, statistical sampling provides a critical tool to ensure effective oversight of these expenditures. Sampling is used by the providers in their own efforts to monitor their performance and by the various organizations within HHS. There are a wide range of different software tools for performing statistical analysis. RAT-STATS has a unique niche in that it provides a straightforward tool for individuals who need a simple but robust method for selecting and analyzing statistical samples. The RAT-STATS software was originally created in 1978 and has gone through several upgrades since that time. Unlike a full statistical package that attempts to answer all types of questions for a wide range of users, RAT-STATS serves as a streamlined solution to handle the specific task of developing valid statistical samples and estimates within the healthcare oversight setting.</p> <p>For example, an OIG investigator may pull a simple random sample in order to estimate damages for a provider suspected of fraud. RAT-STATS generates valid pseudo-random numbers and outputs all of the information needed to replicate the sample. Once the investigator finishes reviewing the sample, he or she can then enter the results into RAT-STATS to get the final statistical estimate. While the investigator may need some basic training in statistics, he or she would not need the same level of expertise as would be required to navigate the many options available in a full service statistical or data analysis package.</p> <p>The objective of the current challenge is to develop the foundation for an upgraded version of RAT-STATS. The current version of RAT-STATS is well validated; however, its user interface can be difficult to navigate and the underlying code makes the software costly to update. OIG needs a new, modern version of the software that is easy to use and can be extended in a cost effective manner. In addition, the new version of the software must be 508 compliant.</p> <p>The current version of the RAT-STATS software can be found at <a href="https://web.archive.org/web/20180501000101/http://www.oig.hhs.gov/compliance/rat-stats/index.asp">http://www.oig.hhs.gov/compliance/rat-stats/index.asp</a>.</p> |
<p><strong>Grand Prize: $25,000.00</strong></p> <p>In addition to the cash payment, winners will be listed on the new version of the RAT-STATS software (once released), on the RAT-STATS download website, and in the RAT-STATS User Manual.</p> <p><strong>Finalist: $3,000.00</strong></p> <p><strong>Finalist: $3,000.00</strong></p> <p><strong>Finalist: $3,000.00</strong></p> |
<p>In order to take part in this Challenge participants must create a software package that replicates the operation of four of the functions of the original RAT-STATS software: (1) Single Stage of Random Numbers; (2) Unrestricted Attribute Appraisal; (3) Unrestricted Variable Appraisal; and (4) Stratified Variable Appraisal. The steps required to add additional statistical modules to the software must be reasonable and fully documented, and the submission must contain all of the required elements (refer to the Submissions page for a complete list).</p> <p>In addition, participants must agree to the following terms and conditions.</p> <p>Teams of one or more members can participate in this Challenge. There is no maximum team size. Each team must have a captain, and each individual may only be part of a single team. Individual team members and team captains must register in accordance with the Registration Process section below. The role of the team captain is to serve as the corresponding participant with OIG about the Challenge and to submit the team’s Challenge entry. While OIG will notify all registered Challenge participants by email of any amendments to the Challenge, the team captain is expected to keep the team members informed about matters germane to the Challenge.</p> <p>(1) To be eligible to win the Challenge prize, each participant (individual or entity) must —</p> <ul> <li>a. register to participate in the Challenge under the rules promulgated by OIG as published in this Notice;</li> <li>b. have complied with all the requirements under this section;</li> <li>c. in the case of a private entity, be incorporated in and maintain a primary place of business in the United States or, in the case of an individual, be a citizen or permanent resident of the United States;</li> <li>d. not be a Federal entity or Federal employee acting within the scope of their employment;</li> <li>e. not be an employee of OIG, a judge of the Challenge, or any other party involved with the design, production, execution, or distribution of the Challenge or the immediate family of such a party (i.e., spouse, parent, step-parent, child, or stepchild); and</li> <li>f. be at least 18 years old at the time of submission.</li> </ul> <p>(2) Federal contractors may not use Federal funds from a contract to develop their Challenge submissions or to fund efforts in support of their Challenge submission.</p> <p>(3) Federal grantees may not use Federal funds to develop COMPETES Act Challenge applications unless consistent with the purpose of their grant award.</p> <p>(4) An individual or entity shall not be deemed ineligible because the individual or entity used Federal facilities or consulted with Federal employees during a competition if the facilities and employees are made available to all individuals and entities participating in the competition on an equitable basis.</p> <p>(5) By participating in this Challenge, each individual (whether competing singly or in a group) and entity agrees to assume any and all risks and waive claims against the Federal Government and its related entities (as defined in the COMPETES Act), except in the case of willful misconduct, for any injury, death, damage, or loss of property, revenue, or profits, whether direct, indirect, or consequential, arising from participation in this Challenge, whether the injury, death, damage, or loss arises through negligence or otherwise.</p> <p>(6) No individual (whether competing singly or in a group) or entity participating in the Challenge is required to obtain liability insurance or demonstrate financial responsibility in order to participate in this Challenge.</p> <p>(8) By participating in this Challenge, each individual (whether participating singly or in a group) and entity grants to OIG, in any existing or inchoate copyright or patent rights owned by the individual or entity, an irrevocable, paid-up, royalty-free, nonexclusive worldwide license to use, reproduce, post, link to, share, and display publicly on the Web the submission, except for source code. This license includes without limitation posting or linking to the submission, except for source code, on OIG’s public facing website (http://www.oig.hhs.gov/compliance/rat-stats/index.asp). In developing its future software systems, OIG may include algorithms and software from Challenge entries and may consult with individuals or teams that submitted entries. Thus, the license also permits OIG to develop a future software system, independently or with others, using any algorithms or software from Challenge entries, including those obtained from other Challenges or solicitations, and OIG may freely use, reproduce, modify, and distribute the resulting future software system without restriction. Each participant will retain all other intellectual property rights in their submissions, as applicable.</p> <p>(9) OIG reserves the right, at its sole discretion, to (a) cancel, suspend, or modify the Challenge through amendment to this Federal Register notice, and/or (b) not award any prizes if no entries meet the stated requirements. In addition, OIG reserves the right to disqualify any Challenge participants or entries in instances where cheating or other misconduct is identified.</p> <p>(10) Each individual (whether participating singly or in a group) or entity agrees to follow all applicable Federal, State, and local laws, regulations, and policies.</p> <p>(11) Each individual (whether participating singly or in a group) and entity participating in this Challenge must comply with all terms and conditions of these rules, and participation in this Challenge constitutes each such participant’s full and unconditional agreement to abide by these rules. Winning is contingent upon fulfilling all requirements herein.</p> <p>(12) Each individual (whether participating singly or in a group) and entity grants to OIG and its contractors assisting OIG with this Challenge the right to review the submission, study the algorithms and the code, and run the software on other sets of images.</p> <p>(13) Submissions must not infringe upon any copyright, patent, trade secrets, or any other rights of any third party. Each individual (whether participating singly or in a group) or entity warrants that she or the team is the sole author and owner of any copyrightable work that the submission comprises, that the submission is wholly original with the participant or is an improved version of an existing work that the participant has sufficient rights to use and improve. In addition, the submission must not trigger any reporting or royalty obligation to any third party. A submission must not include proprietary, classified, confidential, or sensitive information.</p> <p>(14) The licenses for any code used as part of a submission must be compatible with each other and must allow OIG to distribute and modify the software both within and outside the agency without incurring any reporting or royalty obligations to any third party.</p> <p>(15) The submission must not contain malicious code such as viruses, timebombs, cancelbots, worms, trojan horses, or other potentially harmful programs or other material or information.</p> <p>(16) The submission must be unique and must not represent a modification of a previous submission from another team.</p> <p>(17) Submitted software must be fully functional and operate correctly on Microsoft Windows systems configured in accordance with the applicable United States Government Configuration Baseline (USGCB) and applicable configurations (https://usgcb.nist.gov/). The group policy settings associated with this configuration are also available on the NIST website (https://usgcb.nist.gov/usgcb/microsoft_content.html).</p> <p>(18) Submitted software must be a stand-alone product that is designed for end users to run in the standard user context without requiring elevated administrative privileges.</p> <p>(19) Submitted software must not require or make use of any network capabilities.</p> <p>(20) Submitted software must be section 508 compliant. For example, it must be possible to run all functions using a keyboard, the software must include a well-defined indicator of current program focus, any results provided in image format must also be available in text, and color coding shall not be used as the only means of conveying information. For more details refer to section 36 CFR § 1194.21</p> |
<p><strong>Finalist Criteria</strong></p> <p>The first five eligible submissions that comply with the rules of this Federal Register Notice (see also the Rules page of this website), follow the detailed submission instructions, are complete, are able to fully replicate RAT-STATS on 60 target cases, and include a description of a reasonable method for adding additional modules will each be declared a finalist. Once five finalists have been identified, no further entries from teams not previously identified as finalists will be reviewed even if they are submitted prior to the May 15, 2017, 5:00 p.m. EST deadline. Moreover, the contest will end on May 15, 2017, at 5:00 p.m. EST even if fewer than five finalists have been selected. <br /><br />Finalists may update their entries without losing their finalist designation. Updates from the finalists will be accepted until 5:00 p.m. EST on the fourteenth day after the fifth finalist has been identified or May 15, 2017, 5:00 p.m. EST, whichever comes first. The newest entry from each team will be used for all judging purposes unless otherwise requested by the team. </p> <p><strong>Grand Prize Criteria</strong></p> <p>Twelve OIG employees who use the current version of the RAT-STATS software will review the submissions from the five finalists and vote on which package they would prefer to serve as the RAT-STATS replacement. The package with the most votes will be the winner of the Grand Prize. Ties will be broken by the Office of Inspector General, Office of Audit Services, Director of Quantitative Methods. The instructions to the judges will be as follows, "Please review the operation of the following software packages and select one that you would prefer to use in the future for selecting and analyzing your statistical samples." Critically, the instructions do not ask which software is most like RAT-STATS, but rather which software they would prefer to use. As a result, during the review for the Grand Prize, judges will consider features even if they were not part of the original RAT-STATS package.</p> |
<p>Submissions must be entered electronically using the challenge.gov website and must contain a ZIP file with each of the following elements.</p> <p>(1) Executable software that replicates the 4 target RAT-STATS functions: Single Stage of Random Numbers, Unrestricted Attribute Appraisal, Unrestricted Variable Appraisal, and the Stratified Variable Appraisal.</p> <p>(2) Source code for the executable that is both human- and machine-readable. The source code can be written in any programming language(s). The source code must be commented sufficiently so that another user can understand the underlying operation of the code.</p> <p>(3) A text file written in English documenting and explaining any non-cosmetic differences between the submission and the original RAT-STATS software.</p> <p>(4) A text file written in English summarizing the changes required to add additional RAT-STATS functions to the submission.</p> <p>(5) A text file written in English documenting all software licenses associated with the source code used as part of the project. The text file must describe the nature of each individual license and their overall compatibility.</p> <p>(6) A one-page text file written in English that contains the following:</p> <ol> <li>Names and email addresses of the team captain and all team members</li> <li>A five or more character identifier for the entry that is used as a prefix in the names of all of the team’s submitted files</li> <li>A brief description of the submission</li> </ol> |
Results: We are pleased to announce the winners of the Statistical Software for Healthcare Oversight Challenge. The grand prize winner and the two finalists were able to create new versions of RAT-STATS that were 508 compliant, replicated the four required RAT-STATS modules, and met all other competition requirements. The Grand Prize winner was selected from the three initial finalists by a panel of 12 users. We have also included additional acknowledgements for the teams that provided the best and the most unique graphical user interfaces (these results have also been posted on the OIG website at https://oig.hhs.gov/compliance/rat-stats/prize/).
Grand Prize Winner and Finalist: Doug Brown (Team Catalyst)
Overall, Team Catalyst's entry delivered the most intuitive user experience. The entry maintained the familiarity of the original RAT-STATS design while still finding room to add several new and useful features. For example, the entry included custom confidence intervals which allowed users to calculate confidence intervals at other than the standard 80, 90, or 95 percent confidence levels. The entry went beyond the 4 modules required by the contest and included functions for stratified attribute samples and for generating sets of random numbers. As with the other finalists, the Team Catalyst entry was able to replicate RAT-STATS on 60 test cases and met the key requirements for 508 compliance.
Finalist: Murray Miron
Among a number of notable new features, Murray's entry included an integrated help system that connected relevant help information with each input element within the software. The accessibility of the information greatly reduced the need for users to refer back to a separate help screen or manual. Murray's entry also included the fastest algorithm for calculating exact confidence intervals for large attribute samples. In addition, the entry went beyond the four required modules to also include a function for determining the sample size of a design given an estimated error rate. As with the other finalists, Murray's entry was able to replicate RAT-STATS on 60 test cases and met the key requirements for 508 compliance.
Finalist: Dave Kaniss and Tim Kaniss (Team ENGRdynamics)
Team ENGRdyanmics' simplified the original RAT-STATS Design by combining the stratified and unrestricted variable appraisal functions into a single module. The combined module had an elegant design and was still very effective in accounting for the differences between the stratified and unrestricted analyses. The team also included a number of additional useful new features such as the ability to import files by dragging them into the target input field. As with the other finalists, the Team ENGRdynamics' entry was able to replicate RAT-STATS on 60 test cases and met the key requirements for 508 compliance.
Best Graphical User Interface - Corey Berry and Aja Berry (Team CBTek)
The graphical user interface for CBTek's entry looked great, met 508 compliance standards, and allowed for easy access to all key program functions and operations. The interface had the added advantage that it was customizable to fit the needs of the individual user. Of final note, the team's splash screen and program icon were the best of all the entries submitted.
Graphic User Interface Honorable Mention – Sudeep Shouche and Premraj Narkhede (Team Dolcera)
Team Dolcera provided a unique entry which split several of the RAT-STATS modules into a series of steps. The component nature of the design simplified the input and made the function requirements easier to follow for new users.
Contest Rules and Description
Important Note: Once five finalists have been identified, no further entries from teams not previously identified as finalists will be reviewed even if they are submitted prior to the May 15, 2017, 5:00 p.m. EST deadline. Moreover, the contest will end on May 15, 2017, at 5:00 p.m. EST even if fewer than five finalists have been selected.
Finalists may update their entries without losing their finalist designation. Updates from the finalists will be accepted until 5:00 p.m. EST on the fourteenth day after the fifth finalist has been identified or May 15, 2017, 5:00 p.m. EST, whichever comes first. The newest entry from each team will be used for all judging purposes unless otherwise requested by the team.
Current Status: Status as of 3:00 PM EST May 31 2017: Three finalists.
Registration: To register for the competition, create an account on the challenge.gov website and send an email with the name(s) of your team member(s) to [email protected].
Background: Each year HHS handles hundreds of millions of Medicare and Medicaid claims valued at over a trillion dollars. Due to the high volume of claims, statistical sampling provides a critical tool to ensure effective oversight of these expenditures. Sampling is used by the providers in their own efforts to monitor their performance and by the various organizations within HHS. There are a wide range of different software tools for performing statistical analysis. RAT-STATS has a unique niche in that it provides a straightforward tool for individuals who need a simple but robust method for selecting and analyzing statistical samples. The RAT-STATS software was originally created in 1978 and has gone through several upgrades since that time. Unlike a full statistical package that attempts to answer all types of questions for a wide range of users, RAT-STATS serves as a streamlined solution to handle the specific task of developing valid statistical samples and estimates within the healthcare oversight setting.
For example, an OIG investigator may pull a simple random sample in order to estimate damages for a provider suspected of fraud. RAT-STATS generates valid pseudo-random numbers and outputs all of the information needed to replicate the sample. Once the investigator finishes reviewing the sample, he or she can then enter the results into RAT-STATS to get the final statistical estimate. While the investigator may need some basic training in statistics, he or she would not need the same level of expertise as would be required to navigate the many options available in a full service statistical or data analysis package.
The objective of the current challenge is to develop the foundation for an upgraded version of RAT-STATS. The current version of RAT-STATS is well validated; however, its user interface can be difficult to navigate and the underlying code makes the software costly to update. OIG needs a new, modern version of the software that is easy to use and can be extended in a cost effective manner. In addition, the new version of the software must be 508 compliant.
The current version of the RAT-STATS software can be found at http://www.oig.hhs.gov/compliance/rat-stats/index.asp.
Grand Prize: $25,000.00
In addition to the cash payment, winners will be listed on the new version of the RAT-STATS software (once released), on the RAT-STATS download website, and in the RAT-STATS User Manual.
Finalist: $3,000.00
Finalist: $3,000.00
Finalist: $3,000.00
In order to take part in this Challenge participants must create a software package that replicates the operation of four of the functions of the original RAT-STATS software: (1) Single Stage of Random Numbers; (2) Unrestricted Attribute Appraisal; (3) Unrestricted Variable Appraisal; and (4) Stratified Variable Appraisal. The steps required to add additional statistical modules to the software must be reasonable and fully documented, and the submission must contain all of the required elements (refer to the Submissions page for a complete list).
In addition, participants must agree to the following terms and conditions.
Teams of one or more members can participate in this Challenge. There is no maximum team size. Each team must have a captain, and each individual may only be part of a single team. Individual team members and team captains must register in accordance with the Registration Process section below. The role of the team captain is to serve as the corresponding participant with OIG about the Challenge and to submit the team’s Challenge entry. While OIG will notify all registered Challenge participants by email of any amendments to the Challenge, the team captain is expected to keep the team members informed about matters germane to the Challenge.
(1) To be eligible to win the Challenge prize, each participant (individual or entity) must —
- a. register to participate in the Challenge under the rules promulgated by OIG as published in this Notice;
- b. have complied with all the requirements under this section;
- c. in the case of a private entity, be incorporated in and maintain a primary place of business in the United States or, in the case of an individual, be a citizen or permanent resident of the United States;
- d. not be a Federal entity or Federal employee acting within the scope of their employment;
- e. not be an employee of OIG, a judge of the Challenge, or any other party involved with the design, production, execution, or distribution of the Challenge or the immediate family of such a party (i.e., spouse, parent, step-parent, child, or stepchild); and
- f. be at least 18 years old at the time of submission.
(2) Federal contractors may not use Federal funds from a contract to develop their Challenge submissions or to fund efforts in support of their Challenge submission.
(3) Federal grantees may not use Federal funds to develop COMPETES Act Challenge applications unless consistent with the purpose of their grant award.
(4) An individual or entity shall not be deemed ineligible because the individual or entity used Federal facilities or consulted with Federal employees during a competition if the facilities and employees are made available to all individuals and entities participating in the competition on an equitable basis.
(5) By participating in this Challenge, each individual (whether competing singly or in a group) and entity agrees to assume any and all risks and waive claims against the Federal Government and its related entities (as defined in the COMPETES Act), except in the case of willful misconduct, for any injury, death, damage, or loss of property, revenue, or profits, whether direct, indirect, or consequential, arising from participation in this Challenge, whether the injury, death, damage, or loss arises through negligence or otherwise.
(6) No individual (whether competing singly or in a group) or entity participating in the Challenge is required to obtain liability insurance or demonstrate financial responsibility in order to participate in this Challenge.
(8) By participating in this Challenge, each individual (whether participating singly or in a group) and entity grants to OIG, in any existing or inchoate copyright or patent rights owned by the individual or entity, an irrevocable, paid-up, royalty-free, nonexclusive worldwide license to use, reproduce, post, link to, share, and display publicly on the Web the submission, except for source code. This license includes without limitation posting or linking to the submission, except for source code, on OIG’s public facing website (http://www.oig.hhs.gov/compliance/rat-stats/index.asp). In developing its future software systems, OIG may include algorithms and software from Challenge entries and may consult with individuals or teams that submitted entries. Thus, the license also permits OIG to develop a future software system, independently or with others, using any algorithms or software from Challenge entries, including those obtained from other Challenges or solicitations, and OIG may freely use, reproduce, modify, and distribute the resulting future software system without restriction. Each participant will retain all other intellectual property rights in their submissions, as applicable.
(9) OIG reserves the right, at its sole discretion, to (a) cancel, suspend, or modify the Challenge through amendment to this Federal Register notice, and/or (b) not award any prizes if no entries meet the stated requirements. In addition, OIG reserves the right to disqualify any Challenge participants or entries in instances where cheating or other misconduct is identified.
(10) Each individual (whether participating singly or in a group) or entity agrees to follow all applicable Federal, State, and local laws, regulations, and policies.
(11) Each individual (whether participating singly or in a group) and entity participating in this Challenge must comply with all terms and conditions of these rules, and participation in this Challenge constitutes each such participant’s full and unconditional agreement to abide by these rules. Winning is contingent upon fulfilling all requirements herein.
(12) Each individual (whether participating singly or in a group) and entity grants to OIG and its contractors assisting OIG with this Challenge the right to review the submission, study the algorithms and the code, and run the software on other sets of images.
(13) Submissions must not infringe upon any copyright, patent, trade secrets, or any other rights of any third party. Each individual (whether participating singly or in a group) or entity warrants that she or the team is the sole author and owner of any copyrightable work that the submission comprises, that the submission is wholly original with the participant or is an improved version of an existing work that the participant has sufficient rights to use and improve. In addition, the submission must not trigger any reporting or royalty obligation to any third party. A submission must not include proprietary, classified, confidential, or sensitive information.
(14) The licenses for any code used as part of a submission must be compatible with each other and must allow OIG to distribute and modify the software both within and outside the agency without incurring any reporting or royalty obligations to any third party.
(15) The submission must not contain malicious code such as viruses, timebombs, cancelbots, worms, trojan horses, or other potentially harmful programs or other material or information.
(16) The submission must be unique and must not represent a modification of a previous submission from another team.
(17) Submitted software must be fully functional and operate correctly on Microsoft Windows systems configured in accordance with the applicable United States Government Configuration Baseline (USGCB) and applicable configurations (https://usgcb.nist.gov/). The group policy settings associated with this configuration are also available on the NIST website (https://usgcb.nist.gov/usgcb/microsoft_content.html).
(18) Submitted software must be a stand-alone product that is designed for end users to run in the standard user context without requiring elevated administrative privileges.
(19) Submitted software must not require or make use of any network capabilities.
(20) Submitted software must be section 508 compliant. For example, it must be possible to run all functions using a keyboard, the software must include a well-defined indicator of current program focus, any results provided in image format must also be available in text, and color coding shall not be used as the only means of conveying information. For more details refer to section 36 CFR § 1194.21
Finalist Criteria
The first five eligible submissions that comply with the rules of this Federal Register Notice (see also the Rules page of this website), follow the detailed submission instructions, are complete, are able to fully replicate RAT-STATS on 60 target cases, and include a description of a reasonable method for adding additional modules will each be declared a finalist. Once five finalists have been identified, no further entries from teams not previously identified as finalists will be reviewed even if they are submitted prior to the May 15, 2017, 5:00 p.m. EST deadline. Moreover, the contest will end on May 15, 2017, at 5:00 p.m. EST even if fewer than five finalists have been selected.
Finalists may update their entries without losing their finalist designation. Updates from the finalists will be accepted until 5:00 p.m. EST on the fourteenth day after the fifth finalist has been identified or May 15, 2017, 5:00 p.m. EST, whichever comes first. The newest entry from each team will be used for all judging purposes unless otherwise requested by the team.
Grand Prize Criteria
Twelve OIG employees who use the current version of the RAT-STATS software will review the submissions from the five finalists and vote on which package they would prefer to serve as the RAT-STATS replacement. The package with the most votes will be the winner of the Grand Prize. Ties will be broken by the Office of Inspector General, Office of Audit Services, Director of Quantitative Methods. The instructions to the judges will be as follows, "Please review the operation of the following software packages and select one that you would prefer to use in the future for selecting and analyzing your statistical samples." Critically, the instructions do not ask which software is most like RAT-STATS, but rather which software they would prefer to use. As a result, during the review for the Grand Prize, judges will consider features even if they were not part of the original RAT-STATS package.
Submissions must be entered electronically using the challenge.gov website and must contain a ZIP file with each of the following elements.
(1) Executable software that replicates the 4 target RAT-STATS functions: Single Stage of Random Numbers, Unrestricted Attribute Appraisal, Unrestricted Variable Appraisal, and the Stratified Variable Appraisal.
(2) Source code for the executable that is both human- and machine-readable. The source code can be written in any programming language(s). The source code must be commented sufficiently so that another user can understand the underlying operation of the code.
(3) A text file written in English documenting and explaining any non-cosmetic differences between the submission and the original RAT-STATS software.
(4) A text file written in English summarizing the changes required to add additional RAT-STATS functions to the submission.
(5) A text file written in English documenting all software licenses associated with the source code used as part of the project. The text file must describe the nature of each individual license and their overall compatibility.
(6) A one-page text file written in English that contains the following:
- Names and email addresses of the team captain and all team members
- A five or more character identifier for the entry that is used as a prefix in the names of all of the team’s submitted files
- A brief description of the submission