Merge pull request #1 from SC-Tech-Program/master

syncing with head repository

AD-AE-Appendices_Authors.md

@@ -19,49 +19,72 @@ Artifact Evaluation (AE) Appendices are optional, but strongly encouraged.
 
 ## <a name="requirements"></a>AD and AE Appendix Requirements 
 
-**Q1. Are AD and AE appendices required in order to submit to SC19?**
+**Are AD and AE appendices required in order to submit to SC19?**
 
-A1. An AD appendix is _required_ for all Technical Program submissions.
+An AD appendix is _required_ for all Technical Program submissions.
 An AE appendix is _optional_ but strongly encouraged.
 
-**Q2. Do I need to make my software open source in order to complete the AD appendix?**
+**Do I need to make my software open source in order to complete the AD appendix?**
 
-A2. No. You are not asked to make any changes to your computing environment in order to complete the appendix. 
+No. You are not asked to make any changes to your computing environment in order to complete the appendix. 
 The AD appendix is meant to _describe_ the computing environment in which you produced your results.
 Any author-created software _does_ need to be open source, however, to be eligible for the ACM Artifacts Available badge (see below).
 
-**Q3. How should I format my AD Appendix**
+**How should I format my AD Appendix**
 
-Q3. You don't need to worry about formatting the Appendices. You will be presented with an online form during the paper submission with questions that you will answer directly on the submission site. After answering the questions, the system will automatically generate a PDF of the Appendix for you.
+You don't need to worry about formatting the Appendices. You will be presented with an online form during the paper submission with questions that you will answer directly on the submission site. After answering the questions, the system will automatically generate a PDF of the Appendix for you.
 
-**Q4. What information do I need to provide in the AD/AE Appendix online form?**
+**What information do I need to provide in the AD/AE Appendix online form?**
 
-Q4. A printout of the questions included in the AD/AE Appendix online form is provided in the [`Author-Kit`](https://github.com/SC-Tech-Program/Author-Kit) repository. Be sure to familiarize yourself with these _before writing your paper_, and ideally before or while you are producing your results.
+A printout of the questions included in the AD/AE Appendix online form is provided in the [`Author-Kit`](https://github.com/SC-Tech-Program/Author-Kit) repository. Be sure to familiarize yourself with these _before writing your paper_, and ideally before or while you are producing your results.
 
 ## <a name="review"></a>Review Process 
 
-**Q5. Who will review my appendices?**
+**Who will review my appendices?**
 
-A5. The AD & AE Appendices will be reviewed _with your paper_ by the Technical Program committee, but the artifact URLs will be removed from the version they review, as a precaution in support of double-blind review.
+The AD & AE Appendices will be reviewed _with your paper_ by the Technical Program committee, but the artifact URLs will be removed from the version they review, as a precaution in support of double-blind review.
 In addition, the **AD/AE Appendices Committee** will review the unredacted appendices, and will check that artifacts are indeed available in the URLs provided. They will also help authors improve their appendices, in a double-open arrangement.
 
-**Q6. How will review of appendices interact with the double-blind review process?**
+**How will review of appendices interact with the double-blind review process?**
 
-A6. The AD appendix should describe the data, software and hardware artifacts involved in producing the results.
+The AD appendix should describe the data, software and hardware artifacts involved in producing the results.
 Reviewers _could_ discover the author's identity if they embark on an online search, but they will be asked _not to_, in support of double-blind review. Author-provided artifact URLs will be redacted from the appendices provided to the reviewers.
 
 ## <a name="impact"></a>Impact of AD and AE Appendices 
 
-**Q7. What's the impact of an Artifact Description appendix on scientific reproducibility?**
+**What's the impact of an Artifact Description appendix on scientific reproducibility?**
 
-A7. Reproducibility depends on, as a first step, sharing the provenance of results with transparency, and the AD appendix is an instrument of documentation and transparency. A good AD appendix helps researchers document their results, and helps other researchers build from them.
+Reproducibility depends on, as a first step, sharing the provenance of results with transparency, and the AD appendix is an instrument of documentation and transparency. A good AD appendix helps researchers document their results, and helps other researchers build from them.
 
-**Q8. What's the impact of an Artifact Evaluation appendix on scientific reproducibility?**
+**The paper text explains why I believe my answers are right and shows all my work.  Why do I need to provide an Artifact Evaluation appendix?**
 
-A8. An artifact-evaluation effort can increase the trustworthiness of computational results.
+There are many good reasons for formalizing the artifact evaluation process.  Standard practice varies across disciplines, and SC is an international, multi-disciplinary conference. Labeling the evaluation as such improves our ability to review the paper and improves reader confidence in the veracity of the results when approaching the work from a different background.
+
+## <a name="data"></a>Artifacts
+
+**What are "author created" artifacts and why make the distinction?**
+
+Author created artifacts are the hardware, software, or data created by the paper's authors.  Only these artifacts need be made available to facilitate reproducibility.  Proprietary, closed source artifacts (e.g. commercial software and CPUs) will necessarily be part of many research studies.  These proprietary artifacts should be described to the best of the author's ability but do not need to be provided.
+
+**What about proprietary author-created artifacts?**
+
+The ideal case for reproducibility is to have all author created artifacts publically available with a stable identifier.  Papers involving proprietary, closed source author-created artifacts should indicate the availability of the artifacts and describe them as much as possible.  Note that results dependent on closed source artifacts are not reproducible and are therefore ineligible for most of the ACM's artifact review badges.  See https://www.acm.org/publications/policies/artifact-review-badging.
+
+**Are the numbers used to draw our charts a data artifact?**
+
+Not necessarily.  Data artifacts are the data (input or output) required to reproduce the results, not necessarily the results themselves.  For example, if your paper presents a system that generates charts from datasets then providing an input dataset would facilitate reproducibility.  However, if the paper merely *uses charts to elucidate results* then the input data to whatever tool you used to draw those charts isn't required to reproduce the paper's results.  The tool which drew the chart isn't part of the study, so the input data to that tool is not a data artifact of this work.  
+
+**Help! My data is HUGE! How do I make it publically available with a stable identifier?**
+
+Use Zenodo (https://help.zenodo.org/).  Contact them for information on how to upload extremely large datasets.  You can easily upload datasets of 50GB or less, have multiple datasets, and there is no size limit on communities.
+
+
+**What's the impact of an Artifact Evaluation appendix on scientific reproducibility?**
+
+An artifact-evaluation effort can increase the trustworthiness of computational results.
 It can be particularly effective in the case of results obtained using specialized computing platforms, not available to other researchers. Leadership computing platforms, novel testbeds, and experimental computing environments are of keen interest to the supercomputing community.
 Access to these systems is typically limited, however. Thus, most reviewers cannot independently check results, and the authors themselves may be unable to recompute their own results in the future, given the impact of irreversible changes in the environment (compilers, libraries, components, etc.).
-The various forms of Artiface Evaluation improve confidence that computational results from these special platforms are correct.
+The various forms of Artifact Evaluation improve confidence that computational results from these special platforms are correct.
 
 ## <a name="badges"></a>ACM Artifacts Available and Artifacts Evaluated Badges 
 

Panels_Authors.md

@@ -11,7 +11,7 @@ Panel Format
 
 **Q: What is an SC panel?**
 
-A: A panel provides a forum that promotes lively, highly interactive discussions on a wide variety of topics among the panelists and audience. The panel format includes a moderator and up to six panelists. A panel is not a set of mini-presentations that do not leave time for discussion with the audience. A panel is non-commercial.
+A: A panel provides a forum that promotes lively, highly interactive discussions on a wide variety of topics among the panelists and audience. The panel format includes a moderator and panelists. We recommend no more than six panelists as this is the maximum number of chairs available for panelists. A panel is not a set of mini-presentations that do not leave time for discussion with the audience. A panel is non-commercial.
 
 **Q: What is the length of a panel session?**
 
@@ -22,7 +22,7 @@ Submissions and Notifications
 
 **Q: When will I know if my proposed panel will be included in the conference?**
 
-A: Accept or reject notifications will be sent to all submitters by late June.
+A: Accept or reject notifications will be sent to all submitters by 22 June.
 
 
 **Q: Where can I find the Panels program online?**
@@ -32,20 +32,25 @@ A: The program for Panels will be posted here within two weeks of the notificati
 
 **Q: Can I submit more than one panel proposal?**
 
-A: You are welcome to submit multiple proposals. A committee of peers selects panels in a peer-review process. Because a limited number of slots are available, any individual may be a member (i.e., contact, moderator, or panelist) of at most two panels.
+A: You are welcome to submit multiple proposals. A committee of peers selects panels in a peer-review process. Because a limited number of slots are available, any individual may be a member (i.e., contact, moderator, or panelist) of at most two accepted panels.
 
 
 **Q: Do panels need to have a title?**
 
-A: You should indicate your panel’s title when submitting your proposal, as well as the name and affiliation of each discussant and the moderator. You do not have to include titles for individual presentations. A contact for the panel proposal should also be provided. You must include a brief abstract, in the form of a call for participation, describing the panel topic and questions to be addressed exactly as you want them to appear in the conference program.
+A: Yes. You must indicate your panel’s title when submitting your proposal, as well as the name and affiliation of each panelist and the moderator. A contact for the panel proposal should also be provided. You must include a brief abstract, in the form of a call for participation, describing the panel topic and questions to be addressed exactly as you want them to appear in the conference program.
 
 
 **Q: What material do I need to submit?**
 
 A. Your submission should include:
 
-* A brief abstract, in the form of a call for participation, describing the panel topic and questions to be addressed exactly as you want them to appear in the conference program;
-* A proposal of no more than two pages, uploaded as a single PDF file, that describes the panel in more detail. This proposal should also include your panel title. The panel proposal may include information such as a description of the panel topic or position statements by the panelists. You must convince the Panels review committee that the panel will truly be an interactive session and not deteriorate into long-winded, disjointed, and boring mini-presentations or fluffy entertainment. Do not waste space giving a technical history; instead, explain why the panel will attract a good audience and why the audience will find the panel to be insightful and impactful.
+* A title;
+* Moderator information with a short biography (maximum 150 words);
+* Contact person information if different from the moderator;
+* Panelist information with a short biography (maximum 150 words);
+* A brief abstract, in the form of a call for participation, describing the panel topic and questions to be addressed exactly as you want them to appear in the conference program (maximum 150 words);
+* A proposal of no more than two pages, uploaded as a single PDF file, that describes the panel in more detail. This proposal should also include your panel title. The panel proposal may include information such as a description of the panel topic or position statements by the panelists. You must convince the Panels review committee that the panel will truly be an interactive session and not deteriorate into long-winded, disjointed, and boring mini-presentations or fluffy entertainment. Do not waste space giving a technical history; instead, explain why the panel will attract a good audience and why the audience will find the panel to be insightful and impactful;
+* A diversity justification, describing how you plan to create diversity among panelists with respect to topical background, institution type, geography and demographic characteristics like seniority and gender (maximum 70 words).
 
 Review Criteria and Conflict of Interest (COI)
 ----------------
@@ -108,7 +113,7 @@ Infrastructure and Organization
 
 **Q: What infrastructure is provided for SC panels?**
 
-A: Each panel’s infrastructure includes one projector, one screen, one table for the panelists with multiple chairs, one clip-on or one podium microphone for the moderator, and microphones for the panelists. One or multiple aisle microphone will be provided. The moderator should direct attendees to the aisle microphone for questions so that everyone can hear.
+A: Each panel’s infrastructure includes one projector, two screens, two tables for the panelists with six chairs, one fixed microphone at the podium, a lavalier lapel microphone, and three or four microphones on the panelist tables. A stand microphone is provided in each of the three main aisles in the room.
 
 **Q: I am a panelist or a moderator. When should I arrive at and leave my designated room?**