-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
removing all files other than ace to datomic specific files
- Loading branch information
1 parent
a732951
commit 23a9e12
Showing
2,136 changed files
with
42 additions
and
1,687,821 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,48 +1,58 @@ | ||
#db | ||
# AceDB to Datomic | ||
|
||
*The database migration project at WormBase.* | ||
- Reading and writing ACeDB model files. | ||
- Generating Datomic schemas based on ACeDB models. | ||
- Model-driven import of ACeDB data into multiple databases (currently Datomic and MongoDB). | ||
- Emulating an ACeDB server (currently incomplete). | ||
|
||
**THIS IS A PRIVATE REPOSITORY** | ||
Installation | ||
|
||
## FULL-SCALE DATOMIC PROTOTYPE | ||
sudo yum -y install perl-autodie perl-IPC-System-Simple | ||
|
||
This can be found in the `pseudoace` directory. See the [wiki](https://github.com/wormbase/db/wiki) | ||
for more information. | ||
|
||
## LAYOUT | ||
##Ace to Datomic | ||
|
||
README.md | ||
data/ -- sample data files, organized by date | ||
ec2/ -- scripts and credentials for developing | ||
and benchmarking on the AWS cloud | ||
platforms/ -- one directory for each system under evaluation | ||
This tool is used to import databases from ACeDB to Datomic | ||
|
||
##DEVELOPMENT PROCEDURE | ||
|
||
Follow the GitFlow mechanism for branching and commiting changes as detailed [here](https://datasift.github.io/gitflow/IntroducingGitFlow.html) | ||
|
||
## DATA | ||
- To get help run the following command | ||
|
||
smallace datasets, versioned by date, prepared by @khowe. Please | ||
reference which dataset you use in your documentation. | ||
lein run ace-to-datomic --help | ||
|
||
## CONVENTIONS | ||
###Help Output | ||
|
||
I suggest we use some standard bioinformatics conventions. For each platform: | ||
Ace to dataomic is tool for importing data from ACeDB into to Datomic database | ||
|
||
* scripts in bin/ (or scripts/) | ||
* lib/ if required, etc. | ||
* If your code requires third party modules, please document them and (if possible) include them as submodules. | ||
* There is a stub README in each directory. This would be a good place to collect documentation for now. | ||
Usage: ace-to-datomic [options] action | ||
|
||
## DOCUMENTATION STANDARDS | ||
Options: | ||
--model PATH Specify the model file that you would like to use that is found in the models folder e.g. models.wrm.WS250.annot | ||
--url URL Specify the url of the Dataomic transactor you would like to connect. Example: datomic:free://localhost:4334/WS250 | ||
--schema-filename PATH Specify the name of the file for the schema view to be written to when selecting Action: generate-schema-view exampls schema250.edn | ||
--log-dir PATH Specifies the path to and empty directory to store the Datomic logs in. Example: /datastore/datomic/tmp/datomic/import-logs-WS250/ | ||
--acedump-dir PATH Specifies the path to the directory of the desired acedump. Example /datastore/datomic/tmp/acedata/WS250/ | ||
--backup-file PATH Secify the path to the file in which you would like to have the database dumped into | ||
--datomic-database-report-filename PATH Specify the relative or full path to the file that you would like the report to be written to | ||
-v, --verbose | ||
-f, --force | ||
-h, --help | ||
|
||
Minimally, documentation should include the shell commands necessary to load your resource, expected | ||
input files and any output created. | ||
Actions: (required options for each action are provided in square brackets) | ||
create-database Select this option if you would like to create a Datomic database from a schema. Required options [model, url] | ||
generate-datomic-schema-view Select if you would like the schema to the database to be exported to a file. Required options [schema-filename, url] | ||
acedump-to-datomic-log Select if you are importing data from ACeDB to Datomic and would like to create the Datomic log files [url, log-dir, acedump-dir] | ||
sort-datomic-log Select if you would like to sort the log files generated from your ACeDB dump [log-dir] | ||
import-logs-into-datomic Select if you would like to import the sorted logs back into datomic [log-dir, url] | ||
excise-tmp-data Select in order to remove all the tmp data that was created in the database to import the data [url] | ||
test-datomic-data Select if you would like to perform tests on the generated database [url acedump-dir] | ||
all-import-actions Select if you would like to perform all actions from acedb to datomic [model url schema-filename log-dir acedump-dir] | ||
generate-datomic-database-report Select if you want to generate a summary report of the contents of a particular Datomic database [url datomic-database-report-filename] | ||
list-databases Select if you would like to get a list of the database names [url] | ||
delete-database Select this option if you would like to delete a database in datomic [url]. If the force option you will not be asked if you are certain about your decision | ||
backup-database Select if you would like to backup a datomic database into a file | ||
|
||
Better yet, assume a naive user and vanilla system. Include all commands necessary to get that | ||
system up and running with your platform and loaded with the test data. This will enable any of us | ||
to test, verify or extend any of the platforms with minimal work by launching a new instance on AWS. | ||
We can (and should) automate the installation, configuration and loading of each platform so that we | ||
can easily evaluate more complicated scenarios. | ||
|
||
Example command | ||
|
||
lein run ace-to-datomic generate-datomic-database-report --url datomic:free://localhost:4334/WS250 --datomic-database-report-filename /home/ec2-user/git/db/pseudoace/hello.t -v | ||
|
Binary file not shown.
Binary file not shown.
Binary file not shown.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.