Skip to content

Commit

Permalink
removing all files other than ace to datomic specific files
Browse files Browse the repository at this point in the history
  • Loading branch information
adamjohnwright committed Feb 12, 2016
1 parent a732951 commit 23a9e12
Show file tree
Hide file tree
Showing 2,136 changed files with 42 additions and 1,687,821 deletions.
74 changes: 42 additions & 32 deletions README.md
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,48 +1,58 @@
#db
# AceDB to Datomic

*The database migration project at WormBase.*
- Reading and writing ACeDB model files.
- Generating Datomic schemas based on ACeDB models.
- Model-driven import of ACeDB data into multiple databases (currently Datomic and MongoDB).
- Emulating an ACeDB server (currently incomplete).

**THIS IS A PRIVATE REPOSITORY**
Installation

## FULL-SCALE DATOMIC PROTOTYPE
sudo yum -y install perl-autodie perl-IPC-System-Simple

This can be found in the `pseudoace` directory. See the [wiki](https://github.com/wormbase/db/wiki)
for more information.

## LAYOUT
##Ace to Datomic

README.md
data/ -- sample data files, organized by date
ec2/ -- scripts and credentials for developing
and benchmarking on the AWS cloud
platforms/ -- one directory for each system under evaluation
This tool is used to import databases from ACeDB to Datomic

##DEVELOPMENT PROCEDURE

Follow the GitFlow mechanism for branching and commiting changes as detailed [here](https://datasift.github.io/gitflow/IntroducingGitFlow.html)

## DATA
- To get help run the following command

smallace datasets, versioned by date, prepared by @khowe. Please
reference which dataset you use in your documentation.
lein run ace-to-datomic --help

## CONVENTIONS
###Help Output

I suggest we use some standard bioinformatics conventions. For each platform:
Ace to dataomic is tool for importing data from ACeDB into to Datomic database

* scripts in bin/ (or scripts/)
* lib/ if required, etc.
* If your code requires third party modules, please document them and (if possible) include them as submodules.
* There is a stub README in each directory. This would be a good place to collect documentation for now.
Usage: ace-to-datomic [options] action

## DOCUMENTATION STANDARDS
Options:
--model PATH Specify the model file that you would like to use that is found in the models folder e.g. models.wrm.WS250.annot
--url URL Specify the url of the Dataomic transactor you would like to connect. Example: datomic:free://localhost:4334/WS250
--schema-filename PATH Specify the name of the file for the schema view to be written to when selecting Action: generate-schema-view exampls schema250.edn
--log-dir PATH Specifies the path to and empty directory to store the Datomic logs in. Example: /datastore/datomic/tmp/datomic/import-logs-WS250/
--acedump-dir PATH Specifies the path to the directory of the desired acedump. Example /datastore/datomic/tmp/acedata/WS250/
--backup-file PATH Secify the path to the file in which you would like to have the database dumped into
--datomic-database-report-filename PATH Specify the relative or full path to the file that you would like the report to be written to
-v, --verbose
-f, --force
-h, --help

Minimally, documentation should include the shell commands necessary to load your resource, expected
input files and any output created.
Actions: (required options for each action are provided in square brackets)
create-database Select this option if you would like to create a Datomic database from a schema. Required options [model, url]
generate-datomic-schema-view Select if you would like the schema to the database to be exported to a file. Required options [schema-filename, url]
acedump-to-datomic-log Select if you are importing data from ACeDB to Datomic and would like to create the Datomic log files [url, log-dir, acedump-dir]
sort-datomic-log Select if you would like to sort the log files generated from your ACeDB dump [log-dir]
import-logs-into-datomic Select if you would like to import the sorted logs back into datomic [log-dir, url]
excise-tmp-data Select in order to remove all the tmp data that was created in the database to import the data [url]
test-datomic-data Select if you would like to perform tests on the generated database [url acedump-dir]
all-import-actions Select if you would like to perform all actions from acedb to datomic [model url schema-filename log-dir acedump-dir]
generate-datomic-database-report Select if you want to generate a summary report of the contents of a particular Datomic database [url datomic-database-report-filename]
list-databases Select if you would like to get a list of the database names [url]
delete-database Select this option if you would like to delete a database in datomic [url]. If the force option you will not be asked if you are certain about your decision
backup-database Select if you would like to backup a datomic database into a file

Better yet, assume a naive user and vanilla system. Include all commands necessary to get that
system up and running with your platform and loaded with the test data. This will enable any of us
to test, verify or extend any of the platforms with minimal work by launching a new instance on AWS.
We can (and should) automate the installation, configuration and loading of each platform so that we
can easily evaluate more complicated scenarios.

Example command

lein run ace-to-datomic generate-datomic-database-report --url datomic:free://localhost:4334/WS250 --datomic-database-report-filename /home/ec2-user/git/db/pseudoace/hello.t -v

Binary file removed data/2014.03.14-smallace.tar.gz
Binary file not shown.
Binary file removed data/2014.06.30-smallace.tar.gz
Binary file not shown.
Binary file removed data/2014.09.16-mediumace.tar.gz
Binary file not shown.
2 changes: 0 additions & 2 deletions ec2/README.md

This file was deleted.

8 changes: 0 additions & 8 deletions geneace/.gitignore

This file was deleted.

22 changes: 0 additions & 22 deletions geneace/project.clj

This file was deleted.

100 changes: 0 additions & 100 deletions geneace/scripts/batch_newfeature.pl

This file was deleted.

129 changes: 0 additions & 129 deletions geneace/scripts/id_resurrect.pl

This file was deleted.

Loading

0 comments on commit 23a9e12

Please sign in to comment.