Skip to content
This repository has been archived by the owner on Aug 22, 2023. It is now read-only.

Better ortholog cache #8

Open
8 tasks
gcornut opened this issue Apr 2, 2017 · 0 comments
Open
8 tasks

Better ortholog cache #8

gcornut opened this issue Apr 2, 2017 · 0 comments

Comments

@gcornut
Copy link
Member

gcornut commented Apr 2, 2017

For now the cache is in memory and grows indefinitely
TODO

  • Add cache persistance (to disk)
  • Flatten cache for easier management
    {:org {:prot :ortho-group}} => {[:org :prot] :ortho-group}
  • Set up caching strategy
  • Link memory cache, disk cache and inparanoid webservices all together

Pseudo-code:

(if in-memory-cache?
   (get-from-memory-cache)
   (save-in-memory-cache
     (if in-disk-cache?
        (get-from-disk-cache)
        (save-in-disk-cache
          (get-from-inparanoid-ws)))))

Proposed caching strategy:

  • LRU with size threshold (with clojure core.cache)
  • Write to memory and disk on new entry (or periodically write to disk?)

Persistence:

  • Use nippy for writting/reading clojure data structures in binary
  • Split the cache in organism pairs (like in PMB)

TODO cache loader:

  • Download OrthologXML by organism pairs
  • Parse & load to cache
  • Add approx loaded percent
  • Add progress monitoring interface
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant