small errors in did.database

[stevevanhooser]

Small errors to fix, will move to separate issues:

• did.database.get_docs does not check to make sure that the docs exist on the current branch. It can return a document even if it is not on the current branch. It shouldn't, because down the road people might not have permission to access documents that are not on their branch.

• We need to catch and explicitly disallow the situation where someone deletes a document and then tries to add it back (with the same id). While there are a few instances where this could work (if it exists on no other branches), it is most straightforward just to keep a list of deleted document IDs and not allow them to be re-added. Currently, because the field data is not removed from the table, this causes the unusual behavior of the document re-claiming its old field data (which is not updated with any new information). The fix is just not to allow one to add a new document with the same id as any document that has been deleted.

• When documents are removed and exist on no branches, their cached files are not deleted; they should be.

• When documents are removed and exist on no branches, their field data is not removed.

• If someone produces a query like "find field value > []", it produces an sqlite error instead of returning no matches. We should try to avoid sqlite errors.

• When there are multiple superclass combinations of depends_on, the validator can be fooled (stimulus_response_parameters_basic, for example). For the time being, I fixed this by having the validator only check that all of the required depends_on types are there, and it ignores any extra that might be present. But we shouldn't need to do that.

• We had initially thought that we would not enter any data from matrixes for the purpose of search. Unfortunately, we need to be able to search this sometimes. Ideally, we would have the system use the 'queryable' flag from the schema indicate whether a matrix should be entered into the database. But for now we ought to put them all in.

• When running parallel processes, we get errors that "the database is locked". The right behavior would be for it to wait for up to 60 seconds, trying again at random intervals every 1 second or so, to complete its task.

• We should have a string array type

Just to think about

• The past ndi.database objects were not handles. Not sure if it matters that did.database objects are. Still thinking