- Project Status
- Installation
- Usage
- Use a PyFilesytem2 filesystem as DAV provider for WsgiDAV
- Use a WsgiDAV DAV provider as filesystem for PyFilesystem2
- Try out WsgiDAV on Google Cloud Platform
- Use the Datastore DAV provider, Datastore FS filesystem or Datastore DB explorer
- Use the Firestore DAV provider, Firestore FS filesystem or Firestore DB explorer
- Try other combinations
- Original CloudDAV Project
The master branch here provides the following building blocks for WsgiDAV 3.0.x and Python 3.7+ with current Google Cloud services as back-end.
- Datastore DAV Provider for a virtual filesystem built on Google Cloud Firestore in Datastore mode
- Firebase Domain Controller to validate id tokens with Google Cloud Identity Platform (Firebase Authentication)
- Cachelib Lock Manager to support in-memory locks using cachelib (either memcache, redis or in-memory)
- Property Manager partially integrated with the DAV Provider (?)
- Firestore DAV Provider for a virtual filesystem built on Google Cloud Firestore in native mode
In addition, the following new filesystem providers are available for PyFilestem2:
- Datastore FS for basic support of Google Cloud Firestore in Datastore mode as filesystem for PyFilestem2
- Datastore DB for a read-only database explorer of Google Cloud Firestore in Datastore mode with PyFilestem2
- Firestore FS for basic support of Google Cloud Firestore in native mode as filesystem for PyFilestem2
- Firestore DB for a read-only database explorer of Google Cloud Firestore in native mode with PyFilestem2
And finally, you can now swap PyFilesystem2 filesystems and WsgiDAV DAV providers and use them in both environments:
- DAVProvider2FS for basic support of WsgiDAV DAV providers as filesystem for PyFilesystem2
- FS2DAVProvider for basic support of PyFilesystem2 filesystems as DAV provider for WsgiDAV
A list of custom PyFilesystem2 filesystems is available here, and a list of custom WsgiDAV DAV providers is available here.
The purpose here is not to provide a production-ready version for use on Google Cloud, but to experiment with various newer back-end services and explore the differences with older versions. And of course have fun while doing it :-)
You can either clone the public repo:
$ git clone https://github.com/mikespub-org/mp-fs-wsgidav.git
Or download the zip file:
$ wget https://github.com/mikespub-org/mp-fs-wsgidav/archive/master.zip
Copy the src/mapper/dav_provider_from_fs.py file to your project, and import FS2DAVProvider.
Example using FS2DAVProvider() as DAV provider in WsgiDAV:
from wsgidav.wsgidav_app import WsgiDAVApp
from .dav_provider_from_fs import FS2DAVProvider
from fs.osfs import OSFS
source_fs = OSFS("/tmp")
dav_provider = FS2DAVProvider(source_fs)
config = {"provider_mapping": {"/": dav_provider}}
config["simple_dc"] = {"user_mapping": {"*": True}} # allow anonymous access or use domain controller
app = WsgiDAVApp(config)
# run_wsgi_app(app)
Copy the src/mapper/fs_from_dav_provider.py file to your project, and import DAVProvider2FS.
Example using DAVProvider2FS() as filesystem in PyFilesystem2:
from .fs_from_dav_provider import DAVProvider2FS
from wsgidav.fs_dav_provider import FilesystemProvider
dav_provider = FilesystemProvider("/tmp")
dav_fs = DAVProvider2FS(dav_provider)
# dav_fs.environ["wsgidav.auth.user_name"] = "tester"
# dav_fs.environ["wsgidav.auth.roles"] = ["admin"]
dav_fs.listdir("/")
If you want to try out the Datastore or Firestore DAV provider, the Firebase domain controller etc., you can use this project and deploy it to App Engine.
- Create a Google Cloud App Engine project
- Create a Google Cloud Datastore or Firestore database in the same project
- Install Google Cloud Identity Platform from the Marketplace and configure an identity provider (e.g. Google)
- Rename the src/app.yaml.template file to app.yaml and update the FIREBASE_PROJECT_ID and FIREBASE_API_KEY environment variables
- Deploy the project to App Engine with the gcloud tool
- Go to the homepage of your newly deployed project, login using the /auth/ link and define yourself as admin using the /_admin link
- Remove anonymous read-write access in the WsgiDAV config file and redeploy the project to App Engine
If you want to use the Datastore DAV provider / FS filesystem / DB explorer yourself, you'll need to copy the src/data/ directory to your project.
Configure the service account credentials to use:
$ export GOOGLE_APPLICATION_CREDENTIALS="~/datastore-user.cred.json"
Example using Flask view functions as read-only database explorer via browser:
$ python3 -m data.views
Example using DatastoreDB() as read-only database explorer via command line:
$ python3 -m data.datastore_db [<kind> [<id> [<propname>]]]
Example using DatastoreDB() as read-only database explorer in PyFilesystem2:
from .data.datastore_db import DatastoreDB
data_db = DatastoreDB()
data_db.listdir("/")
Example using DatastoreFS() as filesystem in PyFilesystem2:
from .data.datastore_fs import DatastoreFS
data_fs = DatastoreFS("/")
data_fs.listdir("/")
Example using DatastoreDAVProvider() as DAV provider in WsgiDAV:
from wsgidav.wsgidav_app import WsgiDAVApp
from .data.datastore_dav import DatastoreDAVProvider
dav_provider = DatastoreDAVProvider()
config = {"provider_mapping": {"/": dav_provider}}
config["simple_dc"] = {"user_mapping": {"*": True}} # allow anonymous access or use domain controller
app = WsgiDAVApp(config)
# run_wsgi_app(app)
If you want to use the Firestore DAV provider / FS filesystem / DB explorer yourself, you'll need to copy the src/fire/ directory to your project.
Configure the service account credentials to use:
$ export GOOGLE_APPLICATION_CREDENTIALS="~/firestore-user.cred.json"
Example using Flask view functions as read-only database explorer via browser:
$ python3 -m fire.views
Example using FirestoreDB() as read-only database explorer via command line:
$ python3 -m fire.firestore_db [<coll> [<id> [<coll> [<id> [...]]]]]
$ python3 -m fire.firestore_db <coll>[/<id>[/<coll>]] <id>[<propname>]
Example using FirestoreDB() as read-only database explorer in PyFilesystem2:
from .fire.firestore_db import FirestoreDB
fire_db = FirestoreDB()
fire_db.listdir("/")
Example using FirestoreFS() as filesystem in PyFilesystem2:
from .fire.firestore_fs import FirestoreFS
fire_fs = FirestoreFS("/")
fire_fs.listdir("/")
Example using FirestoreDAVProvider() as DAV provider in WsgiDAV:
from wsgidav.wsgidav_app import WsgiDAVApp
from .fire.firestore_dav import FirestoreDAVProvider
dav_provider = FirestoreDAVProvider()
config = {"provider_mapping": {"/": dav_provider}}
config["simple_dc"] = {"user_mapping": {"*": True}} # allow anonymous access or use domain controller
app = WsgiDAVApp(config)
# run_wsgi_app(app)
You can also combine DatastoreDB() or FirestoreDB() with FS2DAVProvider() to provide a browser/WebDAV interface to your Datastore entities or Firestore documents - see try_db2dav.py.
Or try DatastoreDAVProvider() with DAVProvider2FS() as a slower alternative to DatastoreFS() - see try_dav2fs.py.
Or some other combination you can think of, like migrating from Datastore to Firestore via PyFilesystem2...
The original version of CloudDAV from 2010 was meant to run on Google App Engine with Python 2.5 and WsgiDAV 0.5.0
It provided the following architecture building blocks:
- Big Table DAV Provider for a virtual filesystem built on App Engine Datastore
- Google Domain Controller to provide basic authentication (username/password) with Google Login
- Memcache Lock Manager to support in-memory locks using App Engine Memcache
- Property Manager partially integrated with the DAV Provider
The current version at https://github.com/mar10/clouddav and here in the python27_base branch from 2019-12 was ported to "work" with Python 2.7 and WsgiDAV 3.0.x - it remains in alpha.
For more details see History.