Skip to content

Resource API Transition Plan

Jump to bottom
NielsCharlier edited this page Dec 7, 2015 · 4 revisions

Resource API Transition Plan

The [ResourceStore Design] outlines an API for stream based access to files in the data directory. Transitioning the codebase to use this API will require a bit of planning.

{toc}

Planning

GeoServerResourceLoader

  • Update internals of GeoServerResourceLoader
  • Add Resource access methods to GeoServerResourceLoader
  • Remove use of GeoServerResourceLoader search locations
    • WEB_INF/web.xml file access

GeoServerDataDirectory and GeoServerSecurityManager

  • Update internals of GeoServerDataDirectory
    • Already uses GeoServerResourceLoader
    • Update methods to use Resource internally
  • Add Resource access methods to GeoServerDataDirectory
  • Update internals of GeoServerSecurityManager to use Resource
  • Add Resource access methods to GeoServerSecurityManager

File interaction:

  • Research

    • Review use org.geoserver.platform.FileWatcher
    • Review use org.geoserver.security.file.FileWatcher
    • Review GWC LockProvider

  • Port LockProvider and integrate with ResourceStore

    • Bridge from GeoServer LockProvider to GWC LockProvider

  • Port FileWatcher implementations to use events

    • Update internals of platform FileWatcher so subclasses continue to function unchanged
    • Update internals of security FileWatcher so subclasses continue to function unchanged

  • Update security to use Resource and Resource events directly rather than FileWatcher

Update GeoServer codebase module by module (see also technical debt)

  • Direct use of ResourceStore/Resource and Resource.out () for “stream access”
  • Replace GeoServerDataDirectory.findOrCreateDir() by ResourceStore.get()
  • Use of Resource.file () for “file access” (i.e. to pass file or URL reference to external library)
  • Replace GeoServerResourceLoader “directory access” with use of Resource
  • Modules:
    • main
    • geowebcache
    • wms
    • wfs
    • web
    • kml
    • rest & restconfig
    • security
    • extensions
      • importer
      • wps
      • csw
      • xslt
      • monitor
      • css
      • control-flow
      • printing
      • ogr
    • community
      • geofence, geofence-server
      • script
      • netcdf
      • validation
      • rest-upload
      • wms-eo
      • wps-download
      • hz-cluster
      • jms-cluster
      • ftp

GeoServerResourceLoader

  • Update internals of GeoServerResourceLoader
    • This “migrates” the codebase to use ResourceStore for lookup
  • Introduce methods for Resource access
  • Remove use of GeoServerResourceLoader search locations
    • GeoServerExtensions.file ( path ) provided for access to servletContext
    • Fix GeoServerJ2eeRoleService access of WEB*INF/web.xml
  • Transition checklist
    • org.geoserver
    • org.geoserver.catalog
    • org.geoserver.catalog.impl
    • org.geoserver.catalog.rest
    • org.geoserver.catalog.util
    • org.geoserver.config
    • org.geoserver.config.util
    • org.geoserver.csw
    • org.geoserver.csw.store.internal
    • org.geoserver.data.test
    • org.geoserver.ftp
    • org.geoserver.gwc.config
    • org.geoserver.gwc.layer
    • org.geoserver.h2
    • org.geoserver.inspire
    • org.geoserver.jdbcconfig
    • org.geoserver.jdbcconfig.internal
    • org.geoserver.logging
    • org.geoserver.monitor
    • org.geoserver.monitor.auditlog
    • org.geoserver.ows
    • org.geoserver.security
    • org.geoserver.security.impl
    • org.geoserver.security.ldap
    • org.geoserver.spatialite
    • org.geoserver.template
    • org.geoserver.test
    • org.geoserver.wcs
    • org.geoserver.web
    • org.geoserver.web.admin
    • org.geoserver.web.util
    • org.geoserver.web.wicket
    • org.geoserver.wfs
    • org.geoserver.wfs.response
    • org.geoserver.wfs.web
    • org.geoserver.wfs.xml
    • org.geoserver.wfs.xml.v1*1_0
    • org.geoserver.wms
    • org.geoserver.wms.eo
    • org.geoserver.wps
    • org.vfny.geoserver.global
    • org.vfny.geoserver.wms.responses.map.htmlimagemap

web.xml regression

GeoServerJ2eeRoleService makes use of the following:

File webXML = loader.find( "web.xml" );

This is the only example of using search locations to find content outside of the data directory.

Functionality moved to GeoServerExtensions in order to use servlet context:

File webXML = GeoServerExtensions.file( "WEB_INF/web.xml" );

GeoServerDataDirectory

  • Update internals of GeoServerDataDirectory
  • Add Resource access methods to GeoServerDataDirectory
  • Transition to Resource use where possible
  • Checklist (106 references)
    • org.geoserver.catalog
    • org.geoserver.community.css.web
    • org.geoserver.config
    • org.geoserver.csw.store.simple
    • org.geoserver.data
    • org.geoserver.ftp
    • org.geoserver.gwc
    • org.geoserver.kml.icons
    • org.geoserver.monitor.hib
    • org.geoserver.script
    • org.geoserver.script
    • org.geoserver.security
    • org.geoserver.security.filter
    • org.geoserver.security.impl
    • org.geoserver.security.ldap
    • org.geoserver.template
    • org.geoserver.test
    • org.geoserver.web.admin
    • org.geoserver.wfs.xslt.config
    • org.geoserver.wms
    • org.geoserver.wps
    • org.vfny.geoserver.global

findDataDir / findDataDir regression

The following regressions were found, often due to code taking advantageous of gaps in the implementation of GeoServerDataDirectory.

GWCIntegrationTest:

File h2DefaultStore = dd.findDataFile("gwc/diskquota_page_store_h2");

GetCoverageTest.onSetUp has the following:

File data = getDataDirectory().findDataDir("watertemp");

In these cases I am fixing the test case, but will be more concerned if production code is effected by the dataRoot (false) implementation issues as shown below.

{warning} The code is asking to “data/watertemp” but manages to find “watertemp” due to a strange turn of events:

  • findDataDir (“watertemp”) is implemented as:

    return dataDir( false, "watertemp" );

  • Where dataDir ends up with:

    return resourceLoader.find( dataRoot(false), "watertemp");

  • Which is where things go wrong, dataRoot (false) fails to find the data directory, resulting in null

    return resourceLoader.find( null, "watertemp");

  • Allowing resourceLoader.find to locate “watertemp” (rather than “data/watertemp”). {warning}

findStyleFile / findOrCreateStyleFile / findStyleSldFile / findStyleSldFile regression

These methods determine a file location based on a StyleInfo and a filename. A few test cases such as DynamicDimensionsTest made use of a relative path when unpacking class path resources:

testData.addStyle(styleName, "../temperature.sld", getClass(), getCatalog());

This resulted in temperature.sld being unpacked outside the styles directory:

styles/
temperature.sld

And a relative path coming out of StyleInfo:

String path = Paths.path("styles",style.getFilename());
  • Changed testData add style to ignore relative paths when determining the target filename.
  • Added Paths.convert ( path, filename ) to allow for relative filenames (as long as they stay inside the data directory)

org.vfny.geoserver.global.GeoserverDataDirectory

Action: refactor this one out of existence!

It is mostly used to access the data directory:

File dataDir = GeoserverDataDirectory.getGeoserverDataDirectory();
File storeDir = new File(dataDir, "data/wcs/mosaicfordelete");

It is also used to resolve file references:

if (path.startsWith("file:")) {
   File fixedPath = GeoserverDataDirectory.findDataFile(path);
   entry.setValue(DataUtilities.fileToURL(fixedPath).toExternalForm());
}

  • Replace each reference to GeoserverDataDirectory with GeoServerResoruceLoader

  • Replace GeoserverDataDirectory.getResourceLoader () with GeoServerExtensions.bean (GeoServerResource)

  • Feedback: Make GeoServerExtensions easier to use

    • Introduced GeoServerExtensionsHelper

GeoServerSecurityManager

  • Update internals of GeoServerSecurityManager to use Resource

  • Add Resource access methods to GeoServerSecurityManager

  • Transition GeoServerSecurityManager direct file access with Resource use where possible

  • Checklist (106 references)

    • SecurityManager.getSecurityRoot () - 18 items
    • getRoleRoot () - 14 items
    • getPasswordPolicyRoot - 3 items
    • getUserGroupRoot - 13 items
    • getAuthRoot - 3 items
    • getFilterRoot - 5 items
    • getRoot - 3 items
    • getMasterPasswordProviderRoot - 5 items

File Events

  • Platform FileWatcher
  • Security FileWatcher
  • ⚠️ GeoServerTemplateLoader - looks to be complicated

©2020 Open Source Geospatial Foundation

Home
Proposals
Release Schedule

Clone this wiki locally