OpenQuake Engine 3.3.0

[Graeme Weatherill (@g-weatherill)]

• Adds GMPE suite for national PSHA for Germany

[Daniele Viganò (@daniviga)]

• Added a warning box when an unsupported browser is used to view the WebUI
• Updated Docker containers to support a multi-node deployment
  with a shared directory
• Moved the Docker containers source code from oq-builders
• Updated the documentation related to the shared directory
  which is now mandatory for multi-node deployments

[Matteo Nastasi (@nastasi-oq)]

• Removed tests folders

[Stéphane Drouet (@stephane-on)]

• Added Drouet & Cotton (2015) GMPE including 2017 erratum

[Michele Simionato (@micheles)]

• Optimized the memory occupation in classical calculations (Context.poe_map)
• Fixed a wrong counting of the ruptures in split fault sources with
  an hypo_list/slip_list causing the calculation to fail
• Made the export of uniform hazard spectra fast
• Made the std hazard output properly exportable
• Replaced the ~ in the header of the UHS csv files with a -
• Restored the individual_curves flag even for the hazard curves
• Implemented dGMPE weights per intensity measure type
• Extended --reuse-hazard to all calculators
• Fixed a bug in event_based_risk from GMFs with coefficients of variations

[Graeme Weatherill (@g-weatherill)]

• Adds magnitude scaling relation for Germany

[Michele Simionato (@micheles)]

• Used floats for the the GSIM realization weights, not Python Decimals
• Added a flag fast_sampling, by default False
• Added an API /extract/src_loss_table/<loss_type>
• Removed the rupture filtering from sample_ruptures and optimized it in
  the RuptureGetter by making use of the bounding box
• Raised the limit on ses_per_logic_tree_path from 216 to 232;
• Added a parameter max_num_sites to increase the number of sites accepted
  by an event based calculation up to 2 ** 32 (the default is still 2 ** 16)
• Added a command oq compare to compare hazard curves and maps within
  calculations
• Extended the engine to read transparently zipped source models and exposures
• Restored the check for invalid source IDs in applyToSources
• Extended the command oq zip to zip source models and exposures
• Parallelized the associations event ID -> realization ID
• Improved the message when assets are discarded in scenario calculations
• Implemented aggregation by multiple tags, plus a special case for the
  country code in event based risk

[Marco Pagani (@mmpagani)]

• Added two modified versions of the Bindi et al. (2011) to be used in a
  backbone approach to compute hazard in Italy
• Added a modified version of Berge-Thierry et al. 2003 supporting Mw

[Michele Simionato (@micheles)]

• Changed the way loss curves and loss maps are stored in order to unify
  the aggregation logic with the one used for the average losses
• Now it is possible to compute the ruptures without specifying the sites
• Added an early check for the case of missing intensity measure types
• Deprecated the case of exposure, site model and region_grid_spacing all
  set at the same time
• Implemented multi-exposure functionality in event based risk
• Changed the event based calculator to store the ruptures incrementally
  without keeping them all in memory
• Refactored the UCERF event based calculator to work as much as possible
  the regular calculator
• Optimized the management and storage of the aggregate losses in the event
  based risk calculation; also, reduced the memory consumption
• Changed the default for individual_curves to "false", which is the right
  default for large calculations
• Optimized the saving of the events
• Removed the save_ruptures flag in the job.ini since ruptures must be saved
  always
• Optimized the rupture generation in case of sampling and changed the
  algorithm and seeds
• Fixed a bug with the IMT SA(1) considered different from SA(1.0)
• Removed the long-time deprecated GMF exporter in XML format for event_based
• Added a re-use hazard feature in event_based_risk in single-file mode
• Made the event ID unique also in scenario calculations with
  multiple realizations
• Removed the annoying hidden .zip archives littering the export directory
• Added an easy way to read the exposure header
• Added a way to run Python scripts using the engine libraries via oq shell
• Improved the minimum_magnitude feature
• Fixed the check on missing hazard IMTs
• Reduced substantially the memory occupation in event based risk
• Added the option spatial_correlation=no correlation for risk calculations
  from ShakeMaps
• Removed the experimental calculator ucerf_risk
• Optimized the sampling of time-independent sources for the case of
  prefilter_sources=no
• Changed the algorithm associating events to SESs and made the event based
  hazard calculator faster in the case of many SESs
• Reduced substantially the memory consumption in event based risk
• Made it possible to read multiple site model files in the same calculation
• Implemented a smart single job.ini file mode for event based risk
• Now warnings for invalid parameters are logged in the database too
• Fixed oq export avg_losses-stats for the case of one realization
• Added oq export losses_by_tag and oq export curves_by_tag
• Extended oq export to work in a multi-user situation
• Forbidden event based calculations with more than max_potential_paths
  in the case of full enumeration
• Saved a large amount of memory in event_based_risk calculations
• Added a command oq export losses_by_tag/<tagname> <calc_id>
• Extended oq zip to zip the risk files together with the hazard files
• Changed the building convention for the event IDs and made them unique
  in the event loss table, even in the case of full enumeration
• Optimized the splitting of complex fault sources
• Fixed the ShakeMap download procedure for uncertainty.zip archives
  with an incorrect structure (for instance for ci3031111)
• Disabled the spatial correlation in risk-from-ShakeMap by default
• Optimized the rupture sampling where there is a large number of SESs
• Extended the reqv feature to multiple tectonic region types and
  removed the spinning/floating for the TRTs using the feature
• Reduced the GMPE logic tree upfront for TRTs missing in the source model
• Fixed the ShakeMap downloader to use the USGS GeoJSON feed
• Improved the error message when there are more than 65536 distinct tags
  in the exposure
• Turned vs30measured into an optional parameter

[Chris Van Houtte (@cvanhoutte)]

• Added siteclass as a site parameter, and reference_site_class as
  a site parameter than can be specified by the user in the ini file
• Added new classes to mcverry_2006.py to take siteclass as a predictor
• Updated comments in mcverry_2006.py
• Added new mcverry_2006 test tables to account for difference in site
  parameter
• Added qa_test_data classical case_32

[Michele Simionato (@micheles)]

• Fixed the rupture exporter for Canada
• Extended the oq prepare_site_model to optionally generate the
  fields z1pt0, z2pt5 and vs30measured
• It is now an error to specify both the sites and the site model in the
  job.ini, to avoid confusion with the precedency
• Implemented a reader for site models in CSV format
• Made the export_dir relative to the input directory
• Better error message for ShakeMaps with zero stddev
• Added a source_id-filtering feature in the job.ini
• Added a check on non-homogeneous tectonic region types in a source group
• Fixed the option oq engine --config-file that broke a few releases ago
• Replaced nodal_dist_collapsing_distance and
  hypo_dist_collapsing_distance with pointsource_distance and made
  use of them in the classical and event based calculators

[Graeme Weatherill (@g-weatherill)]

• Fixes to hmtk completeness tables for consistent rates and addition of
  more special methods to catalogue

[Michele Simionato (@micheles)]

• Restricted ChiouYoungs2008SWISS01 to StdDev.TOTAL to avoid a bug
  when computing the GMFs with inter/intra stddevs
• Raised an error if assets are discarded because too far from the hazard
  sites (before it was just a warning)
• Added an attribute .srcidx to every event based rupture and stored it
• Fixed an issue with the Byte Order Mark (BOM) for CSV exposures prepared
  with Microsoft Excel
• Reduced the site collection instead of just filtering it; this fixes
  a source filtering bug and changes the numbers in case of GMF-correlation
• Added a command oq prepare_site_model to prepare a sites.csv file
  containing the vs30 and changed the engine to use it
• Added a cutoff when storing a PoE=1 from a CSV file, thus avoiding NaNs
  in classical_damage calculations
• Reduced the data transfer in the risk model by only considering the
  taxonomies relevant for the exposure
• Extended oq engine --run to accept a list of files
• Optimized the saving of the risk results in event based in the case of
  many sites and changed the command oq show portfolio_loss to show
  mean and standard deviation of the portfolio loss for each loss type

[Marco Pagani (@mmpagani)]

• Added a first and preliminary version of the GMM for the Canada model
  represented in an analytical form.
• Added a modified version of Atkinson and Macias to be used for the
  calculation of hazard in NSHMP2014.
• Added support for PGA to the Si and Midorikawa (1999).

[Michele Simionato (@micheles)]

• Made it possible to run the risk over an hazard calculation of another user
• Worked around the OverflowError: cannot serialize a bytes object larger
  than 4 GiB in event based calculations
• Started using Python 3.6 features
• Fixed the check on vulnerability function ID uniqueness for NRML 0.5
• Ruptures and GMFs are now computed concurrently, thus mitigating the
  issue of slow tas...

OpenQuake Engine 3.2.0

[Matteo Nastasi (@nastasi-oq)]

• specified 'amd64' as the only architecture supported by ubuntu packages

[Michele Simionato (@micheles)]

• Changed the source writer: now the srcs_weights are written in the XML
  file only if they are nontrivial
• Changed the algorithm assigning the seeds: they are now generated before
  the source splitting; also, a seed-related bug in the splitting was fixed
• For event based, moved the rupture generation in the prefiltering phase

[Daniele Viganò (@daniviga)]

• Fixed a bug with CTRL-C when using the processpool distribution

[Robin Gee (@rcgee)]

• Raised the source ID length limit in the validation from 60 to 75 characters
  to allow sources with longer IDs

[Michele Simionato (@micheles)]

• Introduced a multi_node flag in openquake.cfg and used it to
  fully parallelize the prefiltering in a cluster
• Used the rupture seed as rupture ID in event based calculations
• Changed the deprecation mechanism of GSIMs to use a class attribute
  superseded_by=NewGsimClass
• Solved the pickling bug in event based hazard by using generator tasks
• Improved the distribution of the risk tasks by changing the weight

[Pablo Heresi (@pheresi)]

• Contributed the HM2018CorrelationModel

[Michele Simionato (@micheles)]

• Restored the individual_curves flag that for the moment is used for the
  risk curves
• Introduced two experimental new parameters floating_distance and
  spinning_distance to reduce hypocenter distributions and nodal plane
  distributions of ruptures over the corresponding distances
• Optimized the parsing of the logic tree when there is no "applyToSources"
• Made the IMT classes extensible in client code
• Reduced the hazard maps from 64 to 32 bit, to be consistent with the
  hazard curves and to reduce by half the download time

[Graeme Weatherill (@g-weatherill)]

• Implements a fix of Montalva et al (2016) for new coefficients (now
  Montalva et al. (2017))

[Michele Simionato (@micheles)]

• Parallelized the reading of the source models
• Optimized oq info --report by not splitting the sources in that case
• Speedup the download of the hazard curves, maps and uhs
• Honored concurrent_tasks in the prefiltering phase too
• It is now legal to compute uniform hazard spectra for a single period
• Added command oq plot_memory
• Introduced a MultiGMPE concept
• Saved the size of the datastore in the database and used it in the WebUI

[Graeme Weatherill (@g-weatherill)]

• Adds geotechnical related IMTs

[Michele Simionato (@micheles)]

• Renamed /extract/agglosses -> /extract/agg_losses and same for aggdamages
• Supported equivalent epicentral distance with a reqv_hdf5 file
• Fixed the risk from ShakeMap feature in the case of missing IMTs
• Changed the way gmf_data/indices and ruptures are stored
• Added experimental support for dask
• Added 11 new site parameters for geotechnic hazard
• Changed the SiteCollection to store only the parameters required by the
  GSIMs

[Robin Gee (@rcgee)]

• The number of sites is now an argument in the method _get_stddevs()
  in the GMPE of Kanno, 2006

[Michele Simionato (@micheles)]

• Changed the serialization of ruptures to HDF5: the geometries are now
  stored in a different dataset
• Bug fix: the asset->site association was performed even when not needed
• Made it possible to serialize to .hdf5 multipoint sources and
  nonparametric gridded sources
• Added a check on source model logic tree files: the uncertaintyModel
  values cannot be repeated in the same branchset
• Added a flag std_hazard_curves; by setting it to true the user can
  compute the standard deviation of the hazard curves across realizations

[Marco Pagani (@mmpagani)]

• Added Thingbaijam et al. (2017) magnitude-scaling relationship

[Michele Simionato (@micheles)]

• Added an /extract/ API for event_based_mfd
• Fixed a bug in the classical_damage calculators: multiple loss types
  were not treated correctly

[Marco Pagani (@mmpagani)]

• Adding tests to the method computing decimal time

[Michele Simionato (@micheles)]

• Removed the event_based_rupture calculator and three others
• Added a field size_mb to the output table in the database and made
  it visible in the WebUI as a tooltip
• Added a command oq check_input job.ini to check the input files
• Made the loss curves and maps outputs from an event based risk calculation
  visible to the engine and the WebUI (only the stats)
• Added a check on duplicated branchIDs in GMPE logic trees

[Daniele Viganò (@daniviga)]

• Fixed a bug when reading exposure with utf8 names on systems with non-utf8
  terminals (Windows)
• Changed the openquake.cfg file and added a dbserver.listen parameter
• Added the hostname in the WebUI page. It can be customize by the user
  via the local_settings.py file

[Michele Simionato (@micheles)]

• Added a Content-Length to the outputs downloadable from the WebUI
• Fixed a bug when extracting gmf_data from a hazard calculation with a
  filtered site collection
• Stored an attributed events.max_gmf_size
• Added a check on exposures with missing loss types
• Added a LargeExposureGrid error to protect the user by tricky exposures
  (i.e. France with assets in the Antilles)
• Changed the event_based_risk calculator to compute the loss curves and
  maps directly; removed the asset_loss_table
• Changed the event_based_risk calculator to distribute by GMFs always
• Optimized the memory consumption in the UCERF classical calculator
• Added a parameter minimum_magnitude in the job.ini
• Added an utility utils/combine_mean_curves.py

OpenQuake Engine 3.1.0

[Marco Pagani (@mmpagani) and Changlong Li (@mstlgzfdh)]

• Added a version of the Yu et al. (2013) GMPE supporting Mw

[Michele Simionato (@micheles)]

• Reduced the data transfer in the UCERF calculators
• Stored the zipped input files in the datastore for reproducibility
• Fixed a regression when reading GMFs from an XML in absence of a sites.csv
  file

[Robin Gee (@rcgee)]

• Extend oq to_shapefile method to also work with YoungsCoppersmithMFD
  and arbitraryMFD MFD typologies.

[Michele Simionato (@micheles)]

• Now the hazard statistics can be computed efficiently even in a single
  calculation, i.e. without the --hc option
• Added a check on the Python version in the oq command
• Reduced the data transfer when sending the site collection
• Changed the default filter_distance

[Daniele Viganò (@daniviga)]

• Fixed a bug where the PID was not saved into the database
  when using the command line interface
• Made it impossible to fire multiple CTRL-C in sequence
  to allow processes teardown and tasks revocation when Celery is used

[Michele Simionato (@micheles)]

• Used scipy.spatial.distance.cdist in Mesh.get_min_distance
• Prefiltered sites and assets in scenario calculations
• Made it possible to specify the filter_distance in the job.ini
• Made rtree optional again and disabled it in macOS
• Optimized the SiteCollection class and doubled the speed of distance
  calculations in most continental scale calculations
• Fixed an ordering bug in event based risk from GMFs when using a
  vulnerability function with PMF
• Replaced Rtree with KDtree except in the source filtering
• Parallelized the source prefiltering
• Removed the tiling feature from the classical calculator
• Undeprecated hazardlib.calc.stochastic.stochastic_event_set and
  made its signature right
• Removed the source typology from the ruptures and reduced the rupture
  hierarchy
• Removed the mesh spacing from PlanarSurfaces
• Optimized the instantiation of the rtree index
• Replaced the old prefiltering mechanism with the new one

[Daniele Viganò (@daniviga)]

• Managed the case of a dead controlling terminal (SIGHUP)

[Michele Simionato (@micheles)]

• Removed Decimal numbers from the PMF distribution in hazardlib
• Fixed another tricky bug with rtree filtering across the international
  date line
• Added a parameter prefilter_sources with values rtree|numpy|no
• Removed the prefiltering on the workers, resulting in a huge speedup
  for gridded ruptures at the cost of a larger data transfer
• Changed the losses_by_event output to export a single .csv file with
  all realizations
• Added a cross_correlation parameter used when working with shakemaps
• Now sites and exposure can be set at the same time in the job.ini
• Introduced a preclassical calculator
• Extended the scenario_damage calculator to export dmg_by_event
  outputs as well as losses_by_event outputs if there is a consequence
  model
• Unified region and region_constraint parameters in the job.ini
• Added a check to forbid duplicated GSIMs in the logic tree
• Introduced some changes to the realizations exporter (renamed field
  uid -> branch_path and removed the model field)
• Added a command oq celery inspect
• Reduced the check on too many realizations to a warning, except for
  event based calculations
• Improved the hazard exporter to exports only data for the filtered
  site collection and not the full site collection
• Extended the BCR exporter to export the asset tags

[Catalina Yepes (@CatalinaYepes)]

• Revised/enhanced the risk demos

[Michele Simionato (@micheles)]

• Added a warning about the option optimize_same_id_sources when the user
  should take advantage of it

[Daniele Viganò (@daniviga)]

• celery-status script converted into oq celery status command
• Removed Django < 1.10 backward compatibility
• Updated Python dependices (numpy 1.14, scipy 1.0.1,
  Django 1.10+, Celery 4+)

[Michele Simionato (@micheles)]

• Implemented scenario_risk/scenario_damage from shakemap calculators
• Exported the asset tags in the asset based risk outputs
• Fixed a numeric issue for nonparametric sources causing the hazard curves
  to saturate at high intensities
• Added an utility to download shakemaps
• Added an XML exporter for the site model
• Slight change to the correlation module to fix a bug in the SMTK
• Added a distribution mechanism threadpool

OpenQuake Engine 3.0.1

[Michele Simionato (@micheles)]

• Fixed a numeric bug affecting nonparametric sources and causing saturating
  hazard curves at high intensity

OpenQuake Engine 3.0.0

[Michele Simionato (@micheles)]

• Fixed a bug with newlines in the logic tree path breaking the CSV exporter
  for the realizations output
• When setting the event year, each stochastic event set is now considered
  independent
• Fixed a bug in the HMTK plotting libraries and added the ability to
  customize the figure size
• Fixed bug in the datastore: now we automatically look for the attributes
  in the parent dataset, if the dataset is missing in the child datastore
• Extended extract_losses_by_asset to the event based risk calculator
• Stored in source_info the number of events generated per source
• Added a script utils/reduce_sm to reduce the source model of a calculation
  by removing all the sources not affecting the hazard
• Deprecated openquake.hazardlib.calc.stochastic.stochastic_event_set
• Fixed the export of ruptures with a GriddedSurface geometry
• Added a check for wrong or missing <occupancyPeriods> in the exposure
• Fixed the issue of slow tasks in event_based_risk from precomputed GMFs
  for sites without events
• Now the engine automatically associates the exposure to a grid if
  region_grid_spacing is given and the sites are not specified otherwise
• Extracting the site mesh from the exposure before looking at the site model
• Added a check on probs_occur summing up to 1 in the SourceWriter
• oq show job_info now shows the received data amount while the
  calculation is progressing

[Daniele Viganò (@daniviga)]

• Removed support for Python 2 in setup.py
• Removed files containing Python 2 dependencies
• Added support for WebUI groups/permissions on the
  export outputs and datastore API endpoints

[Michele Simionato (@micheles)]

• Fixed oq show for multiuser with parent calculations
• Fixed get_spherical_bounding_box for griddedSurfaces
• Implemented disaggregation by source only for the case
  of a single realization in the logic tree (experimental)
• Replaced celery with celery_zmq as distribution mechanism
• Extended oq info to work on source model logic tree files
• Added a check against duplicated fields in the exposure CSV
• Implemented event based with mutex sources (experimental)
• Add an utility to read XML shakemap files in hazardlib
• Added a check on IMTs for GMFs read from CSV

[Daniele Viganò (@daniviga)]

• Changed the default DbServer port in Linux packages from 1908 to 1907

[Michele Simionato (@micheles)]

• Logged rupture floating factor and rupture spinning factor
• Added an extract API for losses_by_asset
• Added a check against GMF csv files with more than one realization
• Fixed the algorithm setting the event year for event based with sampling
• Added a command oq importcalc to import a remote calculation in the
  local database
• Stored avg_losses-stats in event based risk if there are multiple
  realizations
• Better error message in case of overlapping sites in sites.csv
• Added a an investigation time attribute to source models with
  nonparametric sources
• Bug fix: in some cases the calculator event_based_rupture was generating
  too few tasks and the same happened for classical calculation with
  `optimize_same_id_sources=true
• Changed the ordering of the epsilons in scenario_risk
• Added the ability to use a pre-imported risk model
• Very small result values in scenario_damage (< 1E-7) are clipped to zero,
  to hide numerical artifacts
• Removed an obsolete PickleableSequence class
• Fixed error in classical_risk when num_statistics > num_realizations
• Fixed a TypeError when reading CSV exposures with occupancy periods
• Extended the check on duplicated source IDs to models in format NRML 0.5
• Added a warning when reading the sources if .count_ruptures() is
  suspiciously slow
• Changed the splitting logic: now all sources are split upfront
• Improved the splitting of complex fault sources
• Added a script to renumber source models with non-unique source IDs
• Made the datastore of calculations using GMPETables relocatable; in
  practice you can run the Canada model on a cluster, copy the .hdf5 on
  a laptop and do the postprocessing there, a feat previously impossible.

[Valerio Poggi (@klunk386)]

• Included a method to export data directly from the Catalogue() object into
  standard HMTK format.

[Michele Simionato (@micheles)]

• Now the parameter disagg_outputs is honored, i.e. only the specified
  outputs are extracted from the disaggregation matrix and stored
• Implemented statistical disaggregation outputs (experimental)
• Fixed a small bug: we were reading the source model twice in disaggregation
• Added a check to discard results coming from the wrong calculation
  for the distribution mode celery_zmq
• Removed the long time deprecated commands
  oq engine --run-hazard and oq engine --run-risk
• Added a distribution mode celery_zmq
• Added the ability to use a preimported exposure in risk calculations
• Substantial cleanup of the parallelization framework
• Fixed a bug with nonparametric sources producing negative probabilities

OpenQuake Engine 2.9.0

[Michele Simionato (@micheles)]

• Deprecated the NRML format for the GMFs

[Matteo Nastasi (@nastasi-oq)]

• Debian package moved to Python 3.5

[Graeme Weatherill (@g-weatherill)]

• Small bug fix for Derras et al (2014) GMPE when Rjb = 0.0

[Michele Simionato (@micheles)]

• Improved the .rst reports for classical calculations with tiling
• Reduced the data transfer in the event based risk calculator by
  reading the event IDs directly from the workers
• Integrated the gmf_ebrisk calculator inside the event based calculator
• Improved the weighting algorithm for the sources in the event based
  rupture calculator
• Improved error message for source model files declared as nrml/0.5 when
  they actually are nrml/0.4
• Optimized the classical_bcr calculator for the case of many realizations
• Extended the exposure CSV importer to manage the retrofitted field

[Marco Pagani (@mmpagani), Changlong Li]

• Adds the Yu et al. (2013) GMPEs

[Michele Simionato (@micheles)]

• Fixed a bug in the hazard outputs: they were displayed in the WebUI even
  if they were missing
• Implemented splitting of nonparametric sources

[Marco Pagani (@mmpagani)]

• Fixed the 'get_closest_points' method for the
  openquake.hazardlib.geo.surface.gridded.GriddedSurface class

[Michele Simionato (@micheles)]

• Now the source model paths are relative to the source model logic tree file
• Fixed an international date line bug when using rtree for prefiltering
• Deprecated nrml.parse, it is now called nrml.to_python
• Improved the task distribution by splitting the AreaSources upfront
  and by improving the weighting algorithm

[Daniele Viganò (@daniviga)]

• TMPDIR can be customized via the openquake.cfg file
• Updated dependencies compatibility in setup.py

[Michele Simionato (@micheles)]

• If the hazard is precomputed, setting the site_model_file,
  gsim_logic_tree_file or source_model_logic_tree_file gives a warning
• Removed the obsolete API /extract/qgis- and added extract/hcurves,
  extract/hmaps, extract/uhs for use with the QGIS plugin
• Removed the deprecated GeoJSON exporters
• Fixed a bug with oq engine --run job.ini --exports npz
• Fixed the ordering of the IMTs in hazardlib
• oq engine --delete-calculation now aborts the calculation first
• Added some documentation about how to access the datastore
• Introduced a minimum_distance for the GSIMs
• Fixed several small issues with the UCERF calculators; now ucerf_hazard
  can be used as a precalculator of gmf_ebrisk
• Initial support for disaggregation by source
• Added the ability to import large exposures as CSV (experimental)
• Changed the source weights to be proportional to the number of GSIMs
  relevant for the tectonic region type, thus improving the task distribution

[Daniele Viganò (@daniviga)]

• The RPM python3-oq-engine package replaced python-oq-engine
• RPM packages moved to Python 3.5

[Michele Simionato (@micheles)]

• Added the ability to dump a specific calculation
• Changed the signature of the extract command to oq extract what calc_id,
  where what is a path info plus query string;

[Graeme Weatherill (@g-weatherill)]

• Implements significant duration GMPEs of Bommer et al. (2009) and Afshari &
  Stewart (2016)
• Adds significant duration IMT definitions to support IMTs

[Michele Simionato (@micheles)]

• Run the DbServer as a detached process
• Improved the test coverage for event based with GMF correlation
• Optimized the event based risk calculator from ruptures: now the ruptures
  are instantiated in the workers and not in the controller if possible
• Exported the parameter ses_per_logic_tree_path in the ruptures.csv file
• Improved the display names for the risk outputs
• Added a /v1/:calc_id/abort route to the engine server and Abort buttons
  to the WebUI
• Fixed the seeds properly in the case of vulnerability functions with PMFs:
  now even if the ground motion fields are all equal, the risk numbers
  will be different since there is a different seed per each field
• Stored a rupture loss table in event based risk calculations
• Made sure that the number of effective ruptures is stored in csm_info
• Fixed the HMTK tests to work with numpy from 1.11 to 1.14
• Added a command oq shell to open an embedded (I)Python shell
• Turned the 'realizations' output into a dynamic output

[Matteo Nastasi (@nastasi-oq)]

• Split package from python-oq-engine to python-oq-engine,
  python-oq-engine-master and python-oq-engine-worker
• Implemented an API /v1/on_same_fs to check filesystem accessibility
  between engine and a client application

[Michele Simionato (@micheles)]

• Reduced the data transfer when computing the hazard curves in postprocessing
• Removed the FilteredSiteCollection class

[Nick Ackerley (@nackerley)]

• Some improvements to the plotting routines of the HMTK

[Michele Simionato (@micheles)]

• Removed the ability to run oq engine --run job_h.ini,job_r.ini
• Forbidden the site model in gmf_ebrisk calculations
• When the option --hc is given the ruptures can now be read directly
  from the workers, thus saving some startup time
• Optimized the storage of the ruptures: the site IDs are not saved anymore
• Added a check for missing risk_investigation_time
• Reduced the data transfer in the gmf_ebrisk calculator
• Now the gmf_ebrisk calculator builds the aggregate loss curves too
• Extended the gmf_ebrisk calculator to use the GMFs produced by an event
  based hazard calculation, both via CSV and via the --hc option
• Fixed a performance bug in the computations of the aggregate loss curves
  by reading the full event loss table at once
• Fixed oq zip to work with gmf_ebrisk calculations
• Fixed a serious bug in the gmf_ebrisk calculator: the results were in most
  cases wrong and dependent on the number of spawned tasks
• Now the master_seed controls the generation of the epsilons in all
  situations (before in event_based_risk without asset_correlation it was
  managed by random_seed)
• Changed the management of the epsilons in the gmf_ebrisk calculator to
  be the same as in the event_based_risk calculator
• Added a check on the input files: the listed paths must be relative paths
• Fixed a bug when storing the disagg-bins in the case the lenghts of the
  arrays are not the same for all sites
• In the case of a single tile the prefiltering was applied twice: fixed the
  problem and generally improved the filtering/weighting of the sources
• Fixed the CSV exporter for disaggregation outputs when iml_disagg is set

[Graeme Weatherill (@g-weatherill)]

• Fixed ASK14 GMPE behaviour to remove ValueError
• Implements comprehensive suite of NGA East ground motion models for
  central and eastern United States
• Minor modification of check_gsim functions to permit instantiated classes
  to be passed

OpenQuake Engine 2.8.0

[Michele Simionato (@micheles)]

• iml_disagg and poes_disagg cannot coexist in the job.ini file
• Added a check on conditional_loss_poes in the event_based_risk calculator:
  now it requires asset_loss_table to be set

[Anirudh Rao (@raoanirudh)]

• Sorted taxonomies before comparison in the BCR calculator

[Michele Simionato (@micheles)]

• Optimized the disaggregation calculation by performing the PMF extraction
  only once at the end of the calculation and not in the workers
• Added an oq zip command
• Avoided running an useless classical calculation if iml_disagg is given

[Valerio Poggi (@klunk386)]

• Implemented subclasses for ZhaoEtAl2006Asc and AtkinsonBoore2006 to account
  for the distance filter used by SGS in their PSHA model for Saudi Arabia.
  Distance threshold is hard coded to 5km in this implementation.

[Michele Simionato (@micheles)]

• Added a warning if the aggregated probability of exceedence (poe_agg) in
  a disaggregation output is very dissimilar from poes_disagg
• Removed the flag split_sources
• Optimized the operation arrange_data_in_bins in the disaggregation
  calculator and reduced the data transfer by the number of tectonic
  region types
• Improved the sending of the sources to the workers, especially for the
  MultiPointSources
• Better error message if the user sets a wrong site_id in the sites.csv file
• Now the distance and lon lat bins for disaggregation are built directly
  from the integration distance
• Used uniform bins for disaggregation (before they were potentially
  different across realizations / source models)
• Improved the error message if the user forgets both sites and sites.csv
  in a calculation starting from predetermined GMFs
• Improved the error message if the user specifies a non-existing file in
  the job.ini
• Change the ordering of the TRT bins in disaggregation: now they are
  ordered lexicographically
• Added more validity checks on the job.ini file for disaggregation
• Changed the .hdf5 format generated by oq extract -1 hazard/rlzs; you can
  still produce the old format by using oq extract -1 qgis-hazard/rlzs
• Optimized the disaggregation calculator by instantiating
  scipy.stats.truncnorm only once per task and not once per rupture
• Optimized the disaggregation calculator when iml_disagg is specified,
  by caching duplicated results
• Made sure that oq dbserver stop also stops the zmq workers if the zmq
  distribution is enabled
• Added a check when disaggregating for a PoE too big for the source model
• If iml_disagg is given, forbid intensity_measure_types_and_levels
• Fixed the disaggregation outputs when iml_disagg is given: the PoE has
  been removed by the name of the output and correct PoE is in the XML file
• Fixed oq export loss_curves/rlzs for event_based_risk/case_master
• Removed the obsolete parameter loss_curve_resolution
• Fixed a Python 3 unicode error with oq engine --run job.zip
• Added a command oq abort <calc_id>
• Stored the avg_losses in classical risk in the same way as in
  event_based_risk and made them exportable with the same format
• Removed the outputs losses_by_tag from the event based risk calculators
  and changed the default for the avg_losses flag to True
• WebUI: now every job runs in its own process and has name oq-job-
• Refactored the WebUI tests to use the DbServer and django.test.Client
• Added an experimental feature optimize_same_id_sources
• Fixed a bug in gmf_ebrisk when there are zero losses and added more
  validity checks on the CSV file
• The parameter number_of_ground_motion_fields is back to being optional in
  scenario calculators reading the GMFs from a file, since it can be inferred
• Removed the deprecated risk outputs dmg_by_tag, dmg_total,
  losses_by_tag, losses_total
• Deprecated the .geojson exporters for hazard curves and maps
• We now keep the realization weights in case of logic tree sampling (before
  they were rescaled to 1 / R and considered all equals)
• Optimized sampling for extra-large logic trees
• Added a check on missing source_model_logic_tree
• Fix to the gmf_ebrisk calculator: the realization index in the event loss
  table was incorrect and too many rows were saved
• Added a way to restrict the source logic model tree by setting a sm_lt_path
  variable in the job.ini (experimental)
• Fixed the precedence order when reading openquake.cfd

OpenQuake Engine 2.7.0

[Michele Simionato (@micheles)]

• Fixed the risk exporters for tags containing non-ASCII characters

[Valerio Poggi (@klunk386)]

• Implemented the Pankow and Pechmann 2004 GMPE (not verified)

[Graeme Weatherill (@g-weatherill)]

• Added Derras et al. (2014) GMPE
• Implemented the Zhao et al. (2016) GMPE for active shallow crustal,
  subduction interface and subduction slab events
• Adds 'rvolc' (volcanic path distance) to the distance context

[Michele Simionato (@micheles)]

• The outputs loss_curves-rlzs and loss_curves-stats are now visible again
  as engine outputs (before they were hidden)
• Added a debug command oq plot_assets and fixed oq plot_sites
• Added a flag --multipoint to the command oq upgrade_nrml
• Deprecated several outputs: hcurves-rlzs, agg_loss_table, losses_total,
  dmg_by_tag, dmg_total, losses_by_tag, losses_by_tag-rlzs
• Extended the command oq extract job_id to check the database
• Optimized the scenario damage calculator by vectorizing the calculation
  of the damage states
• Extended the FragilityFunctions to accept sequences/arrays of intensity
  levels, as requested by Hyeuk Ryu

[Daniele Viganò (@daniviga)]

• Added support for groups in the WebUI/API server

[Michele Simionato (@micheles)]

• Added an experimental distribution mode of kind "zmq"
• Implemented an API /extract/agglosses/loss_type?tagname1=tagvalue1&...
  with the ability to select all tagvalues (*) for a given tagname
• Deprecated reading hazard curves from CSV, since it was an experimental
  features and nobody is using it
• Changed the exporter of the event loss table to export all realizations
  into a single file

[Graeme Weatherill (@g-weatherill)]

• Adds the Bindi et al. (2017) GMPEs for Joyner-Boore and Hypocentral
  Distance

[Michele Simionato (@micheles)]

• Made it mandatory to specify the sites for all calculators reading the
  GMFs from a CSV file
• Tested also the case of calculators requiring a shared_dir
• Improved the error checking when parsing vulnerability functions with PMF

[Daniele Viganò (@daniviga)]

• Fixed a bug in oq reset command which was not stopping
  the DbServer properly

[Michele Simionato (@micheles)]

• Implemented an API /extract/aggcurves/loss_type?tagname1=tagvalue1&...
• Implemented an API /extract/aggdamages/loss_type?tagname1=tagvalue1&...
• Every time a new calculation starts, we check if there is a newer version
  of the engine available on GitHub
• Changed the search logic for the configuration file openquake.cfg
• Implemented an API /extract/agglosses/loss_type?tagname1=tagvalue1&...
• Fixed several bugs in the gmf_ebrisk calculator, in particular in presence
  of asset correlation and missing values on some sites
• Fixed a bug with logging configured a WARN level instead of INFO level
  if rtree was missing (affecting only oq run)
• Changed the ScenarioDamage demo to use two GSIMs
• Added a node <tagNames> in the exposure
• Added a web API to extract the attributes of a datastore object
• Fixed oq to_shapefile and oq from_shapefile to work with NRML 0.5
  (except MultiPointSources)
• Added information about the loss units to the agg_curve outputs
• oq extract hazard/rlzs now extracts one realization at the time
• The rupture filtering is now applied during disaggregation
• Changed the /extract wen API to return a compressed .npz file
• Fixed a bug with multi-realization disaggregation, celery and no
  shared_dir: now the calculation does not hang anymore

OpenQuake Engine 2.6.0

[Michele Simionato (@micheles)]

• Fixed the GMF .npz export when the GMFs are extracted from a file
• Stored the number of nonzero losses per asset and realization in
  event_based_risk calculations with asset_loss_table=True

[Daniele Viganò (@daniviga)]

• Fixed 'openquake' user creation in RPM when SELinux is in enforcing mode
• Changed the behaviour during RPM upgrades:
  the old openquake.cfg configuration file is left untouched and the new one
  installed as openquake.cfg.rpmnew

[Michele Simionato (@micheles)]

• Added a check on number_of_ground_motion_fields when the GMFs are
  extracted from a NRML file
• Added a command oq extract able to extract hazard outputs into HDF5 files
• Fixed a bug when reading GMFs from a NRML file: the hazard sites were
  read from the exposure (incorrectly) and not from the GMFs
• Fixed a bug in MultiMFDs of kind arbitraryMFD

[Valerio Poggi (@klunk386)]

• Implemented the Atkinson (2010) GMPE as subclass Atkinson2010Hawaii
  of BooreAtkinson2008

[Michele Simionato (@micheles)]

• Used the new loss curves algorithm for the asset loss curves and loss maps
• Added a generic extract functionality to the web API
• Fixed a bug when computing the rjb distances with multidimensional meshes
• Changed the GMF CSV exporter to export the sites too; unified it with the
  event based one

[Daniele Viganò (@daniviga)]

• Changed the 'CTRL-C' behaviour to make sure that all children
  processes are killed when a calculation in interrupted

[Michele Simionato (@micheles)]

• Fixed a bug in the statistical loss curves exporter for classical_risk
• Replaced the agg_curve outputs with losses by return period outputs
• Turned the DbServer into a multi-threaded server
• Used zmq in the DbServer
• Fixed correct_complex_sources.py
• Fixed oq export hcurves-rlzs -e hdf5
• Changed the source weighting algorithm: now it is proportional to the
  the number of affected sites
• Added a command oq show dupl_sources and enhances oq info job.ini
  to display information about the duplicated sources
• Added a flag split_sources in the job.ini (default False)
• Updated the demos to the format NRML 0.5

[Valerio Poggi (@klunk386)]

• Implemented the Munson and Thurber 1997 (Volcanic) GMPE

[Graeme Weatherill (@g-weatherill)]

• Adapts CoeffsTable to be instantiated with dictionaries as well as strings

[Daniele Viganò (@daniviga)]

• Extended the 'oq reset' command to work on multi user installations

[Michele Simionato (@micheles)]

• Fixed a bug: if there are multiple realizations and no hazard stats,
  it is an error to set hazard_maps=true or uniform_hazard_spectra=true
• Implemented aggregation by asset tag in the risk calculators
• Fixed a small bug in the HMTK (in get_depth_pmf)
• Extended the demo LogicTreeCase1ClassicalPSHA to two IMTs and points
• Added a documentation page oq-commands.md
• Removed the automatic gunzip functionality and added an automatic
  checksum functionality plus an oq checksum command
• Made the demo LogicTreeCase2ClassicalPSHA faster
• Fixed the export by realization of the hazard outputs
• Changed the generation of loss_maps in event based risk, without the option
  --hc: now it is done in parallel, except when reading the loss ratios
• Renamed --version-db to --db-version, to avoid
  confusions between oq --version and oq engine -version
• Fixed bug in the exported outputs: a calculation cannot export the results
  of its parent
• Extended the sz field in the rupture surface to 2 bytes, making it
  possible to use a smaller mesh spacing
• Changed the ordering of the fields in the loss curves and loss maps
  generated by the event based risk calculator; now the insured fields
  are at the end, before they were intermixed with each loss type
• Changed the format of array all_loss_ratios/indices
• The size in bytes of the GMFs was saved incorrectly
• Added an exporter gmf_scenario/rup-XXX working also for event based
• First version of the calculator gmf_ebrisk
• Implemented risk statistics for the classical_damage calculator
• Added a .csv importer for the ground motion fields
• Implemented risk statistics for the classical_bcr calculator

[Armando Scarpati (@hascar)]

• Show to the user the error message when deleting a calculation
  in the WebUI fails

[Michele Simionato (@micheles)]

• Better error message when running a risk file in absence of hazard
  calculation
• Changed the sampling logic in event based calculators
• Imported GMFs from external file into the datastore

[Daniele Viganò (@daniviga)]

• Added the 'celery-status' script in 'utils' to check the
  task distribution in a multi-node celery setup

[Michele Simionato (@micheles)]

• Removed an excessive check from the WebUI: now if an output exists,
  it can be downloaded even if the calculation was not successful

[Armando Scarpati (@hascar)]

• Visualized the calculation_mode in the WebUI

[Michele Simionato (@micheles)]

• Made the upgrade_manager transactional again
• Changed the storage of the GMFs; as a consequence the exported .csv
  has a different format

[Daniele Viganò (@daniviga)]

• Fixed a bug introduced by a change in Django 1.10 that was causing
  the HTTP requests log to be caught by our logging system and
  then saved in the DbServer
• Updated requirements to allow installation of Django 1.11 (LTS)

[Michele Simionato (@micheles)]

• Added two commands oq dump and oq restore
• Added a check that on the number of intensity measure types when
  generating uniform hazard spectra (must be > 1)

OpenQuake Engine 2.5.0

[Armando Scarpati (@hascar)]

• Added a confirmation dialog when trying to remove a calculation via the
  WebUI

[Michele Simionato (@micheles)]

• Hazard maps were not exposed to the engine in event based calculations
• Fixed the check on the DbServer instance: it was failing in presence
  of symbolic links
• Optimized MultiMFD objects for the case of homogeneous parameters
• Added an .npz exporter for the scenario_damage output dmg_by_asset
• Removed the pickled CompositeSourceModel from the datastore
• Improved the error message when the rupture mesh spacing is too small
• Unified the versions of baselib, hazardlib and engine
• Raised a clear error if the user does not set the calculation_mode
• Made it is possible to pass the hdf5 full path to the DataStore class
• Made it possible to use CELERY_RESULT_BACKEND != 'rpc://'

[Michele Simionato (@micheles), Daniele Viganò (@daniviga)]

• Merged the oq-hazardlib repository into oq-engine.
  The python-oq-hazardlib package is now provided by python-oq-engine

[Michele Simionato (@micheles)]

• Added CSV exports for the agg_curve outputs
• Fixed a bug in oq export hcurves-rlzs --exports hdf5
• Restored the parameter sites_per_tile with a default of 20,000, i.e.
  tiling starts automatically if there are more than 20,000 sites
• Better error message for invalid exposures
• Removed the deprecated XML outputs of the risk calculators
• Added an end point v1/calc/XXX/oqparam to extract the calculation
  parameters as a JSON dictionary
• Fixed the excessive logic tree reduction in event based calculators
• Improved the command oq db
• Fixed an encoding bug when logging a filename with a non-ASCII character
• Fixed a bug when exporting a GMF with ruptureId=0
• Added a parameter disagg_outputs to specify the kind of disaggregation
  outputs to export
• Raised an early error if the consequence model is missing some taxonomies
• Restored the tiling functionality in the classical calculator; to enable
  it, set num_tiles in the job.ini file
• If there are no statistical hazard curves to compute, do not transfer
  anything
• Fixed a small bug in oq plot and added a test

[Daniele Viganò (@daniviga)]

• Added collectstatic and createsuperuser subcommands to the
  oq webui command
• Added a local_settings.py.pam template to use PAM as the authentication
  provider for API and WebUI
• Now the command oq webui start tries to open a browser tab
  with the WebUI loaded