Skip to content

Latest commit

 

History

History
104 lines (73 loc) · 7.4 KB

README.md

File metadata and controls

104 lines (73 loc) · 7.4 KB

Turbulence-Degraded-Characters

A imagery database for machine learning experiments, atmospheric turbulence characterization, and turbulence restoration research.

The TDC database is suitable for machine learning experiments, atmospheric turbulence characterization, and turbulence restoration research. The imagery is generated using the brightness function [1] method of turbulence simulation as implemented in the WONAT library. The imaging scenario is defined by a 100 mm aperture telescope imaging a 550 mm wide square field over a range of 1 km. The wavelength of the simulation is set to 550 nm. The sample spacing in the object plane is 2.75 mm corresponding to critical sampling at the diffraction limit of the optics. The phase screen model of homogeneous turbulence throughout this volume of atmosphere is in conformance with the Kolmorogov power spectrum. The imagery spans turbulence strengths from to corresponding to through where is the coherence diameter and is the refractive index structure parameter. This scenario was selected because it can be reproduced experimentally with widely available equipment.

The 62 target classes consist of characters in a single font (numbers, lowercase letters, uppercase letters - see original_character_images.zip). Each uppercase character is approximately 47 mm tall and the lowercase letters are somewhat shorter. 1000 samples are generated for each character for a total of 62,000 samples per turbulence level. A coarse correction for turbulence-induced global motion is applied to each sample to constrain the results on a 50x50 pixel grid. Examples of the 'E' character for are shown in the first figure below. The second Figure shows a single example of the 'E' character from each of the five levels of turbulence in the database. The database consists of five HDF5 files, corresponding to through . Within each HDF5 file, dataset labels correspond to target classes (e.g, 'E', 'e', '3', and so on).

Six examples of the 'E' character for in the D/r_0=2$ case.

Six examples of the 'E' character for in the case.

Examples of the 'E' character across $D/r_0 = 1$ through $5$.

Examples of the 'E' character across 1 through 5.

Bulk Data Access - Amazon S3

The Turbulence Degraded Characters data is available from Amazon Simple Storage Service (S3) in Requester Pays buckets (i.e., you are charged by Amazon to access, browse, and download this data). Please see Requester Pays Buckets in the Amazon S3 Guide for more information on this service. Your use of Amazon S3 is subject to Amazon's Terms of Use. The accessibility of SDMS data from Amazon S3 is provided "as is" without warranty of any kind, expressed or implied, including, but not limited to, the implied warranties of merchantability and fitness for a particular use. Please do not contact SDMS for assistance with Amazon services, you can post a GitHub issue and the authors will try and assist.

The name of the S3 bucket is s3://sdms-turbulence-degraded-characters/

Tools

We do not track development of tools interacting with Amazon S3, nor endorse any particular tool. However, in development of this facility we found the Python package s3cmd to be useful on Mac OS X and Linux. For Windows the AWS team provided the following suggestions s3browser.com and cross ftp.

s3cmd Example

The examples shown below were developed and tested with Linux and Mac OS X using the Python distribution of Enthought Canopy. First you must establish a AWS account and download your AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Next use the following commands to export your key id and secret access key to the shell environment replacing the values of AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with the values from your newly established AWS account:

export AWS_ACCESS_KEY_ID="JSHSHEUESKSK65"
export AWS_SECRET_ACCESS_KEY="kskjskjsAEERERERlslkhdjhhrhkjdASKJSKJS666789"

Install s3cmd with pip:

pip install s3cmd

To list the directory of the sdms-turbulence-degraded-characters bucket use the following command:

s3cmd --requester-pays ls s3://sdms-turbulence-degraded-characters/

Please note all the turbulence degraded characters data products are in a single directory.

To determine the amount of disk space in the sdms-turbulence-degraded-characters bucket use the command:

s3cmd --requester-pays --recursive --human du s3://sdms-turbulence-degraded-characters/

To retrieve a file:

s3cmd --requester-pays get s3://sdms-turbulence-degraded-characters/1_labeled.hdf5

Optimizing Data Download Costs

The Turbulence Degraded Characters data set is large at approximately 1.5 GB, to download the entire data set it would cost $0.14. This assumes AWS download fees are $0.09 a gigabyte. Most researchers will not require the entire Turbulence Degraded Characters data set, carefully consider the data that you require to meet your research needs and download costs. We have divided the dataset into small hdf5 files to make the download easier and to make it possible for a researcher to select the data they need.

Citation

When reporting results that use the TDC dataset, please cite:

LeMaster, Daniel A., Steven Leung, and Olga L. Mendoza-Schrock. "Joint object classification and turbulence strength estimation using convolutional neural networks." Applied Optics 60.25 (2021): G40-G48. https://www.osapublishing.org/ao/abstract.cfm?uri=ao-60-25-G40

References

[1] Lachinova, Svetlana L., et al. "Anisoplanatic imaging through atmospheric turbulence: Brightness function approach." Atmospheric Optics: Models, Measurements, and Target-in-the-Loop Propagation. Vol. 6708. International Society for Optics and Photonics, 2007.

Copyright information

This page and data set is in the public domain under 17 U.S.C. 105.