How to remove specific coastlines in the output.pkl file?

[Mascagni-ML]

Is it possible to remove specific coastlines by date in the output.pkl file?

By analyzing the selected coastlines in the GIS software with the GeoJson file generated from the output.pkl file, I identified 3 coastlines out of a set of 807 that I wanted to remove from the previous selection.

In the GIS it was easy to identify the date of the coastlines to be removed, delete them and save the new GeoJson without the unwanted coastlines.

However, to continue the analysis in the CoastSat notebook with the .pkl file, I was unable to remove the unwanted coastlines that I removed in the GeoJson.

In fact, I even generated a script using the pickle library, which supposedly removed the unwanted coastlines from the output.pkl file, but I was unable to continue with the analyses on the CoastSat notebook because the metadata stored in the output variable was probably different from the modified .pkl file, resulting in several errors in the code.

Thus, I really would like a more practical solution to continue the analyses without having to manually select over 800 coastlines again, adjusting thresholds, etc, to build my database, taking care not to accidentally select these 3 unwanted coastlines.

[Mascagni-ML]

I think I got it here.
The problem was with how I was calling the adjusted .pkl file in the coastsat notebook, after running the routine I made to filter the coastlines.

So, here is the code to filter coastlines from the .pkl file, selected by mistake in the previous step, from specific dates and satellite missions.

##################################################################

Step 01 - Load Python Libraries

import sys
sys.path.append("C:/your/path/to/coastsat")
import pickle
from datetime import datetime

Step 02 - Load the coastsat .pkl original file

file_path = "C:/your/path/to/coastsat/case/filename_output.pkl"
with open(file_path, 'rb') as file:
coastsat_data = pickle.load(file)

Step 03 (optional) - Check available dictionary keys in coastsat original .pkl

print(f"Keys in coastsat_data: {coastsat_data.keys()}")

Step 04 - Accessing the required fields from the pkl dictionary

dates = coastsat_data.get('dates', [])
shorelines_list = coastsat_data.get('shorelines', [])
satnames = coastsat_data.get('satname', [])

Step 05 (optional) - Verify the structure

print(f"First 10 dates: {dates[:10]}")
print(f"First 10 satnames: {satnames[:10]}")

Step 06 - List of unwanted dates and corresponding satellites

unwanted_entries = [
('2007-03-26', 'L5'),
('2012-05-18', 'L7'),
('2013-07-16', 'L8'),
('2017-05-15', 'S2'),
('2023-10-30', 'S2')
]

Step 07 - Initiate the filtered lists

filtered_dates = []
filtered_shorelines = []
filtered_satnames = []

Step 08 - Filter original .pkl data to skip unwanted entries

for date, shoreline, satname in zip(dates, shorelines_list, satnames):
# Convert datetime to string for comparison
date_str = date.strftime('%Y-%m-%d')
if (date_str, satname) not in unwanted_entries:
filtered_dates.append(date)
filtered_shorelines.append(shoreline)
filtered_satnames.append(satname)

Step 09 - Update the dictionary with the filtered data

coastsat_data['dates'] = filtered_dates
coastsat_data['shorelines'] = filtered_shorelines
coastsat_data['satname'] = filtered_satnames

Step 10 - Check by printing the summaries before and after filtering

print(f"Original dates count: {len(dates)}")
print(f"Filtered dates count: {len(filtered_dates)}")
print(f"Original shoreline count: {len(shorelines_list)}")
print(f"Filtered shoreline count: {len(filtered_shorelines)}")

Step 11 - Save the corrected .pkl file

output_path = "C:/your/path/to/coastsat/case/filename_output_adj.pkl"
with open(output_path, 'wb') as file:
pickle.dump(coastsat_data, file)

print(f"Adjusted {output_path} file saved successfully!")
####################################################################

And the correct way to call the new adjusted .pkl file in the coastsat notebook:

####################################################################
%matplotlib qt
#output = SDS_shoreline.extract_shorelines(metadata, settings) # original way to call the shoreline data from coastsat pkl file
pkl_file = "C:/your/path/to/coastsat/case/filename_output_adj.pkl"
with open(pkl_file, 'rb') as file:
output = pickle.load(file)
####################################################################

Then remove duplicates and images with inaccurate georeferencing (threhsold at 10m)

####################################################################

Same code as the original, nothing changed here.

output = SDS_tools.remove_duplicates(output) # removes duplicates (images taken on the same date by the same satellite)
output = SDS_tools.remove_inaccurate_georef(output, 10) # remove inaccurate georeferencing (set threshold to 10 m)
####################################################################

Apparently it is working in the following steps of the coastsat notebook. The image of the filtered Lcs is without unwanted noise.