Splitting part of source_detection.ipynb off into separate imaging.ip…

…ynb example notebook (#610)

* Split part of source_detection.ipynb off into imaging.ipynb examples. Update docs accordingly 🈴

* Fix line endings to unix format. 😆

* Add test for new imaging notebook 🎎

* Add function to simulate visibilities, reducing duplication ☎️

* Add Type Annotations to new function ✋

* Fix function input types ☎️

* Add failsave in case list of vis is generated 📗

* Fix return variable type 🚐

* Update run_simulation to accept path in addition to str, fix type hints 👇

* Undo run_simulation to accept paths, cast to str instead 🚚

* Make verbose parameter kw-only and remove mutable default ↕️

* Add deduplication to source detection notebook, fix todo text 🐅

* Add deduplication to source detection notebook, fix todo text 🌾

doc/src/examples/example_structure.md

@@ -8,6 +8,14 @@ The following example showcases a simple telescope simulation using the main pip
 <example_interfe_simu.py>
 ```
 
+## Imaging
+
+The notebook [imaging.ipynb](https://github.com/i4Ds/Karabo-Pipeline/blob/main/karabo/examples/imaging.ipynb), shows how to use different dirty imaging and image cleaning algorithms.
+
+## Source detection
+
+In the example notebook [source_detection.ipynb](https://github.com/i4Ds/Karabo-Pipeline/blob/main/karabo/examples/source_detection.ipynb), we simulate data, perform dirty imaging, clean the image, and then run and evaluate a source detection algorithm.
+
 ## SRCNet
 
 Karabo is used in the SRCNet to generate simulated test data resembling SKAO data.
@@ -29,10 +37,6 @@ See the script [line_emission.py](https://github.com/i4Ds/Karabo-Pipeline/blob/m
 
 This simulation begins with a `SkyModel` instance, and with the definition of the desired `Observation` and `Telescope` details. Then, the `InterferometerSimulation` instance uses the requested backend (OSKAR and RASCIL are currently supported) to compute the corresponding visibilities, and the desired `DirtyImager` instance is used to convert the visibilities into dirty images. Optionally, we can include primary beam effects and correct for such effects in the final dirty images. Finally, we can mosaic different dirty images into one larger image using the `ImageMosaicker` class.
 
-## Source detection
-
-In the example notebook [source_detection.ipynb](https://github.com/i4Ds/Karabo-Pipeline/blob/main/karabo/examples/source_detection.ipynb), we simulate data, perform dirty imaging, clean the image, and then run and evaluate a source detection algorithm.
-
 ## Show telescope config
 
 ```python

doc/src/examples/examples.md

@@ -44,6 +44,14 @@ observation = Observation(
 simulation.run_simulation(telescope, sky, observation)
 ```
 
+## Imaging
+
+The notebook [imaging.ipynb](https://github.com/i4Ds/Karabo-Pipeline/blob/main/karabo/examples/imaging.ipynb), shows how to use different dirty imaging and image cleaning algorithms.
+
+## Source detection
+
+In the example notebook [source_detection.ipynb](https://github.com/i4Ds/Karabo-Pipeline/blob/main/karabo/examples/source_detection.ipynb), we simulate data, perform dirty imaging, clean the image, and then run and evaluate a source detection algorithm.
+
 ## SRCNet
 
 Karabo is used in the SRCNet to generate simulated test data resembling SKAO data.
@@ -65,10 +73,6 @@ See the script [line_emission.py](https://github.com/i4Ds/Karabo-Pipeline/blob/m
 
 This simulation begins with a `SkyModel` instance, and with the definition of the desired `Observation` and `Telescope` details. Then, the `InterferometerSimulation` instance uses the requested backend (OSKAR and RASCIL are currently supported) to compute the corresponding visibilities, and the desired `DirtyImager` instance is used to convert the visibilities into dirty images. Optionally, we can include primary beam effects and correct for such effects in the final dirty images. Finally, we can mosaic different dirty images into one larger image using the `ImageMosaicker` class.
 
-## Source detection
-
-In the example notebook [source_detection.ipynb](https://github.com/i4Ds/Karabo-Pipeline/blob/main/karabo/examples/source_detection.ipynb), we simulate data, perform dirty imaging, clean the image, and then run and evaluate a source detection algorithm.
-
 ## Show telescope config
 
 ```python

karabo/simulation/sample_simulation.py

@@ -0,0 +1,76 @@
+from datetime import datetime
+from typing import Union
+
+from karabo.simulation.interferometer import InterferometerSimulation
+from karabo.simulation.observation import Observation
+from karabo.simulation.sky_model import SkyModel
+from karabo.simulation.telescope import Telescope
+from karabo.simulation.visibility import Visibility
+from karabo.simulator_backend import SimulatorBackend
+
+
+def run_sample_simulation(
+    phase_center: Union[list[float], None] = None, *, verbose: bool = False
+) -> tuple[Visibility, SkyModel]:
+    """
+    Creates example visibilities for use in tests, experiments and examples.
+
+    Args:
+        phase_center: ra and dec of the sky. Defaults to [250, -80] if not provided.
+        verbose: Boolean to decide if console outputs are made during simulation
+        (e.g. for use in ipynb)
+
+    Returns:
+        Visibility: visibilities created by the simulation
+        SkyModel: Sky model used for the simulation
+    """
+
+    if phase_center is None:
+        phase_center = [250, -80]
+
+    if verbose:
+        print("Getting Sky Survey")
+    # Get GLEAM Survey Sky
+    gleam_sky = SkyModel.get_GLEAM_Sky(min_freq=72e6, max_freq=80e6)
+
+    if verbose:
+        print("Filtering Sky Model")
+    sky = gleam_sky.filter_by_radius(0, 0.55, phase_center[0], phase_center[1])
+    sky.setup_default_wcs(phase_center=phase_center)
+
+    if verbose:
+        print("Setting Up Telescope")
+    askap_tel = Telescope.constructor(
+        "ASKAP", version=None, backend=SimulatorBackend.OSKAR
+    )
+
+    if verbose:
+        print("Setting Up Observation")
+    observation_settings = Observation(
+        start_frequency_hz=100e6,
+        start_date_and_time=datetime(2024, 3, 15, 10, 46, 0),
+        phase_centre_ra_deg=phase_center[0],
+        phase_centre_dec_deg=phase_center[1],
+        number_of_channels=16,
+        number_of_time_steps=24,
+    )
+
+    if verbose:
+        print("Generating Visibilities")
+
+    vis = Visibility()
+    vis_path = str(vis.vis_path)
+
+    print(f"Saving file to {vis_path}")
+    interferometer_sim = InterferometerSimulation(
+        vis_path=vis_path, channel_bandwidth_hz=1e6
+    )
+    visibility_askap = interferometer_sim.run_simulation(
+        askap_tel, sky, observation_settings, backend=SimulatorBackend.OSKAR
+    )
+
+    # In case run_simulation returns a list of vis (allowed by type hint)
+    if isinstance(visibility_askap, list):
+        visibility_askap = visibility_askap[0]
+
+    return visibility_askap, sky

karabo/test/test_notebooks.py

@@ -73,3 +73,11 @@ def test_LineEmission_notebook() -> None:
 )
 def test_ImageMosaicker_notebook() -> None:
     _run_notebook(notebook="ImageMosaicker.ipynb")
+
+
+@pytest.mark.skipif(
+    not RUN_NOTEBOOK_TESTS,
+    reason="'Error: The operation was canceled' when running this test on the package",
+)
+def test_imaging_notebook() -> None:
+    _run_notebook(notebook="imaging.ipynb")

karabo/test/test_simulation.py

@@ -1,6 +1,8 @@
 import os
+import sys
 import tempfile
 from datetime import datetime, timedelta
+from io import StringIO
 from pathlib import Path
 
 import numpy as np
@@ -18,8 +20,10 @@
 from karabo.imaging.util import auto_choose_dirty_imager_from_vis
 from karabo.simulation.interferometer import InterferometerSimulation
 from karabo.simulation.observation import Observation, ObservationParallelized
+from karabo.simulation.sample_simulation import run_sample_simulation
 from karabo.simulation.sky_model import SkyModel
 from karabo.simulation.telescope import Telescope
+from karabo.simulation.visibility import Visibility
 from karabo.simulator_backend import SimulatorBackend
 
 
@@ -314,3 +318,50 @@ def test_parallelization_by_observation() -> None:
         assert dirty.header["CRVAL4"] == CENTER_FREQUENCIES_HZ[i]
         assert dirty.header["NAXIS4"] == N_CHANNELS[i]
         assert dirty.header["CDELT4"] == CHANNEL_BANDWIDTHS_HZ[i]
+
+
+def test_run_sample_simulation(
+    verbose: bool = True, phase_center: list = [250, -80]
+) -> None:
+    """
+    Executes the ASKAP sample simulation, captures verbose output,
+    validates the output files, and checks the sky model filtering.
+
+    Args:
+        verbose: Boolean flag to capture verbose output from the simulation.
+        phase_center: List containing the RA and DEC of the phase center
+        for the simulation.
+    """
+
+    # run simulation and capture output
+    old_stdout = sys.stdout
+    sys.stdout = StringIO()
+
+    visibilities, sky = run_sample_simulation(
+        verbose=verbose, phase_center=phase_center
+    )
+
+    output = sys.stdout.getvalue()
+    sys.stdout = old_stdout
+
+    # returns non-None values of the correct type
+    assert visibilities is not None
+    assert sky is not None
+    assert isinstance(visibilities, Visibility)
+    assert isinstance(sky, SkyModel)
+
+    # verbose output content
+    if verbose:
+        expected_messages = [
+            "Getting Sky Survey",
+            "Filtering Sky Model",
+            "Setting Up Telescope",
+            "Setting Up Observation",
+            "Generating Visibilities",
+        ]
+        for message in expected_messages:
+            assert message in output
+
+    # Ensure the visibilities file path is valid
+    vis_path = visibilities.vis_path
+    assert os.path.exists(vis_path)