Add teardown task

[pankajastro]

This PR introduces a teardown node in the Cosmos DAG that deletes SQL files from the remote location that were uploaded by the setup task. This ensures proper cleanup of resources after the task execution.

Key Changes:

Teardown Node Addition:

A new teardown task is added to the Cosmos DAG that deletes the SQL files from the remote location. The files deleted are the ones uploaded by the setup task earlier in the DAG.
New Config: enable_teardown_async_task:

A configuration option enable_teardown_async_task, is introduced to enable or disable this feature. This config is enabled by default.
Setting this config to False will skip the teardown step, while setting it to True will ensure the deletion task runs after the setup.

Applicable Only for AIRFLOW_ASYNC Execution Mode:

This feature is only triggered when the DAG is running in AIRFLOW_ASYNC execution mode

closes: #1232

[netlify]

✅ Deploy Preview for sunny-pastelito-5ecb04 canceled.

Name	Link
🔨 Latest commit	81ae672
🔍 Latest deploy log	https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/67adfb1a67e46f0008c2cf86

[cloudflare-workers-and-pages]

Deploying astronomer-cosmos with    Cloudflare Pages

Latest commit:	81ae672
Status:	✅  Deploy successful!
Preview URL:	https://87afda90.astronomer-cosmos.pages.dev
Branch Preview URL:	https://teardown-task.astronomer-cosmos.pages.dev

View logs

[tatiana]

Hi @pankajastro some early feedback. Please, could you confirm that the teardown task is not emitting any datasets (outlets)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.32%. Comparing base (a43a991) to head (81ae672).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1529      +/-   ##
==========================================
+ Coverage   97.30%   97.32%   +0.02%     
==========================================
  Files          80       80              
  Lines        4784     4821      +37     
==========================================
+ Hits         4655     4692      +37     
  Misses        129      129              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pankajastro]

Hi @pankajastro some early feedback. Please, could you confirm that the teardown task is not emitting any datasets (outlets)

Yes, teardown task do not emit any datasets

[pankajkoti]

LGTM. Looks like we have a conflict with main that needs to be resolved.