Merge branch 'tickets/DM-46919'

.github/workflows/ci.yml

@@ -356,6 +356,10 @@ jobs:
         if: always()
         run: docker logs ${USER}_czar_proxy_1
 
+      - name: Czar HTTP Frontend Log
+        if: always()
+        run: docker logs ${USER}_czar_http_1
+
       - name: Czar CMSD Log
         if: always()
         run: docker logs ${USER}_czar_cmsd_1

admin/local/docker/compose/docker-compose.yml

@@ -397,10 +397,14 @@ services:
           entrypoint --log-level DEBUG czar-http
           --db-uri mysql://qsmaster@czar_mariadb:3306/
           --xrootd-manager czar_xrootd
-          --http-frontend-port 4048
-          --http-frontend-threads 4
+          --czar-name http
+          --http-port 4048
+          --http-threads 4
+          --http-worker-ingest-threads 2
           --http-ssl-cert-file /config-etc/ssl/czar-cert.pem
           --http-ssl-private-key-file /config-etc/ssl/czar-key.pem
+          --http-tmp-dir /tmp
+          --http-conn-pool-size 2
           --log-cfg-file=/config-etc/log/log-czar-proxy.cnf
           --repl-instance-id qserv_proj
           --repl-auth-key replauthkey

doc/admin/data-table-indexes.rst

@@ -247,7 +247,7 @@ Where:
     The required *base* name of the table where the index will be created.
 
 ``overlap`` : *number* := ``0``
-    The optional *overlap* flagg indicating a sub-type of the *chunk* table. The value should be one of the following:
+    The optional *overlap* flag indicating a sub-type of the *chunk* table. The value should be one of the following:
 
     - ``1`` : *full overlap*
     - ``0`` : *chunk*
@@ -301,9 +301,9 @@ Where:
         types (numeric, etc.). Otherwise, an index creation request will fail. 
 
     ``ascending`` : *number*
-        The required sorting order of the column in the index. It translates into ``ASC`` or ``DESC``  optiona
-        in the key definition in ``key_part``. A value of ``0`` will be interpreted as ``DESC``. Any other positive number
-        will be imterpreted as to ``ASC``.
+        The required sorting order of the column in the index. It translates into ``ASC`` or ``DESC`` options
+        in the key definition in ``key_part``. A value of ``0`` will be interpreted as ``DESC``.
+        Any other positive number will be imterpreted as to ``ASC``.
 
 ``auth_key`` : *string*
     The required zauthorization key.
@@ -388,7 +388,7 @@ Where:
     The required *base* name of the table where the index will be created.
 
 ``overlap`` : *number* := ``0``
-    The optional *overlap* flagg indicating a sub-type of the *chunk* table. The value should be one of the following:
+    The optional *overlap* flag indicating a sub-type of the *chunk* table. The value should be one of the following:
 
     - ``1`` : *full overlap*
     - ``0`` : *chunk*
@@ -439,10 +439,10 @@ Where the service path has the following parameters:
 ``table`` : *string*
     The name of the table for which the indexes are required to be collected.
 
-The optional query parameyter is
+The optional query parameter is
 
 ``overlap`` : *number* := ``0``
-    The optional *overlap* flagg indicating a sub-type of the *chunk* table. The value should be one of the following:
+    The optional *overlap* flag indicating a sub-type of the *chunk* table. The value should be one of the following:
 
     - ``1`` : *full overlap*
     - ``0`` : *chunk*

doc/ingest/api/advanced/directors.rst

@@ -87,7 +87,7 @@ The following table illustrates how JSON configurations for all the above-mentio
   |     "longitude_key" : "dec"       |                                           |
   |   }                               | **Note**: Attributes ``latitude_key`` and |
   |                                   | ``longitude_key`` were not provided.      |
-  |                                   | is allowed for the dependent tables.      |
+  |                                   | This is allowed for the dependent tables. |
   |                                   |                                           |
   |                                   | .. code-block:: json                      |
   |                                   |                                           |

doc/ingest/api/advanced/transactions.rst

@@ -119,7 +119,6 @@ Things to consider:
   factor limiting the performance of the workflow.
 - Any failure to ingest a contribution will result in aborting the entire transaction. This can significantly
   impact the workflow's performance, especially if the amount of data to be ingested is large.
-  impact the workflow's performance, especially if the amount of data to be ingested is large.
 
 Best use:
 

doc/ingest/api/concepts/overview.rst

@@ -69,8 +69,8 @@ it should be registered using:
 
 - :ref:`ingest-db-table-management-register-db` (REST)
 
-The newely registered database will be always in the *unpublished* state. If the database already exists in
-the Replication/Ingest and it's in the *published* state it should be *unpublished* state using:
+The newly registered database will be always in the *unpublished* state. If the database already exists in
+the Replication/Ingest and it's in the *published* state it should be put into the *unpublished* state using:
 
 - :ref:`ingest-db-table-management-unpublish-db` (REST)
 
@@ -159,8 +159,8 @@ The REST API for initiating the contribuiton requests is covered in the followin
 Monitor the progress of the ingest activities
 ----------------------------------------------
 
-The workflow should always be avare about the progress of the ingest activities, and about the status of the
-contribution requests. This is need for (at least) three reasons:
+The workflow should always be aware about the progress of the ingest activities, and about the status of the
+contribution requests. This is needed for (at least) three reasons:
 
 #. To know when the ingest activities are finished
 #. To know when the ingest activities (and which requests) are failed

doc/ingest/api/concepts/transactions.rst

@@ -3,14 +3,13 @@
 Transactions
 ============
 
-The distributed transaction mechanism is one of the key technologies that was
-implemented in the Qserv Ingest system to allow for the incremental updates of the overall state of the data and metadata
-while ensuring the consistency of the ingested catalogs. Transactions also play an important role in allowing
-the high-performance ingest activities to be performed in a distributed environment. Transactions if used correct may
-significantly increase the level of parallelism of the ingest workflows. Transactions are not visible to end users.
+The distributed transaction mechanism is a crucial technology in the Qserv Ingest system. It enables incremental updates
+to the data and metadata while maintaining the consistency of ingested catalogs. Transactions also facilitate high-performance
+ingest activities in a distributed environment. When used correctly, transactions can significantly enhance the parallelism
+of ingest workflows. Note that transactions are not visible to end users.
 
 Transactions are open in a scope of a database. It's a responsibility of the workflows to manage transactions as needed
-for the ingest activities uisng the following REST services:
+for the ingest activities using the following REST services:
 
 - :ref:`ingest-trans-management` (REST)
 
@@ -68,10 +67,10 @@ to Qserv users and is queriable. Here is an illustration of a query and the corr
 Checkpointing
 -------------
 
-Transactions also provide the checkpointing mechanism that allows rolling back to a prior consistent state of the affected tables
-should any problem occur during the ingest activities. Transactions may spans across many workers and tables located
-at the workers. It's up to the workflow to decide what contrubutions to ingest and in what order to ingest those in
-a scope of each transaction.
+Transactions also provide a checkpointing mechanism that allows rolling back to a prior consistent state of the affected tables
+should any problem occur during the ingest activities. Transactions may span across many workers and tables located
+at the workers. It's up to the workflow to decide what contributions to ingest and in what order to ingest those in
+the scope of each transaction.
 
 The following diagram illustrates the concept of the transactions in Qserv. There are 3 transactions that are started and
 commited sequentially (in the real life scenarios the transactions can be and should be started and commited in parallel,

doc/ingest/api/index.rst

@@ -1,7 +1,7 @@
 
 .. note::
 
-   Information in this guide corresponds to the version **38** of the Qserv REST API. Keep in mind
+   Information in this guide corresponds to the version **39** of the Qserv REST API. Keep in mind
    that each implementation of the API has a specific version. The version number will change
    if any changes to the implementation or the API that might affect users will be made.
    The current document will be kept updated to reflect the latest version of the API.

doc/user/http-frontend-general.rst

@@ -124,7 +124,7 @@ Example:
         "name" :        "http",
         "id" :          8,
         "instance_id" : "qserv-prod",
-        "version" :     38,
+        "version" :     39,
         "success" :     1
     }
 
@@ -137,27 +137,27 @@ the following request for checking the status of an ongoing query might look lik
 
 .. code-block:: bash
 
-   curl -k 'https://localhost:4041/query-async/status/1234?version=38' -X GET
+   curl -k 'https://localhost:4041/query-async/status/1234?version=39' -X GET
 
 For other HTTP methods used by the API, the version number must be included in the body of the request as shown below:
 
 .. code-block:: bash
 
    curl -k 'https://localhost:4041/query-async' -X POST \
         -H 'Content-Type: application/json' \
-        -d'{"version":38,"query":"SELECT ..."}'
+        -d'{"version":39,"query":"SELECT ..."}'
 
 If the number does not match expectations, such a request will fail and the service will return the following
 response. Here is an example of what will happen if the wrong version number ``29`` is specified instead
-of ``38`` (as specified in the example above):
+of ``39`` (as specified in the example above):
 
 .. code-block:: json
 
     {   "success" : 0,
         "error" :   "The requested version 29 of the API is not in the range
                      supported by the service.",
         "error_ext" : {
-            "max_version" : 38,
+            "max_version" : 39,
             "min_version" : 30
         },
         "warning" : ""