Skip to content

DICOM files deletion

Jump to bottom
Laurent Paul Vallet edited this page Feb 3, 2025 · 5 revisions

Shanoir ↔ PACS mapping

The pacs uses a specific vocabulary that is slightly different from what we use in shanoir:

Shanoir PACS
Study
Examination Study
Dataset Acquisition Serie
Dataset
Image Instance

Whenever a user makes a new import (MR) in shanoir, its data is sent to the PACS. For instance, let's say we have this subject, with this examination:

image

This is how it is translated in the pacs:

image

Rejecting data from the pacs

The deletion process in the pacs has two steps: rejecting and deleting. Picture yourself placing a file in the garbage bin and then emptying that bin. The idea is to call the right request so that we do not have to use the GUI.

Let's see what is available. From the PACS GUI, we can click the three dots in the list and click on the bin to open this menu:

image

As you can see, there are some criterias:

Criteria Action Comment
Rejection Reason Add a reason to reject We mostly use the "quality" reason. Makes a http request ending with '/reject/113001%5EDCM'
Queue Rejection Not sure Adds a http header 'queue=true' (e.g. '/reject/113001^DCM?queue=true')
Batch ID Not sure Adds a http header 'batchID=<batch_id>' (e.g. '/reject/113039^DCM?batchID=123'
Schedule at Sets up a date and time to actually delete the data Adds another header 'scheduledTime=20250203111320' (e.g. '/reject/113001^DCM?queue=true&scheduledTime=20250203111320', for 3rd of february 2025 at 11:13:20)

Once the data is rejected, a new line appears, set with "KO":

image

The logs also show this:

Start rejection of 5 instances of Study[UID=1.4.9.12.34.1.8527.1266790136069950507038323559602984218587], Series[UID=1.4.9.12.34.1.8527.3272003098677166550374252903871818734736], SOPInstance[UID=null].
[email protected]>AS_RECEIVED: RejectedInstance[pk=224, time=Mon Feb 03 10:34:24 UTC 2025, code=(113001, DCM, "Rejected for Quality Reasons")] of Instance[uid=1.4.9.12.34.1.8527.5257943823140364201348210943224850280963, class=1.2.840.10008.5.1.4.1.1.4] of Series[uid=1.4.9.12.34.1.8527.3272003098677166550374252903871818734736] of Study[uid=1.4.9.12.34.1.8527.1266790136069950507038323559602984218587]
[email protected]>AS_RECEIVED: RejectedInstance[pk=225, time=Mon Feb 03 10:34:24 UTC 2025, code=(113001, DCM, "Rejected for Quality Reasons")] of Instance[uid=1.4.9.12.34.1.8527.1114658240494812426734634941823312097264, class=1.2.840.10008.5.1.4.1.1.4] of Series[uid=1.4.9.12.34.1.8527.3272003098677166550374252903871818734736] of Study[uid=1.4.9.12.34.1.8527.1266790136069950507038323559602984218587]
[email protected]>AS_RECEIVED: RejectedInstance[pk=226, time=Mon Feb 03 10:34:24 UTC 2025, code=(113001, DCM, "Rejected for Quality Reasons")] of Instance[uid=1.4.9.12.34.1.8527.3012038432320163851269006250943890246274, class=1.2.840.10008.5.1.4.1.1.4] of Series[uid=1.4.9.12.34.1.8527.3272003098677166550374252903871818734736] of Study[uid=1.4.9.12.34.1.8527.1266790136069950507038323559602984218587]
[email protected]>AS_RECEIVED: RejectedInstance[pk=227, time=Mon Feb 03 10:34:24 UTC 2025, code=(113001, DCM, "Rejected for Quality Reasons")] of Instance[uid=1.4.9.12.34.1.8527.1028865906262213795093139363487086263220, class=1.2.840.10008.5.1.4.1.1.4] of Series[uid=1.4.9.12.34.1.8527.3272003098677166550374252903871818734736] of Study[uid=1.4.9.12.34.1.8527.1266790136069950507038323559602984218587]
[email protected]>AS_RECEIVED: RejectedInstance[pk=228, time=Mon Feb 03 10:34:24 UTC 2025, code=(113001, DCM, "Rejected for Quality Reasons")] of Instance[uid=1.4.9.12.34.1.8527.3188951169585107521652091663471498001831, class=1.2.840.10008.5.1.4.1.1.4] of Series[uid=1.4.9.12.34.1.8527.3272003098677166550374252903871818734736] of Study[uid=1.4.9.12.34.1.8527.1266790136069950507038323559602984218587]
[email protected]>AS_RECEIVED: Create Series[pk=84, uid=2.25.187738652971431839171939916865527743950, no=0, mod=KO]
[email protected]>AS_RECEIVED: Create Instance[pk=6600, uid=2.25.155711986199924700025999216608009681302, class=1.2.840.10008.5.1.4.1.1.88.59, no=0]
[email protected]>AS_RECEIVED: Create Location[pk=6600, systemID=fs1, path=2025/02/03/4BC811C1/F5FA84A4/94D9E1C7, tsuid=1.2.840.10008.1.2.1, size=2758, status=OK, objectType=DICOM_FILE]
[email protected]>AS_RECEIVED: Updated DB in 42 ms
Rejection of 5 instances of Study[UID=1.4.9.12.34.1.8527.1266790136069950507038323559602984218587], Series[UID=1.4.9.12.34.1.8527.3272003098677166550374252903871818734736], SOPInstance[UID=null] completed.

When we will call the deletion of rejected data, this new line 'KO' will also be removed.

Here, I added the 'scheduled at' criteria to remove the data at 10:53:00, I also checked the 'queue rejection' box. Here is what the logs say:

2025-02-03 10:52:20,038 INFO  [org.dcm4chee.arc.qmgt.impl.TaskManagerEJB] (default task-4) Create Task{taskID=11, deviceName='dcm4chee-arc', queueName='Rejection', type=REJECT, status=SCHEDULED, scheduledTime=Mon Feb 03 10:53:00 UTC 2025, numberOfFailures=0, studyInstanceUID=1.4.9.12.34.1.8527.9135575358331081596117405113227004182195, seriesInstanceUID=1.4.9.12.34.1.8527.4139145426170156646630798391731855617414, localAET=AS_RECEIVED}

And then, at 10:53:00:

2025-02-03 10:53:13,486 INFO  [org.dcm4chee.arc.ups.process.impl.UPSProcessingScheduler] (EE-ManagedScheduledExecutorService-default-Thread-4) start UPSProcessingScheduler.execute()
2025-02-03 10:53:13,487 INFO  [org.dcm4chee.arc.ups.process.impl.UPSProcessingScheduler] (EE-ManagedScheduledExecutorService-default-Thread-4) finished UPSProcessingScheduler.execute()
2025-02-03 10:53:20,399 INFO  [org.dcm4chee.arc.qmgt.impl.TaskScheduler] (EE-ManagedScheduledExecutorService-default-Thread-5) start TaskScheduler.execute()
2025-02-03 10:53:20,404 INFO  [org.dcm4chee.arc.qmgt.impl.TaskScheduler] (EE-ManagedExecutorService-default-Thread-322) Start processing Task{taskID=11, deviceName='dcm4chee-arc', queueName='Rejection', type=REJECT, status=IN PROCESS, scheduledTime=2025-02-03 10:53:00.0, numberOfFailures=0, studyInstanceUID=1.4.9.12.34.1.8527.9135575358331081596117405113227004182195, seriesInstanceUID=1.4.9.12.34.1.8527.4139145426170156646630798391731855617414, localAET=AS_RECEIVED}
2025-02-03 10:53:20,406 INFO  [org.dcm4chee.arc.qmgt.impl.TaskScheduler] (EE-ManagedScheduledExecutorService-default-Thread-5) finished TaskScheduler.execute()
2025-02-03 10:53:20,408 INFO  [org.dcm4chee.arc.delete.impl.RejectionServiceImpl] (EE-ManagedExecutorService-default-Thread-322) Start rejection of 5 instances of Study[UID=1.4.9.12.34.1.8527.9135575358331081596117405113227004182195], Series[UID=1.4.9.12.34.1.8527.4139145426170156646630798391731855617414], SOPInstance[UID=null].
2025-02-03 10:53:20,416 INFO  [org.dcm4chee.arc.store.impl.StoreServiceEJB] (EE-ManagedExecutorService-default-Thread-322) [email protected]>AS_RECEIVED: Detect previous RejectedInstance[pk=234, time=2025-02-03 10:48:20.449, code=(113001, DCM, "Rejected for Quality Reasons")] of Instance[uid=1.4.9.12.34.1.8527.7005363033571580342006145892386975487766, class=1.2.840.10008.5.1.4.1.1.4] of Series[uid=1.4.9.12.34.1.8527.4139145426170156646630798391731855617414] of Study[uid=1.4.9.12.34.1.8527.9135575358331081596117405113227004182195]
2025-02-03 10:53:20,417 INFO  [org.dcm4chee.arc.store.impl.StoreServiceEJB] (EE-ManagedExecutorService-default-Thread-322) [email protected]>AS_RECEIVED: Detect previous RejectedInstance[pk=235, time=2025-02-03 10:48:20.45, code=(113001, DCM, "Rejected for Quality Reasons")] of Instance[uid=1.4.9.12.34.1.8527.8006121438127823269603834564441015657054, class=1.2.840.10008.5.1.4.1.1.4] of Series[uid=1.4.9.12.34.1.8527.4139145426170156646630798391731855617414] of Study[uid=1.4.9.12.34.1.8527.9135575358331081596117405113227004182195]
2025-02-03 10:53:20,417 INFO  [org.dcm4chee.arc.store.impl.StoreServiceEJB] (EE-ManagedExecutorService-default-Thread-322) [email protected]>AS_RECEIVED: Detect previous RejectedInstance[pk=236, time=2025-02-03 10:48:20.451, code=(113001, DCM, "Rejected for Quality Reasons")] of Instance[uid=1.4.9.12.34.1.8527.7052030014060385472999376969863203672805, class=1.2.840.10008.5.1.4.1.1.4] of Series[uid=1.4.9.12.34.1.8527.4139145426170156646630798391731855617414] of Study[uid=1.4.9.12.34.1.8527.9135575358331081596117405113227004182195]
2025-02-03 10:53:20,417 INFO  [org.dcm4chee.arc.store.impl.StoreServiceEJB] (EE-ManagedExecutorService-default-Thread-322) [email protected]>AS_RECEIVED: Detect previous RejectedInstance[pk=237, time=2025-02-03 10:48:20.452, code=(113001, DCM, "Rejected for Quality Reasons")] of Instance[uid=1.4.9.12.34.1.8527.8122843385874855924031026455562987520827, class=1.2.840.10008.5.1.4.1.1.4] of Series[uid=1.4.9.12.34.1.8527.4139145426170156646630798391731855617414] of Study[uid=1.4.9.12.34.1.8527.9135575358331081596117405113227004182195]
2025-02-03 10:53:20,418 INFO  [org.dcm4chee.arc.store.impl.StoreServiceEJB] (EE-ManagedExecutorService-default-Thread-322) [email protected]>AS_RECEIVED: Detect previous RejectedInstance[pk=238, time=2025-02-03 10:48:20.452, code=(113001, DCM, "Rejected for Quality Reasons")] of Instance[uid=1.4.9.12.34.1.8527.1275666388075233490855484129139096448103, class=1.2.840.10008.5.1.4.1.1.4] of Series[uid=1.4.9.12.34.1.8527.4139145426170156646630798391731855617414] of Study[uid=1.4.9.12.34.1.8527.9135575358331081596117405113227004182195]
2025-02-03 10:53:20,421 INFO  [org.dcm4chee.arc.store.impl.StoreServiceEJB] (EE-ManagedExecutorService-default-Thread-322) [email protected]>AS_RECEIVED: Create Series[pk=100, uid=2.25.119812991699104776142358714907340332190, no=0, mod=KO]
2025-02-03 10:53:20,422 INFO  [org.dcm4chee.arc.store.impl.StoreServiceEJB] (EE-ManagedExecutorService-default-Thread-322) [email protected]>AS_RECEIVED: Create Instance[pk=7702, uid=2.25.318531451880502462483675854042529743287, class=1.2.840.10008.5.1.4.1.1.88.59, no=0]
2025-02-03 10:53:20,422 INFO  [org.dcm4chee.arc.store.impl.StoreServiceEJB] (EE-ManagedExecutorService-default-Thread-322) [email protected]>AS_RECEIVED: Create Location[pk=7702, systemID=fs1, path=2025/02/03/79D26416/C7F08823/0BB5F8DA, tsuid=1.2.840.10008.1.2.1, size=2758, status=OK, objectType=DICOM_FILE]
2025-02-03 10:53:20,426 INFO  [org.dcm4chee.arc.store.impl.StoreServiceImpl] (EE-ManagedExecutorService-default-Thread-322) [email protected]>AS_RECEIVED: Updated DB in 18 ms
2025-02-03 10:53:20,427 INFO  [org.dcm4chee.arc.delete.impl.RejectionServiceImpl] (EE-ManagedExecutorService-default-Thread-322) Rejection of 5 instances of Study[UID=1.4.9.12.34.1.8527.9135575358331081596117405113227004182195], Series[UID=1.4.9.12.34.1.8527.4139145426170156646630798391731855617414], SOPInstance[UID=null] completed.
2025-02-03 10:53:20,428 INFO  [org.dcm4chee.arc.qmgt.impl.TaskManagerEJB] (EE-ManagedExecutorService-default-Thread-322) Finished processing of Task{taskID=11, deviceName='dcm4chee-arc', queueName='Rejection', type=REJECT, status=COMPLETED, scheduledTime=2025-02-03 10:53:00.0, numberOfFailures=0, studyInstanceUID=1.4.9.12.34.1.8527.9135575358331081596117405113227004182195, seriesInstanceUID=1.4.9.12.34.1.8527.4139145426170156646630798391731855617414, localAET=AS_RECEIVED}

How to delete data:

To delete an examination, in shanoir you can select the examination from the tree and click the "Delete" button in the examination's details:

image

Every hour, the shanoir code will make a http request to remove all the data that is set to "rejected". From the logs:

2025-02-03 10:44:59,973 INFO  [org.dcm4chee.arc.iocm.rs.DeleteRejected] (default task-4) Process DELETE /dcm4chee-arc/reject/113001%5EDCM from [email protected]
2025-02-03 10:45:00,627 INFO  [org.dcm4chee.arc.iocm.rs.DeleteRejected] (default task-4) Deleted 112 instances permanently
2025-02-03 10:45:07,125 INFO  [org.dcm4chee.arc.study.size.StudySizeScheduler] (EE-ManagedScheduledExecutorService-default-Thread-7) start StudySizeScheduler.execute()
2025-02-03 10:45:07,127 INFO  [org.dcm4chee.arc.study.size.StudySizeScheduler] (EE-ManagedScheduledExecutorService-default-Thread-7) finished StudySizeScheduler.execute()
2025-02-03 10:45:07,290 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-6) start PurgeStorageScheduler.execute()
2025-02-03 10:45:07,291 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-6) finished PurgeStorageScheduler.execute()
2025-02-03 10:45:07,291 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedExecutorService-default-Thread-283) Start deletion on Storage[id=fs1, uri=file:///storage/fs1/]
2025-02-03 10:45:07,295 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedExecutorService-default-Thread-283) Start deleting 100 objects from Storage[id=fs1, uri=file:///storage/fs1/]
2025-02-03 10:45:07,591 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedExecutorService-default-Thread-283) Finished deleting 100 (skipped=0, failed=0) objects from Storage[id=fs1, uri=file:///storage/fs1/]
2025-02-03 10:45:07,592 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedExecutorService-default-Thread-283) Start deleting 10 objects from Storage[id=fs1, uri=file:///storage/fs1/]
2025-02-03 10:45:07,643 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedExecutorService-default-Thread-283) Finished deleting 10 (skipped=0, failed=0) objects from Storage[id=fs1, uri=file:///storage/fs1/]
2025-02-03 10:45:07,643 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedExecutorService-default-Thread-283) Finished deletion on Storage[id=fs1, uri=file:///storage/fs1/]

Future improvment

Currently, we make one call per entity. With big examination, this could do lots of calls and maybe overload the logs + server disk space.

To fix this, we could change the deletion that way:

  • delete exam: reject study, call /studies/xyz/reject
  • delete acquisition: reject serie, call /studies/xyz/series/abc/reject
  • delete dataset: reject instances, call /studies/xyz/series/abc/instances/qwer/reject on all dataset_file path
Clone this wiki locally