%%capture --no-display --no-stderr cell_outputto suppress this output. Comment or delete this line in the cells below to restore full output.
![\"clara_pt_brain_mri_segmentation_workflow\"](\"https://developer.download.nvidia.com/assets/Clara/Images/clara_pt_brain_mri_segmentation_workflow.png\")
\\n\n",
- "\n",
- "- The ET is described by areas that show hyper intensity in T1c when compared to T1, but also when compared to \"healthy\" white matter in T1c. \n",
- "- The TC describes the bulk of the tumor, which is what is typically resected. The TC entails the ET, as well as the necrotic (fluid-filled) and the non-enhancing (solid) parts of the tumor. \n",
- "- The WT describes the complete extent of the disease, as it entails the TC and the peritumoral edema (ED), which is typically depicted by hyper-intense signal in FLAIR.\n",
- "\n",
- "To run this example, please make sure you have downloaded BraTS 2018 data, which can be obtained from [Multimodal Brain Tumor Segmentation Challenge (BraTS) 2018](https://www.med.upenn.edu/cbica/brats2018.html) [2-6]. Please download the data to [./dataset_brats18/dataset](./dataset_brats18/dataset). It should result in a sub-folder `./dataset_brats18/dataset/training`.\n",
- "In this example, we split BraTS18 dataset into [4 subsets](./dataset_brats18/datalist) for 4 clients. Each client requires at least a 12 GB GPU to run. "
+ "%%capture --no-display --no-stderr cell_output\n",
+ "!pip install -r requirements.txt"
]
},
{
@@ -50,29 +30,8 @@
"metadata": {},
"source": [
"### Differential Privacy (DP)\n",
- "[Differential Privacy (DP)](https://arxiv.org/abs/1910.00962) [7] is method for ensuring that Federated Learning (FL) preserves privacy by obfuscating the model updates sent from clients to the central server.\n",
- "This example shows the usage of a MONAI-based trainer for medical image applications with NVFlare, as well as the usage of DP filters in your FL training. DP is added as a filter in `config_fed_client.json`. Here, we use the \"Sparse Vector Technique\", i.e. the [SVTPrivacy](https://nvflare.readthedocs.io/en/main/apidocs/nvflare.app_common.filters.svt_privacy.html) protocol, as utilized in [Li et al. 2019](https://arxiv.org/abs/1910.00962) [7] (see [Lyu et al. 2016](https://arxiv.org/abs/1603.01699) [8] for more information)."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "33306153-a8c5-4c2b-9eea-28c3e0d705a6",
- "metadata": {},
- "source": [
- "## Prepare local configs\n",
- "First, we add the image and datalist directory roots to `config_train.json` files for generating the absolute path to the dataset by replacing the `DATASET_ROOT` and `DATALIST_ROOT` placeholders. In the current folder structure, it will be `${PWD}/dataset_brats18/dataset` for `DATASET_ROOT` and `${PWD}/dataset_brats18/datalist` for `DATALIST_ROOT` but you can update the below `sed` commands if the data is located somewhere else."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "id": "be8a7f47-2a93-4992-85c0-d597f4ecf3d5",
- "metadata": {},
- "outputs": [],
- "source": [
- "%%bash\n",
- "sed -i \"s|DATASET_ROOT|${PWD}/dataset_brats18/dataset|g\" config_train.json\n",
- "sed -i \"s|DATALIST_ROOT|${PWD}/dataset_brats18/datalist|g\" config_train.json"
+ "[Differential Privacy (DP)](https://arxiv.org/abs/1910.00962) [7] is a method for ensuring that Federated Learning (FL) preserves privacy by obfuscating the model updates sent from clients to the central server. \n",
+ "This example shows the usage of a CIFAR-10 training code with NVFlare, as well as the usage of DP filters in your FL training. DP is added as a filter in `config_fed_client.json`. Here, we use the \"Sparse Vector Technique\", i.e. the [SVTPrivacy](https://nvflare.readthedocs.io/en/main/apidocs/nvflare.app_common.filters.svt_privacy.html) protocol, as utilized in [Li et al. 2019](https://arxiv.org/abs/1910.00962) [7] (see [Lyu et al. 2016](https://arxiv.org/abs/1603.01699) [8] for more information)."
]
},
{
@@ -83,33 +42,8 @@
"## Run experiments with FL simulator\n",
"### Training with FL simulator\n",
"FL simulator is used to simulate FL experiments or debug codes, not for real FL deployment.\n",
- "In this example, we assume four local GPUs with at least 12GB of memory are available."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "8d256389-4112-46e6-86bd-115c9bf2e189",
- "metadata": {},
- "source": [
- "Then, we can run the FL simulator with 1 client for centralized training"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "a9583165-af58-45ed-a86d-9fbfc74d80ca",
- "metadata": {},
- "outputs": [],
- "source": [
- "!python3 -u -m nvflare.private.fed.app.simulator.simulator './configs/brats_central' -w './workspace_brats/brats_central' -n 1 -t 1 -gpu 0"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "f1121bc2-1118-4b64-8d61-06e1a49bc7ef",
- "metadata": {},
- "source": [
- "Similarly, run the FL simulator with 4 clients for federated learning by running"
+ "\n",
+ "First, train a model using the FedAvg algorithm with four clients without DP."
]
},
{
@@ -119,7 +53,7 @@
"metadata": {},
"outputs": [],
"source": [
- "!nvflare simulator './configs/brats_fedavg' -w './workspace_brats/brats_fedavg' -n 4 -t 4 -gpu 0,1,2,3"
+ "!nvflare simulator './configs/brats_fedavg' -w './workspace_brats/brats_fedavg' -n 4 -t 4"
]
},
{
@@ -137,7 +71,7 @@
"metadata": {},
"outputs": [],
"source": [
- "!nvflare simulator './configs/brats_fedavg_dp' -w './workspace_brats/brats_fedavg_dp' -n 4 -t 4 -gpu 0,1,2,3"
+ "!nvflare simulator './configs/brats_fedavg_dp' -w './workspace_brats/brats_fedavg_dp' -n 4 -t 4"
]
},
{
diff --git a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/pt/learners/supervised_learner.py b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/pt/learners/supervised_learner.py
deleted file mode 100644
index e71c16f627..0000000000
--- a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/pt/learners/supervised_learner.py
+++ /dev/null
@@ -1,329 +0,0 @@
-# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import copy
-from abc import abstractmethod
-
-import numpy as np
-import torch
-from torch.utils.tensorboard import SummaryWriter
-
-from nvflare.apis.dxo import DXO, DataKind, MetaKey, from_shareable
-from nvflare.apis.fl_constant import FLContextKey, ReturnCode
-from nvflare.apis.fl_context import FLContext
-from nvflare.apis.shareable import Shareable, make_reply
-from nvflare.apis.signal import Signal
-from nvflare.app_common.abstract.learner_spec import Learner
-from nvflare.app_common.app_constant import AppConstants, ValidateType
-
-
-class SupervisedLearner(Learner):
- def __init__(
- self,
- aggregation_epochs: int = 1,
- train_task_name: str = AppConstants.TASK_TRAIN,
- ):
- """Simple Supervised Trainer.
- This provides the basic functionality of a local learner: perform before-train validation on
- global model at the beginning of each round, perform local training, and send the updated weights.
- No model will be saved locally, tensorboard record for local loss and global model validation score.
- Enabled both FedAvg and FedProx
-
- Args:
- train_config_filename: directory of config file.
- aggregation_epochs: the number of training epochs for a round. Defaults to 1.
- train_task_name: name of the task to train the model.
-
- Returns:
- a Shareable with the updated local model after running `execute()`
- """
- super().__init__()
- # trainer init happens at the very beginning, only the basic info regarding the trainer is set here
- # the actual run has not started at this point
- self.aggregation_epochs = aggregation_epochs
- self.train_task_name = train_task_name
- # Epoch counter
- self.epoch_of_start_time = 0
- self.epoch_global = 0
- # FedProx related
- self.fedproxloss_mu = 0.0
- self.criterion_prox = None
-
- def initialize(self, parts: dict, fl_ctx: FLContext):
- # when a run starts, this is where the actual settings get initialized for trainer
-
- # set the paths according to fl_ctx
- engine = fl_ctx.get_engine()
- ws = engine.get_workspace()
- app_dir = ws.get_app_dir(fl_ctx.get_job_id())
-
- # get and print the args
- fl_args = fl_ctx.get_prop(FLContextKey.ARGS)
- self.client_id = fl_ctx.get_identity_name()
- self.log_info(
- fl_ctx,
- f"Client {self.client_id} initialized with args: \n {fl_args}",
- )
-
- # set local tensorboard writer for local validation score of global model
- self.writer = SummaryWriter(app_dir)
- # set the training-related contexts, this is task-specific
- self.train_config(fl_ctx)
-
- @abstractmethod
- def train_config(self, fl_ctx: FLContext):
- """Traning configurations customized to individual tasks
- This can be specified / loaded in any ways
- as long as they are made available for further training and validation
- some potential items include but not limited to:
- self.lr
- self.fedproxloss_mu
- self.model
- self.device
- self.optimizer
- self.criterion
- self.transform_train
- self.transform_valid
- self.transform_post
- self.train_loader
- self.valid_loader
- self.inferer
- self.valid_metric
- """
- raise NotImplementedError
-
- @abstractmethod
- def finalize(self, fl_ctx: FLContext):
- # collect threads, close files here
- pass
-
- def local_train(
- self,
- fl_ctx,
- train_loader,
- model_global,
- abort_signal: Signal,
- ):
- """Typical training logic
- Total local epochs: self.aggregation_epochs
- Load data pairs from train_loader: image / label
- Compute outputs with self.model
- Compute loss with self.criterion
- Add fedprox loss
- Update model
- """
- for epoch in range(self.aggregation_epochs):
- if abort_signal.triggered:
- return make_reply(ReturnCode.TASK_ABORTED)
- self.model.train()
- epoch_len = len(train_loader)
- self.epoch_global = self.epoch_of_start_time + epoch
- self.log_info(
- fl_ctx,
- f"Local epoch {self.client_id}: {epoch + 1}/{self.aggregation_epochs} (lr={self.lr})",
- )
- for i, batch_data in enumerate(train_loader):
- if abort_signal.triggered:
- return make_reply(ReturnCode.TASK_ABORTED)
- inputs = batch_data["image"].to(self.device)
- labels = batch_data["label"].to(self.device)
-
- # forward + backward + optimize
- outputs = self.model(inputs)
- loss = self.criterion(outputs, labels)
-
- # FedProx loss term
- if self.fedproxloss_mu > 0:
- fed_prox_loss = self.criterion_prox(self.model, model_global)
- loss += fed_prox_loss
-
- self.optimizer.zero_grad()
- loss.backward()
- self.optimizer.step()
- current_step = epoch_len * self.epoch_global + i
- self.writer.add_scalar("train_loss", loss.item(), current_step)
-
- def local_valid(
- self,
- model,
- valid_loader,
- abort_signal: Signal,
- tb_id=None,
- record_epoch=None,
- ):
- """Typical validation logic
- Load data pairs from train_loader: image / label
- Compute outputs with self.model
- Perform post transform (binarization, etc.)
- Compute evaluation metric with self.valid_metric
- Add score to tensorboard record with specified id
- """
- model.eval()
- with torch.no_grad():
- metric = 0
- for i, batch_data in enumerate(valid_loader):
- if abort_signal.triggered:
- return make_reply(ReturnCode.TASK_ABORTED)
- val_images = batch_data["image"].to(self.device)
- val_labels = batch_data["label"].to(self.device)
- # Inference
- val_outputs = self.inferer(val_images, model)
- val_outputs = self.transform_post(val_outputs)
- # Compute metric
- metric_score = self.valid_metric(y_pred=val_outputs, y=val_labels)
- metric += metric_score.item()
- # compute mean dice over whole validation set
- metric /= len(valid_loader)
- # tensorboard record id, add to record if provided
- if tb_id:
- self.writer.add_scalar(tb_id, metric_score, record_epoch)
- return metric
-
- def train(
- self,
- shareable: Shareable,
- fl_ctx: FLContext,
- abort_signal: Signal,
- ) -> Shareable:
- """Typical training task pipeline with potential HE and fedprox functionalities
- Get global model weights (potentially with HE)
- Prepare for fedprox loss
- Local training
- Return updated weights (model_diff)
- """
- if abort_signal.triggered:
- return make_reply(ReturnCode.TASK_ABORTED)
-
- # get round information
- current_round = shareable.get_header(AppConstants.CURRENT_ROUND)
- total_rounds = shareable.get_header(AppConstants.NUM_ROUNDS)
- self.log_info(fl_ctx, f"Current/Total Round: {current_round + 1}/{total_rounds}")
- self.log_info(fl_ctx, f"Client identity: {fl_ctx.get_identity_name()}")
-
- # update local model weights with received weights
- dxo = from_shareable(shareable)
- global_weights = dxo.data
-
- # Before loading weights, tensors might need to be reshaped to support HE for secure aggregation.
- local_var_dict = self.model.state_dict()
- model_keys = global_weights.keys()
- for var_name in local_var_dict:
- if var_name in model_keys:
- weights = global_weights[var_name]
- try:
- # reshape global weights to compute difference later on
- global_weights[var_name] = np.reshape(weights, local_var_dict[var_name].shape)
- # update the local dict
- local_var_dict[var_name] = torch.as_tensor(global_weights[var_name])
- except Exception as e:
- raise ValueError("Convert weight from {} failed with error: {}".format(var_name, str(e)))
- self.model.load_state_dict(local_var_dict)
-
- # local steps
- epoch_len = len(self.train_loader)
- self.log_info(fl_ctx, f"Local steps per epoch: {epoch_len}")
-
- # make a copy of model_global as reference for potential FedProx loss
- if self.fedproxloss_mu > 0:
- model_global = copy.deepcopy(self.model)
- for param in model_global.parameters():
- param.requires_grad = False
- else:
- model_global = None
-
- # local train
- self.local_train(
- fl_ctx=fl_ctx,
- train_loader=self.train_loader,
- model_global=model_global,
- abort_signal=abort_signal,
- )
- if abort_signal.triggered:
- return make_reply(ReturnCode.TASK_ABORTED)
- self.epoch_of_start_time += self.aggregation_epochs
-
- # compute delta model, global model has the primary key set
- local_weights = self.model.state_dict()
- model_diff = {}
- for name in global_weights:
- if name not in local_weights:
- continue
- model_diff[name] = np.subtract(local_weights[name].cpu().numpy(), global_weights[name], dtype=np.float32)
- if np.any(np.isnan(model_diff[name])):
- self.system_panic(f"{name} weights became NaN...", fl_ctx)
- return make_reply(ReturnCode.EXECUTION_EXCEPTION)
-
- # flush the tb writer
- self.writer.flush()
-
- # build the shareable
- dxo = DXO(data_kind=DataKind.WEIGHT_DIFF, data=model_diff)
- dxo.set_meta_prop(MetaKey.NUM_STEPS_CURRENT_ROUND, epoch_len)
-
- self.log_info(fl_ctx, "Local epochs finished. Returning shareable")
- return dxo.to_shareable()
-
- def validate(self, shareable: Shareable, fl_ctx: FLContext, abort_signal: Signal) -> Shareable:
- """Typical validation task pipeline with potential HE functionality
- Get global model weights (potentially with HE)
- Validation on local data
- Return validation score
- """
- if abort_signal.triggered:
- return make_reply(ReturnCode.TASK_ABORTED)
-
- # validation on global model
- model_owner = "global_model"
-
- # update local model weights with received weights
- dxo = from_shareable(shareable)
- global_weights = dxo.data
-
- # Before loading weights, tensors might need to be reshaped to support HE for secure aggregation.
- local_var_dict = self.model.state_dict()
- model_keys = global_weights.keys()
- n_loaded = 0
- for var_name in local_var_dict:
- if var_name in model_keys:
- weights = torch.as_tensor(global_weights[var_name], device=self.device)
- try:
- # update the local dict
- local_var_dict[var_name] = torch.as_tensor(torch.reshape(weights, local_var_dict[var_name].shape))
- n_loaded += 1
- except Exception as e:
- raise ValueError("Convert weight from {} failed with error: {}".format(var_name, str(e)))
- self.model.load_state_dict(local_var_dict)
- if n_loaded == 0:
- raise ValueError(f"No weights loaded for validation! Received weight dict is {global_weights}")
-
- # before_train_validate only, can extend to other validate types
- validate_type = shareable.get_header(AppConstants.VALIDATE_TYPE)
- if validate_type == ValidateType.BEFORE_TRAIN_VALIDATE:
- # perform valid before local train
- global_metric = self.local_valid(
- self.model,
- self.valid_loader,
- abort_signal,
- tb_id="val_metric_global_model",
- record_epoch=self.epoch_global,
- )
- if abort_signal.triggered:
- return make_reply(ReturnCode.TASK_ABORTED)
- self.log_info(fl_ctx, f"val_metric_global_model ({model_owner}): {global_metric:.4f}")
- # validation metrics will be averaged with weights at server end for best model record
- metric_dxo = DXO(data_kind=DataKind.METRICS, data={MetaKey.INITIAL_METRICS: global_metric}, meta={})
- metric_dxo.set_meta_prop(MetaKey.NUM_STEPS_CURRENT_ROUND, len(self.valid_loader))
- return metric_dxo.to_shareable()
- else:
- return make_reply(ReturnCode.VALIDATE_TYPE_UNKNOWN)
diff --git a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/pt/learners/supervised_monai_brats_learner.py b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/pt/learners/supervised_monai_brats_learner.py
deleted file mode 100644
index 9c414aee44..0000000000
--- a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/pt/learners/supervised_monai_brats_learner.py
+++ /dev/null
@@ -1,269 +0,0 @@
-# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import json
-import os
-
-import numpy as np
-import torch
-import torch.optim as optim
-from monai.data import CacheDataset, DataLoader, Dataset, load_decathlon_datalist
-from monai.inferers import SlidingWindowInferer
-from monai.losses import DiceLoss
-from monai.metrics import DiceMetric
-from monai.networks.nets.segresnet import SegResNet
-from monai.transforms import (
- Activations,
- AsDiscrete,
- Compose,
- ConvertToMultiChannelBasedOnBratsClassesd,
- DivisiblePadd,
- EnsureChannelFirstd,
- LoadImaged,
- NormalizeIntensityd,
- Orientationd,
- RandFlipd,
- RandScaleIntensityd,
- RandShiftIntensityd,
- RandSpatialCropd,
- Spacingd,
-)
-from pt.learners.supervised_learner import SupervisedLearner
-from pt.utils.custom_client_datalist_json_path import custom_client_datalist_json_path
-
-from nvflare.apis.fl_constant import ReturnCode
-from nvflare.apis.fl_context import FLContext
-from nvflare.apis.shareable import make_reply
-from nvflare.apis.signal import Signal
-from nvflare.app_common.app_constant import AppConstants
-from nvflare.app_opt.pt.fedproxloss import PTFedProxLoss
-
-
-class SupervisedMonaiBratsLearner(SupervisedLearner):
- def __init__(
- self,
- train_config_filename,
- aggregation_epochs: int = 1,
- train_task_name: str = AppConstants.TASK_TRAIN,
- ):
- """MONAI Learner for BraTS18 segmentation task.
- It inherits from SupervisedLearner.
-
- Args:
- train_config_filename: path for config file, this is an addition term for config loading
- aggregation_epochs: the number of training epochs for a round.
- train_task_name: name of the task to train the model.
-
- Returns:
- a Shareable with the updated local model after running `execute()`
- """
- super().__init__(
- aggregation_epochs=aggregation_epochs,
- train_task_name=train_task_name,
- )
- self.train_config_filename = train_config_filename
- self.config_info = None
-
- def train_config(self, fl_ctx: FLContext):
- """MONAI traning configuration
- Here, we use a json to specify the needed parameters
- """
-
- # Load training configurations json
- engine = fl_ctx.get_engine()
- ws = engine.get_workspace()
- app_config_dir = ws.get_app_config_dir(fl_ctx.get_job_id())
- train_config_file_path = os.path.join(app_config_dir, self.train_config_filename)
- if not os.path.isfile(train_config_file_path):
- self.log_error(
- fl_ctx,
- f"Training configuration file does not exist at {train_config_file_path}",
- )
- with open(train_config_file_path) as file:
- self.config_info = json.load(file)
-
- # Get the config_info
- self.lr = self.config_info["learning_rate"]
- self.fedproxloss_mu = self.config_info["fedproxloss_mu"]
- cache_rate = self.config_info["cache_dataset"]
- dataset_base_dir = self.config_info["dataset_base_dir"]
- datalist_json_path = self.config_info["datalist_json_path"]
- self.roi_size = self.config_info.get("roi_size", (224, 224, 144))
- self.infer_roi_size = self.config_info.get("infer_roi_size", (240, 240, 160))
-
- # Get datalist json
- datalist_json_path = custom_client_datalist_json_path(datalist_json_path, self.client_id)
-
- # Set datalist
- train_list = load_decathlon_datalist(
- data_list_file_path=datalist_json_path,
- is_segmentation=True,
- data_list_key="training",
- base_dir=dataset_base_dir,
- )
- valid_list = load_decathlon_datalist(
- data_list_file_path=datalist_json_path,
- is_segmentation=True,
- data_list_key="validation",
- base_dir=dataset_base_dir,
- )
- self.log_info(
- fl_ctx,
- f"Training Size: {len(train_list)}, Validation Size: {len(valid_list)}",
- )
-
- # Set the training-related context
- self.device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
- self.model = SegResNet(
- blocks_down=[1, 2, 2, 4],
- blocks_up=[1, 1, 1],
- init_filters=16,
- in_channels=4,
- out_channels=3,
- dropout_prob=0.2,
- ).to(self.device)
- self.optimizer = optim.Adam(self.model.parameters(), lr=self.lr, weight_decay=1e-5)
- self.lr_scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(self.optimizer, T_max=100)
- self.criterion = DiceLoss(
- smooth_nr=0,
- smooth_dr=1e-5,
- squared_pred=True,
- to_onehot_y=False,
- sigmoid=True,
- )
-
- if self.fedproxloss_mu > 0:
- self.log_info(fl_ctx, f"using FedProx loss with mu {self.fedproxloss_mu}")
- self.criterion_prox = PTFedProxLoss(mu=self.fedproxloss_mu)
-
- self.transform_train = Compose(
- [
- # load Nifti image
- LoadImaged(keys=["image", "label"]),
- EnsureChannelFirstd(keys="image"),
- ConvertToMultiChannelBasedOnBratsClassesd(keys="label"),
- Spacingd(
- keys=["image", "label"],
- pixdim=(1.0, 1.0, 1.0),
- mode=("bilinear", "nearest"),
- ),
- Orientationd(keys=["image", "label"], axcodes="RAS"),
- RandSpatialCropd(keys=["image", "label"], roi_size=self.roi_size, random_size=False),
- RandFlipd(keys=["image", "label"], prob=0.5, spatial_axis=0),
- RandFlipd(keys=["image", "label"], prob=0.5, spatial_axis=1),
- RandFlipd(keys=["image", "label"], prob=0.5, spatial_axis=2),
- NormalizeIntensityd(keys="image", nonzero=True, channel_wise=True),
- RandScaleIntensityd(keys="image", factors=0.1, prob=1.0),
- RandShiftIntensityd(keys="image", offsets=0.1, prob=1.0),
- ]
- )
- self.transform_valid = Compose(
- [
- LoadImaged(keys=["image", "label"]),
- EnsureChannelFirstd(keys="image"),
- ConvertToMultiChannelBasedOnBratsClassesd(keys="label"),
- Spacingd(
- keys=["image", "label"],
- pixdim=(1.0, 1.0, 1.0),
- mode=("bilinear", "nearest"),
- ),
- DivisiblePadd(keys=["image", "label"], k=32),
- Orientationd(keys=["image", "label"], axcodes="RAS"),
- NormalizeIntensityd(keys="image", nonzero=True, channel_wise=True),
- ]
- )
- self.transform_post = Compose([Activations(sigmoid=True), AsDiscrete(threshold=0.5)])
-
- # Set dataset
- if cache_rate > 0.0:
- self.train_dataset = CacheDataset(
- data=train_list,
- transform=self.transform_train,
- cache_rate=cache_rate,
- num_workers=1,
- )
- self.valid_dataset = CacheDataset(
- data=valid_list,
- transform=self.transform_valid,
- cache_rate=cache_rate,
- num_workers=1,
- )
- else:
- self.train_dataset = Dataset(
- data=train_list,
- transform=self.transform_train,
- )
- self.valid_dataset = Dataset(
- data=valid_list,
- transform=self.transform_valid,
- )
-
- self.train_loader = DataLoader(
- self.train_dataset,
- batch_size=1,
- shuffle=True,
- num_workers=1,
- )
- self.valid_loader = DataLoader(
- self.valid_dataset,
- batch_size=1,
- shuffle=False,
- num_workers=1,
- )
-
- # Set inferer and evaluation metric
- self.inferer = SlidingWindowInferer(roi_size=self.infer_roi_size, sw_batch_size=1, overlap=0.5)
- self.valid_metric = DiceMetric(include_background=True, reduction="mean")
-
- # Brats has 3 classes, so the metric computation needs some change
- def local_valid(
- self,
- model,
- valid_loader,
- abort_signal: Signal,
- tb_id=None,
- record_epoch=None,
- ):
- """Typical validation logic
- Load data pairs from train_loader: image / label
- Compute outputs with self.model
- Perform post transform (binarization, etc.)
- Compute evaluation metric with self.valid_metric
- Add score to tensorboard record with specified id
- """
- model.eval()
- with torch.no_grad():
- metric = 0
- ct = 0
- for i, batch_data in enumerate(valid_loader):
- if abort_signal.triggered:
- return make_reply(ReturnCode.TASK_ABORTED)
- val_images = batch_data["image"].to(self.device)
- val_labels = batch_data["label"].to(self.device)
- # Inference
- val_outputs = self.inferer(val_images, model)
- val_outputs = self.transform_post(val_outputs)
- # Compute metric
- metric_score = self.valid_metric(y_pred=val_outputs, y=val_labels)
- for sub_region in range(3):
- metric_score_single = metric_score[0][sub_region].item()
- if not np.isnan(metric_score_single):
- metric += metric_score_single
- ct += 1
- # compute mean dice over whole validation set
- metric /= ct
- # tensorboard record id, add to record if provided
- if tb_id:
- self.writer.add_scalar(tb_id, metric, record_epoch)
- return metric
diff --git a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/pt/utils/custom_client_datalist_json_path.py b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/pt/utils/custom_client_datalist_json_path.py
deleted file mode 100644
index df685536ed..0000000000
--- a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/pt/utils/custom_client_datalist_json_path.py
+++ /dev/null
@@ -1,30 +0,0 @@
-# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import os
-
-
-def custom_client_datalist_json_path(datalist_json_path: str, client_id: str) -> str:
- """
- Customize datalist_json_path for each client
- Args:
- datalist_json_path: root path containing all jsons
- client_id: e.g., site-2
- """
- # Customize datalist_json_path for each client
- datalist_json_path_client = os.path.join(
- datalist_json_path,
- client_id + ".json",
- )
- return datalist_json_path_client
diff --git a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/requirements.txt b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/requirements.txt
deleted file mode 100644
index 4f2d4382b8..0000000000
--- a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/requirements.txt
+++ /dev/null
@@ -1,10 +0,0 @@
-nvflare~=2.5.0rc
-torch
-torchvision
-tensorboard
-monai
-tqdm
-nibabel
-tensorflow
-seaborn
-matplotlib
diff --git a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/brats_3d_test_only.py b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/brats_3d_test_only.py
deleted file mode 100644
index 883dad7132..0000000000
--- a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/brats_3d_test_only.py
+++ /dev/null
@@ -1,150 +0,0 @@
-# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import argparse
-
-import numpy as np
-import torch
-from monai.data import DataLoader, Dataset, load_decathlon_datalist
-from monai.inferers import SlidingWindowInferer
-from monai.metrics import DiceMetric
-from monai.networks.nets.segresnet import SegResNet
-from monai.transforms import (
- Activations,
- AsDiscrete,
- Compose,
- ConvertToMultiChannelBasedOnBratsClassesd,
- DivisiblePadd,
- EnsureChannelFirstd,
- LoadImaged,
- NormalizeIntensityd,
- Orientationd,
- Spacingd,
-)
-
-
-def main():
- parser = argparse.ArgumentParser(description="Model Testing")
- parser.add_argument("--model_path", type=str)
- parser.add_argument("--dataset_base_dir", default="../dataset_brats18/dataset", type=str)
- parser.add_argument("--datalist_json_path", default="../dataset_brats18/datalist/site-All.json", type=str)
- args = parser.parse_args()
-
- # Set basic settings and paths
- dataset_base_dir = args.dataset_base_dir
- datalist_json_path = args.datalist_json_path
- model_path = args.model_path
- infer_roi_size = (240, 240, 160)
-
- device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
-
- # Set datalists
- test_list = load_decathlon_datalist(
- data_list_file_path=datalist_json_path,
- is_segmentation=True,
- data_list_key="validation",
- base_dir=dataset_base_dir,
- )
- print(f"Testing Size: {len(test_list)}")
-
- # Network, optimizer, and loss
- model = SegResNet(
- blocks_down=[1, 2, 2, 4],
- blocks_up=[1, 1, 1],
- init_filters=16,
- in_channels=4,
- out_channels=3,
- dropout_prob=0.2,
- ).to(device)
- model_weights = torch.load(model_path)
- model_weights = model_weights["model"]
- model.load_state_dict(model_weights)
-
- # Inferer, evaluation metric
- inferer = SlidingWindowInferer(roi_size=infer_roi_size, sw_batch_size=1, overlap=0.5)
- valid_metric = DiceMetric(include_background=True, reduction="mean")
-
- transform = Compose(
- [
- LoadImaged(keys=["image", "label"]),
- EnsureChannelFirstd(keys="image"),
- ConvertToMultiChannelBasedOnBratsClassesd(keys="label"),
- Spacingd(
- keys=["image", "label"],
- pixdim=(1.0, 1.0, 1.0),
- mode=("bilinear", "nearest"),
- ),
- DivisiblePadd(keys=["image", "label"], k=32),
- Orientationd(keys=["image", "label"], axcodes="RAS"),
- NormalizeIntensityd(keys="image", nonzero=True, channel_wise=True),
- ]
- )
- transform_post = Compose([Activations(sigmoid=True), AsDiscrete(threshold=0.5)])
-
- # Set dataset
- test_dataset = Dataset(data=test_list, transform=transform)
- test_loader = DataLoader(
- test_dataset,
- batch_size=1,
- shuffle=False,
- num_workers=1,
- )
-
- # Train
- model.eval()
- with torch.no_grad():
- metric = 0
- metric_tc = 0
- metric_wt = 0
- metric_et = 0
- ct = 0
- ct_tc = 0
- ct_wt = 0
- ct_et = 0
- for i, batch_data in enumerate(test_loader):
- images = batch_data["image"].to(device)
- labels = batch_data["label"].to(device)
- # Inference
- outputs = inferer(images, model)
- outputs = transform_post(outputs)
- # Compute metric
- metric_score = valid_metric(y_pred=outputs, y=labels)
- if not np.isnan(metric_score[0][0].item()):
- metric += metric_score[0][0].item()
- ct += 1
- metric_tc += metric_score[0][0].item()
- ct_tc += 1
- if not np.isnan(metric_score[0][1].item()):
- metric += metric_score[0][1].item()
- ct += 1
- metric_wt += metric_score[0][1].item()
- ct_wt += 1
- if not np.isnan(metric_score[0][2].item()):
- metric += metric_score[0][2].item()
- ct += 1
- metric_et += metric_score[0][2].item()
- ct_et += 1
- # compute mean dice over whole validation set
- metric_tc /= ct_tc
- metric_wt /= ct_wt
- metric_et /= ct_et
- metric /= ct
- print(f"Test Dice: {metric:.4f}, Valid count: {ct}")
- print(f"Test Dice TC: {metric_tc:.4f}, Valid count: {ct_tc}")
- print(f"Test Dice WT: {metric_wt:.4f}, Valid count: {ct_wt}")
- print(f"Test Dice ET: {metric_et:.4f}, Valid count: {ct_et}")
-
-
-if __name__ == "__main__":
- main()
diff --git a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/plot_tensorboard_events.py b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/plot_tensorboard_events.py
deleted file mode 100644
index 075fda9ed6..0000000000
--- a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/plot_tensorboard_events.py
+++ /dev/null
@@ -1,117 +0,0 @@
-# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import glob
-import os
-
-import matplotlib.pyplot as plt
-import seaborn as sns
-import tensorflow as tf
-
-# poc workspace
-client_results_root = "../workspace_brats/"
-
-# All sites used the same validation set for Brats, so only 1 site's record is needed
-site_num = 1
-client_pre = "app_site-"
-sites_fl = [str(site + 1) for site in range(site_num)]
-
-# Central vs. FedAvg vs. FedAvg_DP
-experiments = {
- "brats_central": {"tag": "val_metric_global_model", "site": "All"},
- "brats_fedavg": {"tag": "val_metric_global_model"},
- "brats_fedavg_dp": {"tag": "val_metric_global_model"},
-}
-
-weight = 0.8
-
-
-def smooth(scalars, weight): # Weight between 0 and 1
- last = scalars[0] # First value in the plot (first timestep)
- smoothed = list()
- for point in scalars:
- smoothed_val = last * weight + (1 - weight) * point # Calculate smoothed value
- smoothed.append(smoothed_val) # Save it
- last = smoothed_val # Anchor the last smoothed value
- return smoothed
-
-
-def read_eventfile(filepath, tags=["val_metric_global_model"]):
- data = {}
- for summary in tf.compat.v1.train.summary_iterator(filepath):
- for v in summary.summary.value:
- if v.tag in tags:
- if v.tag in data.keys():
- data[v.tag].append([summary.step, v.simple_value])
- else:
- data[v.tag] = [[summary.step, v.simple_value]]
- return data
-
-
-def add_eventdata(data, config, filepath, tag="val_metric_global_model"):
- event_data = read_eventfile(filepath, tags=[tag])
- assert len(event_data[tag]) > 0, f"No data for key {tag}"
-
- metric = []
- for e in event_data[tag]:
- # print(e)
- data["Config"].append(config)
- data["Epoch"].append(e[0])
- metric.append(e[1])
-
- metric = smooth(metric, weight)
- for entry in metric:
- data["Dice"].append(entry)
-
- print(f"added {len(event_data[tag])} entries for {tag}")
-
-
-def main():
- plt.figure()
- num_site = len(sites_fl)
- i = 1
- # add event files
-
- data = {"Config": [], "Epoch": [], "Dice": []}
-
- for site in sites_fl:
- # clear data for each site
- data = {"Config": [], "Epoch": [], "Dice": []}
- for config, exp in experiments.items():
- spec_site = exp.get("site", None)
- if spec_site is not None:
- record_path = os.path.join(
- client_results_root + config, "simulate_job", client_pre + spec_site, "events.*"
- )
- else:
- record_path = os.path.join(client_results_root + config, "simulate_job", client_pre + site, "events.*")
-
- eventfile = glob.glob(record_path, recursive=True)
- print(record_path, len(eventfile))
- assert len(eventfile) == 1, "No unique event file found!"
- eventfile = eventfile[0]
- print("adding", eventfile)
- add_eventdata(data, config, eventfile, tag=exp["tag"])
-
- ax = plt.subplot(1, num_site, i)
- ax.set_title(site)
- sns.lineplot(x="Epoch", y="Dice", hue="Config", data=data)
- # ax.set_xlim([0, 1000])
- i = i + 1
- plt.subplots_adjust(hspace=0.3)
- plt.show()
-
-
-if __name__ == "__main__":
- main()
diff --git a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/plot_tensorboard_events_poc.py b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/plot_tensorboard_events_poc.py
deleted file mode 100644
index f4b4ee55df..0000000000
--- a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/plot_tensorboard_events_poc.py
+++ /dev/null
@@ -1,125 +0,0 @@
-# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import glob
-import os
-
-import matplotlib.pyplot as plt
-import seaborn as sns
-import tensorflow as tf
-
-# poc workspace
-client_results_root = "../workspace_brats"
-
-# All sites used the same validation set, so only 1 site's record is needed
-site_num = 1
-site_pre = "site-"
-
-# Central vs. FedAvg vs. FedAvg_DP
-experiments = {
- "brats_central": {"tag": "val_metric_global_model", "site": "All"},
- "brats_fedavg": {"tag": "val_metric_global_model"},
- "brats_fedavg_dp": {"tag": "val_metric_global_model"},
-}
-
-weight = 0.8
-
-
-def smooth(scalars, weight): # Weight between 0 and 1
- last = scalars[0] # First value in the plot (first timestep)
- smoothed = list()
- for point in scalars:
- smoothed_val = last * weight + (1 - weight) * point # Calculate smoothed value
- smoothed.append(smoothed_val) # Save it
- last = smoothed_val # Anchor the last smoothed value
- return smoothed
-
-
-def find_job_id(workdir, fl_app_name="prostate_central"):
- """Find the first matching experiment"""
- target_path = os.path.join(workdir, "*", "fl_app.txt")
- fl_app_files = glob.glob(target_path, recursive=True)
- assert len(fl_app_files) > 0, f"No `fl_app.txt` files found in workdir={workdir}."
- for fl_app_file in fl_app_files:
- with open(fl_app_file, "r") as f:
- _fl_app_name = f.read()
- if fl_app_name == _fl_app_name: # alpha will be matched based on value in config file
- job_id = os.path.basename(os.path.dirname(fl_app_file))
- return job_id
- raise ValueError(f"No job id found for fl_app_name={fl_app_name} in workdir={workdir}")
-
-
-def read_eventfile(filepath, tags=["val_metric_global_model"]):
- data = {}
- for summary in tf.compat.v1.train.summary_iterator(filepath):
- for v in summary.summary.value:
- if v.tag in tags:
- if v.tag in data.keys():
- data[v.tag].append([summary.step, v.simple_value])
- else:
- data[v.tag] = [[summary.step, v.simple_value]]
- return data
-
-
-def add_eventdata(data, config, filepath, tag="val_metric_global_model"):
- event_data = read_eventfile(filepath, tags=[tag])
- assert len(event_data[tag]) > 0, f"No data for key {tag}"
-
- metric = []
- for e in event_data[tag]:
- # print(e)
- data["Config"].append(config)
- data["Epoch"].append(e[0])
- metric.append(e[1])
-
- metric = smooth(metric, weight)
- for entry in metric:
- data["Dice"].append(entry)
-
- print(f"added {len(event_data[tag])} entries for {tag}")
-
-
-def main():
- plt.figure()
- i = 1
- # add event files
- data = {"Config": [], "Epoch": [], "Dice": []}
- for site in range(site_num):
- # clear data for each site
- site = site + 1
- data = {"Config": [], "Epoch": [], "Dice": []}
- for config, exp in experiments.items():
- job_id = find_job_id(workdir=client_results_root + "/site-1", fl_app_name=config)
- print(f"Found run {job_id} for {config}")
- spec_site = exp.get("site", None)
- if spec_site is not None:
- record_path = os.path.join(client_results_root, site_pre + spec_site, job_id, "*", "events.*")
- else:
- record_path = os.path.join(client_results_root, site_pre + str(site), job_id, "*", "events.*")
- eventfile = glob.glob(record_path, recursive=True)
- assert len(eventfile) == 1, "No unique event file found!"
- eventfile = eventfile[0]
- print("adding", eventfile)
- add_eventdata(data, config, eventfile, tag=exp["tag"])
-
- ax = plt.subplot(1, site_num, i)
- ax.set_title(site)
- sns.lineplot(x="Epoch", y="Dice", hue="Config", data=data)
- i = i + 1
- plt.subplots_adjust(hspace=0.3)
- plt.show()
-
-
-if __name__ == "__main__":
- main()
diff --git a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/testing_models_3d.sh b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/testing_models_3d.sh
deleted file mode 100755
index 0811a27e20..0000000000
--- a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/testing_models_3d.sh
+++ /dev/null
@@ -1,11 +0,0 @@
-workspace_path="../workspace_brats"
-dataset_path="../dataset_brats18/dataset"
-datalist_path="../dataset_brats18/datalist"
-
-echo "Centralized"
-python3 brats_3d_test_only.py --model_path "${workspace_path}/brats_central/simulate_job/app_server/best_FL_global_model.pt" --dataset_base_dir ${dataset_path} --datalist_json_path "${datalist_path}/site-All.json"
-echo "FedAvg"
-python3 brats_3d_test_only.py --model_path "${workspace_path}/brats_fedavg/simulate_job/app_server/best_FL_global_model.pt" --dataset_base_dir ${dataset_path} --datalist_json_path "${datalist_path}/site-All.json"
-echo "FedAvgDP"
-python3 brats_3d_test_only.py --model_path "${workspace_path}/brats_fedavg_dp/simulate_job/app_server/best_FL_global_model.pt" --dataset_base_dir ${dataset_path} --datalist_json_path "${datalist_path}/site-All.json"
-
diff --git a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/testing_models_3d_poc.sh b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/testing_models_3d_poc.sh
deleted file mode 100755
index 05d4ba78da..0000000000
--- a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.1_privacy_filter/result_stat/testing_models_3d_poc.sh
+++ /dev/null
@@ -1,11 +0,0 @@
-# Replace the job_ids with the ones from your workspace
-job_id_cen="b6c7e274-67f9-402e-8fcd-f54f3cea40a9"
-job_id_avg="9b05aa10-c79f-444c-b6b0-2ad7a3538e79"
-job_id_avg_dp="c98bdbaf-fdf6-4786-b905-b6a1ba7e398c"
-
-echo "Centralized"
-python3 brats_3d_test_only.py --model_path "${workspace_path}/server/transfer/${job_id_cen}/workspace/app_server/best_FL_global_model.pt" --dataset_base_dir ${dataset_path} --datalist_json_path "${datalist_path}/site-All.json"
-echo "FedAvg"
-python3 brats_3d_test_only.py --model_path "${workspace_path}/server/transfer/${job_id_avg}/workspace/app_server/best_FL_global_model.pt" --dataset_base_dir ${dataset_path} --datalist_json_path "${datalist_path}/site-All.json"
-echo "FedAvgDP"
-python3 brats_3d_test_only.py --model_path "${workspace_path}/server/transfer/${job_id_avg_dp}/workspace/app_server/best_FL_global_model.pt" --dataset_base_dir ${dataset_path} --datalist_json_path "${datalist_path}/site-All.json"
diff --git a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.2_differency_privacy/privacy_with_differential_privacy.ipynb b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.2_differency_privacy/privacy_with_differential_privacy.ipynb
index e52a52b58d..924fd7e69f 100644
--- a/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.2_differency_privacy/privacy_with_differential_privacy.ipynb
+++ b/examples/tutorials/self-paced-training/part-3_security_and_privacy/chapter-5_Privacy_In_Federated_Learning/05.2_differency_privacy/privacy_with_differential_privacy.ipynb
@@ -1,9 +1,798 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "id": "1398ef0a-f189-4d04-a8a9-276a17ab0f8b",
+ "metadata": {},
+ "source": [
+ "# Federated Learning with Differential Privacy\n",
+ "\n",
+ "Please make sure you set up a virtual environment and follow [example root readme](../../README.md) before starting this notebook.\n",
+ "Then, install the requirements.\n",
+ "\n",
+ "%%capture --no-display --no-stderr cell_outputto suppress this output. Comment or delete this line in the cells below to restore full output.