Documentation

PARROT: Platform for ARtificial intelligence guided Radiation Oncology Treatment

PARROT is a local web application. It provides different modules that allow users to:

Run AI models.
Visualize CT and (soon) MR images.
Visualize, predict or modify segmentation information.
Visualize, predict, evaluate, and compare dose distributions.

Installation steps

PARROT

There are two ways to install PARROT, downloading the executable file of the last release from the Home page is recommended. Otherwise, the code with building instructions are also available on our Gitlab repository.

Orthanc

Patient data are stored using a Orthanc daemon, a lightweight, standalone DICOM server, which ensures the confidentiality of patient files. To install Orthanc, either click on the link on PARROT's home tab (see Figure 1) or click on the following link: Install Orthanc.

⚠ The Orthanc server will run on port 8042 while nginx server for the application will run on port 2000.

Organisation of the application

The application is divided in multiple tabs that users can visit through the top navigation bar. This section provides a description of each tabs and their features.

Patient Management

Users can import patient DICOM data (files or zip) from this screen to a local Orthanc daemon. A list of patient data present in the Orthanc server is displayed with summarized information. Users can load the patient data to PARROT by clicking on the Load button. Users can delete a patient from the server with the red arrow on the right.

Study Management

A list of studies of the loaded patient present in the Orthanc server is displayed with summarized information. Users can load the data into the app by clicking on the Load button. The left icon displays more complete information from DICOM tags and users can delete a patient study from the server with the red arrow on the right.

AI Marketplace

Eight segmentation models and six dose prediction models are made available on HuggingFace and can be downloaded from the AI Marketplace screen (Figure 4). These models are trained from state-of-the-art architectures such as nnUNet [1], SwinUNETR [2] and HDUNet [3] (AI models). We distinguish two categories of models: the segmentation models and the dose prediction models. A key difference between the two categories is that segmentation models require only the anatomy of the patient (CT scan or MRI image) whereas dose prediction models also require binary masks of tumor volumes (TV) and organs at risk (OARs). For that latter, we implemented a dialog that allows users to map structures present in the current patient data and the input required for the AI dose prediction model (Figure 7).

To be able to use these models, we also provide a Dockerfile that is automatically used to generate a Docker container to then run the models.

Run AI Models

AI Models Management Image — Figure 5: AI Models Management screen.

To run a model prediction, the user simply clicks on the yellow Prediction button on the right hand side of the model line. This will open a Python scripting editor to run the code (Figure 6). Two choices are available to the user: either run the code locally or in one of the Python environments provided. Users also have the flexibility to run their own models by loading their own code in the Python scripting editor.

Code editor — Figure 6: Embedded python code editor.

Patient Modeling

Once the prediction is done, users can upload the result to the study to visualize the output in the Patient Modeling screen (Figure 8).

Plan Evaluation

A comprehensive display of two dose distributions comparison is presented on the last screen (see Figure 9). The two dose distributions can originate from the importation step or result from the inference of ai models. We provide a set of tools for comparison, such as DVH curves, dose statistics, and the fulfillment of clinical goals. Clinical goals can be uploaded following an Excel template so that it fits any clinical requirements of the users institutions. The platform incorporates treatment indications based on NTCP protocols. It has been shown in multiple studies that predicted dose distributions yield meaningful output when used with NTCP models for treatment selection decision support [4][5]. This is a useful decision support tool for the clinical community.

References

[1] Isensee, F., Jaeger, P. F., Kohl, S. A., Petersen, J., & Maier-Hein, K. H. (2021). nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2), 203-211.

[2] Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H. and Xu, D., 2022. Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images. arXiv preprint arXiv:2201.01266.

[3] Nguyen D, Jia X, Sher D, Lin MH, Iqbal Z, Liu H, Jiang S. 3D radiotherapy dose prediction on head and neck cancer patients with a hierarchically densely connected U-net deep learning architecture. Phys Med Biol. 2019 Mar 18;64(6):065020. doi: 10.1088/1361-6560/ab039b. PMID: 30703760.

[4] Huet-Dastarac M, Michiels S, Rivas ST, et al. Patient selection for proton therapy using Normal Tissue Complication Probability with deep learning dose prediction for oropharyngeal cancer. Med Phys. 2023; 50: 6201–6214. https://doi.org/10.1002/mp.16431

[5] Draguet C., Barragán-Montero A., Chocan Vera M., Thomas M., Populaire P., Defraene G., Haustermans K., Lee J., Sterpin E., Automated clinical decision support system with deep learning dose prediction and NTCP models to evaluate treatment complications in patients with esophageal cancer, Radiotherapy and Oncology, Volume 176, 2022, https://doi.org/10.1016/j.radonc.2022.08.031.

Stopping the application

The nginx server will continue running, even when you close the web browser window. To completely stop the server, double click the 'Stop_PARROT.bat' file in the 'parrot' folder (or stop the nginx processes in the Task Manager).
The Orthanc server runs as a Windows service and has to be stopped in the 'Task Manager' (in the tab 'Services'). Don't forget to start the service again when you want to use the 'PARROT application'.