Submitting your solution¶

LUNA22-ISMI is a type 2 challenge. In such a challenge, you are expected to submit your solution as an Algorithm container on grand-challenge.org. This container will then be executed on a hidden test set (which is not available to you).

Wrapping an algorithm into a container, however, is not a trivial task. The documentation on grand-challenge.org has a tutorial with an example of wrapping a segmentation algorithm into a container (a U-Net that segments blood vessels in a retinal fundus image). You may choose to go through the documentation in detail (which gives you full control and knowledge of what you are doing), or you may follow a much simpler set of instructions given below (to kickstart your submission process).

📢 Please note that you must be a verified user on grand-challenge.org to be able to create algorithms for this challenge. 

For this challenge, we have provided an example repository for creating a baseline algorithm container. Please follow the instructions presented below to make your own submission to this challenge.

Step 1 - Fork the example repository¶

• Fork this repository into your personal GitHub account
• Make sure to choose your GitHub username as the Owner
• Install git-lfs
• Clone the forked repository onto your laptop or server

💡If you cloned your repository after installing git-lfs, then git should automatically pull the large files from the repository as part of the cloning process.  But if you cloned the repository before installing git-lfs, you might only have the pointers to the files stored with git-lfs. Execute the following to pull the large files:

  $ git lfs fetch --all
  $ git lfs pull

Step 2 - Modify the forked repository¶

Replace or update the following files with your own versions:

• The weights of your neural networks under models/
• The init function in process.py, if necessary
• The preprocess and predict functions in process.py, if necessary
• [optional] expected_outputs.json (this is very useful if you are testing your container locally. You can obtain the expected malignancy_risk and nodule_type outputs of your trained models by modifying and running the inference script provided in the baseline training repository.)

💡Do not forget to commit and push your changes to the forked repository.

Step 3 - Create an Algorithm page¶

• Give an appropriate title for your algorithm (for example, LUNA22-ISMI-Team11)
• Enter your contact email
• Display Editors: Yes
• Modalities: CT
• Structures: Lung (Thorax)
• Logo: Upload a square logo for your Algorithm (perhaps representative of your team)
• Inputs: CT Image (Image)
• Outputs:
  • Lung nodule malignancy risk (Float)
  • Lung nodule type (Integer)
• Make sure Image requires gpu is checked
• Specify the number of gbs required for your algorithm under Image requires memory gb
• Scroll down and click on Save

Step 4 - Link the forked repository to your Algorithm¶

Follow this documentation to first install the Grand Challenge app in your GitHub account, and then link the forked repository to your Algorithm.

💡Please note that you may have to wait for a couple of hours for grand-challenge.org to build your container. 

Step 5 - Testing your Algorithm on grand-challenge.org¶

• Navigate to your algorithm page
• Click on Try-out Algorithm on the panel on your left-hand side
• Upload a nodule image and click on Submit
• Wait for a while for Grand Challenge to process your image
• Go back to your algorithm page and click on Results
• If the algorithm job was successful, you would see "Succeeded" in the Result column
• Click on the result icon and then click on View Result Details to inspect the outputs of your algorithm. Ideally, this should match what you observe on your laptop or server.

Step 6 - Submitting your Algorithm to the LUNA22-ISMI Challenge¶

• Click on the Submit button at the top of this challenge website
• Select your Algorithm in the drop-down menu
• Click on Save

💡grand-challenge.org can be busy sometimes, so please be patient until your algorithm is executed on the test set.

Step 7 - Visualizing your results on the Leaderboard¶

• Click on the Leaderboard button at the top of this challenge website
• If your evaluation was successful, your result will be automatically populated on the leaderboard

Step 8 - Updating your Algorithm¶

You do not have to create an algorithm page every time you want to update your submission. You can simply update your repository with the newest version of your algorithm, and tag a new release to trigger a new build. As usual, you should wait for the new build to become active, and then you can proceed forward to Steps 5 and 6.