Skip to content

Commit 966977c

Browse files
jo-muellerjo-mueller
authored
Merge pull request #12 from lmanan/main
Exercise notebooks for session on DL
2 parents 7cec257 + 5f5129c commit 966977c

File tree

2 files changed

+828
-0
lines changed

2 files changed

+828
-0
lines changed

‎11_deep_learning/01-Image-Restoration.ipynb‎

Lines changed: 395 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,395 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"### This notebook is adapted from **https://github.com/juglab/n2v**\n",
8+
"In this notebook, we will denoise some Scanning Electron Microscopy Images using an approach called Noise2Void. \n",
9+
"Through this notebook, you will see the complete workflow of an DL approach - (i) data preparation, followed by (ii) training the model and finally (iii) applying the trained model on the test, microscopy images. \n",
10+
"You will find some questions (indicated as `Q`s) for discussion in blue boxes in this notebook :) \n",
11+
"<hr>\n",
12+
"Prior to running this notebook, you need to have the correct environment configured. \n",
13+
"\n",
14+
"#### If not running on google colab\n",
15+
"Open a fresh terminal window and run the following commands:\n",
16+
"\n",
17+
">conda create -n 'dl-biapol' python=3.7 \n",
18+
"conda activate dl-biapol \n",
19+
"pip install tensorflow-gpu=2.4.1 keras=2.3.1 n2v jupyter scikit-image gputools\n",
20+
"\n",
21+
"Finally open this notebook using `jupyter notebook`\n",
22+
"\n",
23+
"#### If running on google colab\n",
24+
"Go to `File>Upload Notebook` and drag and drop this notebook. \n",
25+
"Go to `Runtime > Change Runtime Type > Hardware Accelerator = GPU` \n",
26+
"Create an empty cell following this one and run:\n",
27+
">!pip install tensorflow-gpu==2.4.1 keras==2.3.1 n2v scikit-image gputools\n"
28+
]
29+
},
30+
{
31+
"cell_type": "markdown",
32+
"metadata": {},
33+
"source": [
34+
"### Get imports"
35+
]
36+
},
37+
{
38+
"cell_type": "markdown",
39+
"metadata": {},
40+
"source": [
41+
">We import all our dependencies."
42+
]
43+
},
44+
{
45+
"cell_type": "code",
46+
"execution_count": null,
47+
"metadata": {},
48+
"outputs": [],
49+
"source": [
50+
"from n2v.models import N2VConfig, N2V\n",
51+
"import numpy as np\n",
52+
"from csbdeep.utils import plot_history\n",
53+
"from n2v.utils.n2v_utils import manipulate_val_data\n",
54+
"from n2v.internals.N2V_DataGenerator import N2V_DataGenerator\n",
55+
"from matplotlib import pyplot as plt\n",
56+
"from tifffile import imread\n",
57+
"import urllib, os, zipfile, ssl\n",
58+
"ssl._create_default_https_context = ssl._create_unverified_context"
59+
]
60+
},
61+
{
62+
"cell_type": "markdown",
63+
"metadata": {},
64+
"source": [
65+
"### Download example data"
66+
]
67+
},
68+
{
69+
"cell_type": "markdown",
70+
"metadata": {},
71+
"source": [
72+
">Data by Reza Shahidi and Gaspar Jekely, Living Systems Institute, Exeter. \n",
73+
"You could try opening the <i> train.tif </i> and <i> validation.tif </i> in Fiji."
74+
]
75+
},
76+
{
77+
"cell_type": "code",
78+
"execution_count": null,
79+
"metadata": {},
80+
"outputs": [],
81+
"source": [
82+
"# create a folder for our data.\n",
83+
"if not os.path.isdir('./data'):\n",
84+
" os.mkdir('./data')\n",
85+
"\n",
86+
"# check if data has been downloaded already\n",
87+
"zipPath=\"data/SEM.zip\"\n",
88+
"if not os.path.exists(zipPath):\n",
89+
" #download and unzip data\n",
90+
" data = urllib.request.urlretrieve('https://download.fht.org/jug/n2v/SEM.zip', zipPath)\n",
91+
" with zipfile.ZipFile(zipPath, 'r') as zip_ref:\n",
92+
" zip_ref.extractall(\"data\")\n",
93+
"\n"
94+
]
95+
},
96+
{
97+
"cell_type": "markdown",
98+
"metadata": {},
99+
"source": [
100+
"### Training Data Preparation"
101+
]
102+
},
103+
{
104+
"cell_type": "markdown",
105+
"metadata": {},
106+
"source": [
107+
">For training we load one set of low-SNR images and use the `N2V_DataGenerator` to extract training `X` and validation `X_val` patches. "
108+
]
109+
},
110+
{
111+
"cell_type": "code",
112+
"execution_count": null,
113+
"metadata": {},
114+
"outputs": [],
115+
"source": [
116+
"datagen = N2V_DataGenerator()"
117+
]
118+
},
119+
{
120+
"cell_type": "markdown",
121+
"metadata": {},
122+
"source": [
123+
">We load all the `.tif` files from the `data` directory. \n",
124+
"The function will return a list of images (numpy arrays)."
125+
]
126+
},
127+
{
128+
"cell_type": "code",
129+
"execution_count": null,
130+
"metadata": {},
131+
"outputs": [],
132+
"source": [
133+
"imgs = datagen.load_imgs_from_directory(directory = \"data/\")"
134+
]
135+
},
136+
{
137+
"cell_type": "markdown",
138+
"metadata": {},
139+
"source": [
140+
">Let us look at the images. \n",
141+
"We have to remove the added extra dimensions to display them as `2D` images."
142+
]
143+
},
144+
{
145+
"cell_type": "code",
146+
"execution_count": null,
147+
"metadata": {},
148+
"outputs": [],
149+
"source": [
150+
"plt.imshow(imgs[0][0,...,0], cmap='magma')\n",
151+
"plt.show()\n",
152+
"\n",
153+
"plt.imshow(imgs[1][0,...,0], cmap='magma')\n",
154+
"plt.show()"
155+
]
156+
},
157+
{
158+
"cell_type": "markdown",
159+
"metadata": {},
160+
"source": [
161+
">We will use the first image to extract training patches and store them in `X`. \n",
162+
"We will use the second image to extract validation patches."
163+
]
164+
},
165+
{
166+
"cell_type": "markdown",
167+
"metadata": {},
168+
"source": [
169+
"### Training"
170+
]
171+
},
172+
{
173+
"cell_type": "code",
174+
"execution_count": null,
175+
"metadata": {},
176+
"outputs": [],
177+
"source": [
178+
"patch_shape = (96,96)\n",
179+
"X = datagen.generate_patches_from_list(imgs[:1], shape=patch_shape)\n",
180+
"X_val = datagen.generate_patches_from_list(imgs[1:], shape=patch_shape)"
181+
]
182+
},
183+
{
184+
"cell_type": "markdown",
185+
"metadata": {},
186+
"source": [
187+
"<div class=\"alert alert-block alert-info\"> Q:<b> Why do you think input images chopped into smaller patches? </b>.<br>\n",
188+
" What could be different schemes for extracting patches? \n",
189+
"</div>"
190+
]
191+
},
192+
{
193+
"cell_type": "markdown",
194+
"metadata": {},
195+
"source": [
196+
">Using `N2VConfig` we specify some training parameters. \n",
197+
"For example, `train_epochs` is set to $20$ to indicate that training runs for $20$ epochs. "
198+
]
199+
},
200+
{
201+
"cell_type": "code",
202+
"execution_count": null,
203+
"metadata": {},
204+
"outputs": [],
205+
"source": [
206+
"config = N2VConfig(X, unet_kern_size=3, \n",
207+
" train_steps_per_epoch=int(X.shape[0]/128), train_epochs=20, train_loss='mse', batch_norm=True, \n",
208+
" train_batch_size=128, n2v_perc_pix=0.198, n2v_patch_shape=(64, 64), \n",
209+
" n2v_manipulator='uniform_withCP', n2v_neighborhood_radius=5)"
210+
]
211+
},
212+
{
213+
"cell_type": "markdown",
214+
"metadata": {},
215+
"source": [
216+
">`model_name` is used to identify the model. \n",
217+
"`basedir` is used to specify where the trained weights are saved. \n",
218+
"We shall now create our network model by using the `N2V` method."
219+
]
220+
},
221+
{
222+
"cell_type": "markdown",
223+
"metadata": {},
224+
"source": [
225+
"<div class=\"alert alert-block alert-info\"> Q: <b> Can you identify what is the default learning rate used while training the N2V model? </b>.<br>\n",
226+
"<u> HINT </u> Pressing <i> Shift + Tab </i> shows the docstring for a given function\n",
227+
"</div>"
228+
]
229+
},
230+
{
231+
"cell_type": "markdown",
232+
"metadata": {},
233+
"source": [
234+
"<div class=\"alert alert-block alert-info\"> Q: <b> Is it advantageous to set a high batch size? How about a low batch size? </b><br> \n",
235+
"Also, discuss what would setting <i>BatchNorm = True</i> imply? \n",
236+
"</div>\n"
237+
]
238+
},
239+
{
240+
"cell_type": "code",
241+
"execution_count": null,
242+
"metadata": {},
243+
"outputs": [],
244+
"source": [
245+
"model_name = 'n2v_2D'\n",
246+
"basedir = 'models'\n",
247+
"model = N2V(config, model_name, basedir=basedir)"
248+
]
249+
},
250+
{
251+
"cell_type": "markdown",
252+
"metadata": {},
253+
"source": [
254+
">Running `model.train` will begin the training for $20$ epochs. \n",
255+
"Training the model will likely take some time. "
256+
]
257+
},
258+
{
259+
"cell_type": "code",
260+
"execution_count": null,
261+
"metadata": {},
262+
"outputs": [],
263+
"source": [
264+
"history = model.train(X, X_val)"
265+
]
266+
},
267+
{
268+
"cell_type": "markdown",
269+
"metadata": {},
270+
"source": [
271+
"<div class=\"alert alert-block alert-info\"> Q: <b> What do you think is the difference between <i>n2v_mse</i> and <i>n2v_abs</i>? </b>.<br>\n",
272+
"Also, the last line which is printed out is <i>Loading network weights from 'weights_best.h5'</i>. What defines the <u> best </u> state of the model?\n",
273+
"</div>"
274+
]
275+
},
276+
{
277+
"cell_type": "markdown",
278+
"metadata": {},
279+
"source": [
280+
"### Inference"
281+
]
282+
},
283+
{
284+
"cell_type": "markdown",
285+
"metadata": {},
286+
"source": [
287+
">We load the data we want to process within `input_val`. \n",
288+
"The parameter `n_tiles` can be used if images are to big for the GPU memory. "
289+
]
290+
},
291+
{
292+
"cell_type": "code",
293+
"execution_count": null,
294+
"metadata": {},
295+
"outputs": [],
296+
"source": [
297+
"input_val = imread('data/validation.tif')\n",
298+
"pred_val = model.predict(input_val, axes='YX')"
299+
]
300+
},
301+
{
302+
"cell_type": "markdown",
303+
"metadata": {},
304+
"source": [
305+
">Let's see results on the validation data. (First we look at the complete image and then we look at a zoomed-in view of the image)"
306+
]
307+
},
308+
{
309+
"cell_type": "code",
310+
"execution_count": null,
311+
"metadata": {},
312+
"outputs": [],
313+
"source": [
314+
"plt.figure(figsize=(16,8))\n",
315+
"plt.subplot(1,2,1)\n",
316+
"plt.imshow(input_val,cmap=\"magma\")\n",
317+
"plt.title('Input');\n",
318+
"plt.subplot(1,2,2)\n",
319+
"plt.imshow(pred_val,cmap=\"magma\")\n",
320+
"plt.title('Prediction');"
321+
]
322+
},
323+
{
324+
"cell_type": "code",
325+
"execution_count": null,
326+
"metadata": {},
327+
"outputs": [],
328+
"source": [
329+
"plt.figure(figsize=(16,8))\n",
330+
"plt.subplot(1,2,1)\n",
331+
"plt.imshow(input_val[200:300, 200:300],cmap=\"magma\")\n",
332+
"plt.title('Input - Zoomed in');\n",
333+
"plt.subplot(1,2,2)\n",
334+
"plt.imshow(pred_val[200:300, 200:300],cmap=\"magma\")\n",
335+
"plt.title('Prediction - Zoomed in');"
336+
]
337+
},
338+
{
339+
"cell_type": "markdown",
340+
"metadata": {},
341+
"source": [
342+
"<div class=\"alert alert-block alert-info\"> Q: <b> We demonstrated microscopy image denoising with this N2V notebook where we first trained a model and later applied the trained model on validation images. </b> <br>\n",
343+
" Would you call this a <i>supervised</i> learning approach, an <i>unsupervised</i> learning approach or something else? Discuss! :)"
344+
]
345+
},
346+
{
347+
"cell_type": "code",
348+
"execution_count": null,
349+
"metadata": {},
350+
"outputs": [],
351+
"source": []
352+
},
353+
{
354+
"cell_type": "code",
355+
"execution_count": null,
356+
"metadata": {},
357+
"outputs": [],
358+
"source": []
359+
}
360+
],
361+
"metadata": {
362+
"kernelspec": {
363+
"display_name": "Python 3 (ipykernel)",
364+
"language": "python",
365+
"name": "python3"
366+
},
367+
"language_info": {
368+
"codemirror_mode": {
369+
"name": "ipython",
370+
"version": 3
371+
},
372+
"file_extension": ".py",
373+
"mimetype": "text/x-python",
374+
"name": "python",
375+
"nbconvert_exporter": "python",
376+
"pygments_lexer": "ipython3",
377+
"version": "3.7.13"
378+
},
379+
"toc": {
380+
"base_numbering": 1,
381+
"nav_menu": {},
382+
"number_sections": true,
383+
"sideBar": false,
384+
"skip_h1_title": false,
385+
"title_cell": "Table of Contents",
386+
"title_sidebar": "Contents",
387+
"toc_cell": false,
388+
"toc_position": {},
389+
"toc_section_display": true,
390+
"toc_window_display": false
391+
}
392+
},
393+
"nbformat": 4,
394+
"nbformat_minor": 4
395+
}

0 commit comments

Comments
 (0)