How to select a single GPU in Keras

Scenario: You have multiple GPUs on a single machine, but you want to use just one. By default, Keras allocates memory to all GPUs unless you specify otherwise. You use a Jupyter Notebook to run Keras with the Tensorflow backend.

Here’s how to use a single GPU in Keras with TensorFlow

Run this bit of code in a cell right at the start of your notebook (before importing tensorflow or keras).

Continue reading “How to select a single GPU in Keras”

Conditional Image Generation with PixelCNN Decoders – slides

Last updated on May 31st, 2017

Awhile ago I presented and attempted to explain this work to our reading group:

van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016). Conditional Image Generation with PixelCNN Decoders. In D. D. Lee, M. Sugiyama, U. V Luxburg, I. Guyon, & R. Garnett (Eds.), NIPS (pp. 4790–4798). Retrieved from http://arxiv.org/abs/1606.05328

And also dived a bit into their previous work,
van den Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016). Pixel Recurrent Neural Networks. Arxiv, 48. Retrieved from http://arxiv.org/abs/1601.06759

While I usually post slides to the web shortly after, this time I’ve been scared to do so. There are a few critical points from this paper that I still don’t understand. And while I told myself that I would spend some time to figure this out, it is now months later, and I’ve taken no action. So as now is always the time to continue on in spite of the fear, I’ll let you, dear Internet, have these slides in all there erroneous ways.

Continue reading “Conditional Image Generation with PixelCNN Decoders – slides”

Convolutional Neural Networks for Adjacency Matrices

Last updated on May 31st, 2017

We had our work, BrainNetCNN, published in NeuroImage awhile ago,

Kawahara, J., Brown, C. J., Miller, S. P., Booth, B. G., Chau, V., Grunau, R. E., Zwicker, J., G., Hamarneh, G. (2017). BrainNetCNN: Convolutional neural networks for brain networks; towards predicting neurodevelopment. NeuroImage, 146(Feb), 1038–1049. http://doi.org/10.1016/j.neuroimage.2016.09.046

and I’ve meant to do a blog writeup about this. We recently released our code for BrainNetCNN on GitHub (based on Caffe), which implements the proposed filters designed for adjacency matrices.

We called this library Ann4Brains. In hindsight, we could have called this something more general and cumbersome like Ann4AdjacencyMatrcies, but I still like the zombie feel that Ann4Brains has.

We designed BrainNetCNN specifically with brain connectome data in mind. Thus the tag line of,

“Convolutional Neural Networks for Brain Networks”

seemed appropriate. However, after receiving some emails about using BrainNetCNN for other types of (non-connectome) data, I’ll emphasize that this approach can be applied to any sort of adjacency matrix, and not just brain connectomes.

The core contribution of this work is the filters designed for adjacency matrices themselves. So we’ll go through each of them. But first, let’s make sure we are clear on what the brain connectome (or adjacency matrix) is.

Continue reading “Convolutional Neural Networks for Adjacency Matrices”

Mastering the Game of Go – slides [paper explained]

Last updated on May 31st, 2017

This week I presented to our weekly reading group, this work:

Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., … Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484–489.

To quickly summarize this work…

Basically, they create a policy network, which is a convolutional neural network, that predicts the next move a human player would do from a board state. They create a value network, also a convolutional neural network, that predicts the outcome (win or lose) of the game given the current board state.
Continue reading “Mastering the Game of Go – slides [paper explained]”

TensorFlow – failed call to cuInit: CUDA_ERROR_UNKNOWN

Last updated on December 15th, 2016

Scenario: You’re trying to get your GPU to work in TensorFlow on a Ubuntu Laptop. You’ve already installed Tensorflow, Cuda, and Nvidia drivers.

You run python and import TensorFlow:

import tensorflow as tf

And you see encouraging messages like: "successfully opened CUDA library libcublas.so locally"

But in Python, when you run,

tf.Session()

You get this cryptic error:

failed call to cuInit: CUDA_ERROR_UNKNOWN

Here’s how to fix this.
Continue reading “TensorFlow – failed call to cuInit: CUDA_ERROR_UNKNOWN”

caffe – Check failed: proto.SerializeToOstream(&output)

You suddenly get this error when training/saving a model in Caffe or saving a model in pycaffe.

io.cpp:69] Check failed: proto.SerializeToOstream(&output)
*** Check failure stack trace: ***

Here are two possible reasons for this error

  1. The directory the snapshot is trying to write the .caffemodel into does not exist
  2. You are out of disk space

Continue reading “caffe – Check failed: proto.SerializeToOstream(&output)”

multi-resolution-tract CNN with hybrid pretrained and skin-lesion trained layers

Last updated on January 30th, 2017

Our paper entitled: “Multi-resolution-Tract CNN with Hybrid Pretrained and Skin-Lesion Trained Layers” was accepted and presented as an oral talk in the Machine Learning in Medical Imaging (MLMI) Workshop (part of the MICCAI conference).

In this work, we used a convolutional neural network (CNN) to classify 10 different types of skin lesions, including melanoma and non-melanoma types.

The key technical contribution was to use multiple tracts (or paths) within the neural network, to train (and test) the network on an image using multiple image resolutions simultaneously. Additionally, we extended a CNN pretrained on a single image resolution to work for multiple image resolutions.

Here are our slides presented at MLMI (thanks Aïcha!) showing our deep learning approach to classify skin disease:
Continue reading “multi-resolution-tract CNN with hybrid pretrained and skin-lesion trained layers”

(deep convolutional) generative adversarial nets – slides

Last updated on January 19th, 2017

There’s this really neat new idea on how to train neural networks that recently came out know as generative adversarial nets (GAN).

The basic idea of a GAN is to train two networks to compete with each other (hence the name “adversarial“). One network (called the generator) creates images that look just like real images. The other network (called the discriminator) distinguishes between real images and those images the generator produced.

Thus the two networks compete with each other, where the generator generates images to fool the discriminator, and the discriminator discriminates between the generator’s images and real images.

Continue reading “(deep convolutional) generative adversarial nets – slides”

Deep Dreams and a Neural Algorithm of Artistic Style – slides and explanations

Perhaps you saw an earlier post I wrote about deep dreaming Prague pictures, and you said to your self, “self, I wish I knew more about the techniques to make those crazy looking pictures.”

Well you are in luck since I’ve now posted the slides where I attempted to explain these two works to our reading group: 1) Google’s DeepDream, and 2) A Neural Algorithm of Artistic Style.

Continue reading “Deep Dreams and a Neural Algorithm of Artistic Style – slides and explanations”

What is the derivative of ReLU?

Last updated on September 19th, 2017