TensorFlow – failed call to cuInit: CUDA_ERROR_UNKNOWN

Scenario: You’re trying to get your GPU to work in TensorFlow on a Ubuntu Laptop. You’ve already installed Tensorflow, Cuda, and Nvidia drivers.

You run python and import TensorFlow:

import tensorflow as tf

And you see encouraging messages like: "successfully opened CUDA library libcublas.so locally"

But in Python, when you run,

tf.Session()

You get this cryptic error:

failed call to cuInit: CUDA_ERROR_UNKNOWN

Here’s how to fix this.
Continue reading “TensorFlow – failed call to cuInit: CUDA_ERROR_UNKNOWN”

caffe – Check failed: proto.SerializeToOstream(&output)

You suddenly get this error when training/saving a model in Caffe or saving a model in pycaffe.

io.cpp:69] Check failed: proto.SerializeToOstream(&output)
*** Check failure stack trace: ***

Here are two possible reasons for this error

  1. The directory the snapshot is trying to write the .caffemodel into does not exist
  2. You are out of disk space

Continue reading “caffe – Check failed: proto.SerializeToOstream(&output)”

multi-resolution-tract CNN with hybrid pretrained and skin-lesion trained layers

Our paper entitled: “Multi-resolution-Tract CNN with Hybrid Pretrained and Skin-Lesion Trained Layers” was accepted and presented as an oral talk in the Machine Learning in Medical Imaging (MLMI) Workshop (part of the MICCAI conference).

In this work, we used a convolutional neural network (CNN) to classify 10 different types of skin lesions, including melanoma and non-melanoma types.

The key technical contribution was to use multiple tracts (or paths) within the neural network, to train (and test) the network on an image using multiple image resolutions simultaneously. Additionally, we extended a CNN pretrained on a single image resolution to work for multiple image resolutions.

Here are our slides presented at MLMI (thanks Aïcha!) showing our deep learning approach to classify skin disease:
Continue reading “multi-resolution-tract CNN with hybrid pretrained and skin-lesion trained layers”

(deep convolutional) generative adversarial nets – slides

There’s this really neat new idea on how to train neural networks that recently came out know as generative adversarial nets (GAN).

The basic idea of a GAN is to train two networks to compete with each other (hence the name “adversarial“). One network (called the generator) creates images that look just like real images. The other network (called the discriminator) distinguishes between real images and those images the generator produced.

Thus the two networks compete with each other, where the generator generates images to fool the discriminator, and the discriminator discriminates between the generator’s images and real images.

Continue reading “(deep convolutional) generative adversarial nets – slides”

Deep Dreams and a Neural Algorithm of Artistic Style – slides and explanations

Perhaps you saw an earlier post I wrote about deep dreaming Prague pictures, and you said to your self, “self, I wish I knew more about the techniques to make those crazy looking pictures.”

Well you are in luck since I’ve now posted the slides where I attempted to explain these two works to our reading group: 1) Google’s DeepDream, and 2) A Neural Algorithm of Artistic Style.

Continue reading “Deep Dreams and a Neural Algorithm of Artistic Style – slides and explanations”

What is the derivative of ReLU?

Prague travel pictures and deep dreaming


Here is a combined short summary on my travels to the city of Prague in the Czech Republic along with corresponding images created using Google’s DeepDreams.

What is this DeepDreams you speak of?

Basically, DeepDream is a deep neural network that was trained to recognize objects from millions of images. A deep neural network is composed of a stack of layers. Basically, these layers learn image filters that when applied to an image classify the image (e.g., is this an image of a cat or a dog?).

You give DeepDream an image and specify a layer in the neural network. The original image is then slightly perturbed to create a modified image that causes the specified layer in the neural network to be more activated.

Early layers in the neural network are sensitive to low level concepts like the edges and textures in the image. So if you specify an early layer, your image will be modified to have edges and textures that most activate the early selected layer.

Example CNN. Image goes as input. Early layers (purple) are sensitive to things like edges and textures. Later layers (red) are sensitive to higher level concepts like faces.
Example CNN. Image goes as input. Early layers (purple) are sensitive to things like edges and textures. Later layers (red) are sensitive to higher level concepts like faces.

Later (or deeper) layers in the neural network are activated when they see higher level concepts such as faces. So any areas in the original image that slightly look like a face, will be modified to look more like a face.

Okay, but now you might ask, but what about Prague? How was your trip? Did you like the city?

Yeah it was nice! Thanks for asking. Did you want to see some pictures? Here’s one of an old building.

Original image of some building in Prague... can't remember what/if the significance of this picture was.
Original image of some building in Prague… can’t remember what the significance of this picture was.

Let’s try some deep dreaming on this. We’ll use the neural network known as VGG16 (it’s a famous neural network that performs very well on competitions). We’ll start by telling VGG16 (the neural network) to modify this image so that one of it’s middle layers becomes more activated. Specifically, we will activate layer conv3_1 from VGG16 (if you don’t know what conv3_1 means, that’s okay – it’s just a technical detail specifying what layer to use). This gives us this:

Same Prague building but using DeepDreams with VGG16 conv3_1
Same Prague building but using DeepDreams with VGG16 conv3_1

Now if we activate a deeper layer, conv5_2, we get this crazy looking image,

Continue reading “Prague travel pictures and deep dreaming”

Deep features to classify skin lesions – summary and slides

Here I'm nervously just starting our talk on our approach to skin lesion classification
Me nervously just starting to talk about our approach to skin lesion classification.

We presented our work, “Deep Features to Classify Skin Lesions” at ISBI 2016 in Prague! And I’m happy to report that our work was awarded runner-up for the Best Student Paper Award 🙂

In this work, we looked at how to classify skin lesions from images captured with a digital camera (i.e., non-dermoscopy). Our approach was able to distinguish among 10 different types of skin diseases over 1300 images and achieved an accuracy higher than what was previously reported over the same dataset. We did this by applying deep learning (i.e., pretrained convolutional neural networks) to melanoma and non-melanoma skin images.

Continue reading “Deep features to classify skin lesions – summary and slides”

Deep Visual-Semantic Alignments for Generating Image Descriptions – [slides]

Here are some slides I put together to try and explain/present this great paper:

Karpathy, A., & Fei-Fei, L. (2015). Deep Visual-Semantic Alignments for Generating Image Descriptions. CVPR.

Let’s summarize it in two lines:
The authors proposed a way to combine information from an image and a corresponding text caption. They use a Recurrent Neural Network (RNN) to then generate text captions that describe the image.

Pretty interesting stuff. This is the first paper where I really took a close look at RNNs as well.

Continue reading “Deep Visual-Semantic Alignments for Generating Image Descriptions – [slides]”

Deep Neural Networks to Segment Neuronal Membranes in Electron Microscopy Images and Detect Mitosis in Breast Cancer Histology Images

Here are some slides I made to present the following two papers to our reading group:

Cireșan, D. C., Giusti, A., Gambardella, L. M., & Schmidhuber, J. (2012). Deep neural networks segment neuronal membranes in electron microscopy images. In F. Pereira, C. J. C. Burges, L. Bottou, & K. Q. Weinberger (Eds.), NIPS (pp. 2843–2851). Curran Associates, Inc.

Cireșan, D. C., Giusti, A., Gambardella, L. M., & Schmidhuber, J. (2013). Mitosis detection in breast cancer histology images with deep neural networks. In K. Mori, I. Sakuma, Y. Sato, C. Barillot, & N. Navab (Eds.), MICCAI (Vol. 8150 LNCS, pp. 411–418). Springer Berlin Heidelberg.

These two papers, written by the same authors, used a very similar approach to win two medical image analysis competitions in well known conferences. They give a very nice overview of the approach they took to win these challenges.
Continue reading “Deep Neural Networks to Segment Neuronal Membranes in Electron Microscopy Images and Detect Mitosis in Breast Cancer Histology Images”