Category: caffe

Convolutional Neural Networks for Adjacency Matrices

We had our work, BrainNetCNN, published in NeuroImage awhile ago,

Kawahara, J., Brown, C. J., Miller, S. P., Booth, B. G., Chau, V., Grunau, R. E., Zwicker, J., G., Hamarneh, G. (2017). BrainNetCNN: Convolutional neural networks for brain networks; towards predicting neurodevelopment. NeuroImage, 146(Feb), 1038–1049. http://doi.org/10.1016/j.neuroimage.2016.09.046

and I’ve meant to do a blog writeup about this. We recently released our code for BrainNetCNN on GitHub (based on Caffe), which implements the proposed filters designed for adjacency matrices.

We called this library Ann4Brains. In hindsight, we could have called this something more general and cumbersome like Ann4AdjacencyMatrcies, but I still like the zombie feel that Ann4Brains has.

We designed BrainNetCNN specifically with brain connectome data in mind. Thus the tag line of,

“Convolutional Neural Networks for Brain Networks”

seemed appropriate. However, after receiving some emails about using BrainNetCNN for other types of (non-connectome) data, I’ll emphasize that this approach can be applied to any sort of adjacency matrix, and not just brain connectomes.

The core contribution of this work is the filters designed for adjacency matrices themselves. So we’ll go through each of them. But first, let’s make sure we are clear on what the brain connectome (or adjacency matrix) is.

Continue reading “Convolutional Neural Networks for Adjacency Matrices”

caffe – Check failed: proto.SerializeToOstream(&output)

You suddenly get this error when training/saving a model in Caffe or saving a model in pycaffe.

io.cpp:69] Check failed: proto.SerializeToOstream(&output)
*** Check failure stack trace: ***

Here are two possible reasons for this error

  1. The directory the snapshot is trying to write the .caffemodel into does not exist
  2. You are out of disk space

Continue reading “caffe – Check failed: proto.SerializeToOstream(&output)”

Prague travel pictures and deep dreaming


Here is a combined short summary on my travels to the city of Prague in the Czech Republic along with corresponding images created using Google’s DeepDreams.

What is this DeepDreams you speak of?

Basically, DeepDream is a deep neural network that was trained to recognize objects from millions of images. A deep neural network is composed of a stack of layers. Basically, these layers learn image filters that when applied to an image classify the image (e.g., is this an image of a cat or a dog?).

You give DeepDream an image and specify a layer in the neural network. The original image is then slightly perturbed to create a modified image that causes the specified layer in the neural network to be more activated.

Early layers in the neural network are sensitive to low level concepts like the edges and textures in the image. So if you specify an early layer, your image will be modified to have edges and textures that most activate the early selected layer.

Example CNN. Image goes as input. Early layers (purple) are sensitive to things like edges and textures. Later layers (red) are sensitive to higher level concepts like faces.
Example CNN. Image goes as input. Early layers (purple) are sensitive to things like edges and textures. Later layers (red) are sensitive to higher level concepts like faces.

Later (or deeper) layers in the neural network are activated when they see higher level concepts such as faces. So any areas in the original image that slightly look like a face, will be modified to look more like a face.

Okay, but now you might ask, but what about Prague? How was your trip? Did you like the city?

Yeah it was nice! Thanks for asking. Did you want to see some pictures? Here’s one of an old building.

Original image of some building in Prague... can't remember what/if the significance of this picture was.
Original image of some building in Prague… can’t remember what the significance of this picture was.

Let’s try some deep dreaming on this. We’ll use the neural network known as VGG16 (it’s a famous neural network that performs very well on competitions). We’ll start by telling VGG16 (the neural network) to modify this image so that one of it’s middle layers becomes more activated. Specifically, we will activate layer conv3_1 from VGG16 (if you don’t know what conv3_1 means, that’s okay – it’s just a technical detail specifying what layer to use). This gives us this:

Same Prague building but using DeepDreams with VGG16 conv3_1
Same Prague building but using DeepDreams with VGG16 conv3_1

Now if we activate a deeper layer, conv5_2, we get this crazy looking image,

Continue reading “Prague travel pictures and deep dreaming”

Deep features to classify skin lesions – summary and slides

Here I'm nervously just starting our talk on our approach to skin lesion classification
Me nervously just starting to talk about our approach to skin lesion classification.

We presented our work, “Deep Features to Classify Skin Lesions” at ISBI 2016 in Prague! And I’m happy to report that our work was awarded runner-up for the Best Student Paper Award 🙂

In this work, we looked at how to classify skin lesions from images captured with a digital camera (i.e., non-dermoscopy). Our approach was able to distinguish among 10 different types of skin diseases over 1300 images and achieved an accuracy higher than what was previously reported over the same dataset. We did this by applying deep learning (i.e., pretrained convolutional neural networks) to melanoma and non-melanoma skin images.

Continue reading “Deep features to classify skin lesions – summary and slides”

CAFFE – how to specify which GPU to use in PyCaffe

You are using PyCaffe (Python interface for Caffe) and training a deep neural network directly within Python (although I think the same command holds for MATLAB).

You are on a machine with 2 GPUs and you want to specify which GPU to use for training. This is useful so you can train two different models at the same time on each GPU. Note that here we refer to training two different models on two different GPUs on the same machine, not a single model on two GPUs.

(side note: it seems to me that running two different jobs on the same GPU drastically slows GPU training. It’s so much slower that I only train a single model on a single GPU at a time. Running two different jobs on two different GPUs seems to be okay though)
Continue reading “CAFFE – how to specify which GPU to use in PyCaffe”

How to compile locally on linux without sudo permissions

Here’s an example of how to compile things locally on linux without sudo permissions.

Why in the world would you want to do this? Well you might be on locked down system (say you are at school) and the back and forth between the sys admins may not be effective.

For example, I have been trying to get Caffe to work with Python on my locked down machine at school – and while the sys admins were helpful enough to install Caffe, I was getting strange errors using different versions of Caffe with the different versions of python installed. Here’s how I solved one problem, which I hope is a useful template and reminder for future issues.
Continue reading “How to compile locally on linux without sudo permissions”