Cluster commands

Often I find myself running jobs on the cluster and I can never remember the basic commands. So here’s some useful commands that you can use to run jobs on the cluster.

*** note that I’m running jobs on Simon Fraser University’s (SFU) cluster and I have no idea if these will work on your specific configuration ***

Assume your userid is billybob
Assume you have a cluster script called test.pbs
Continue reading “Cluster commands”

Unity3D – example online game

A few months ago, I romanticized about developing video games and spent an evening trying out Unity3D.

I have to say I’m pretty impressed. With zero prior experience using it and barely going through any sort of tutorials, in a couple of hours I managed to put together a little world to run around in!

Jer's World
An actual screenshot of the 3D world

Continue reading “Unity3D – example online game”

Spinal Cord Segmentation using an A* search

Our paper was accepted in ISBI 2013! 🙂

The paper is called:

Globally Optimal Spinal Cord Segmentation Using a Minimal Path in High Dimensions

I think the title really summarizes the work pretty nicely so I’ll break down each part of the title to give you a brief overview of the paper.

“Spinal Cord Segmentation” = this is the goal of the work. Given a 3D MRI, we want a method that can mark those voxels that belong to the spinal cord. Segmentation means to label the voxels as spinal cord or background.

“High Dimensions” = we represented the segmented spinal cord by a list of 6 numbers. I’ll explain this a bit more since this a bit tricky to understand. We used principle component analysis (PCA) to represent the “shape” of a 2D slice of the spinal cord. PCA allows us to represent each shape by its center point (x,y,z) and three principle components (giving us a total of 6 numbers per 2D slice). We can represent our spinal cord as a stack of these 2D shapes, or equivalently, a list of 6 numbers.

Now the question is, how to find this list of 6 numbers? Well we can search for them. This is similar to finding the shortest path in between 2 points, but instead of searching over 2 or 3 (spatial) dimensions, we search over the 6 dimensions (3 spatial and 3 shape).

“Minimal Path” = a minimal path finds the shortest path in between two points. In our methods, a user enters the start and end point of the spinal cord, and we find the minimal path in 6D between these two points. We use a slightly modified version of the A* search to find this minimal path. A path is defined to be “short” if the shapes “fits” well (e.g. the borders match) with what is in the MRI.

“Globally Optimal” = means that we can find the best segmentation that exists based on how we defined how well the shape “fits”. Our method will always find the global minimum and will not get stuck in any local minima.

You can take a look at the paper here.

Here’s the poster we did for the ISBI conference.

ISBI 2013 poster
Spinal Cord Segmentation Poster

The nice PDF can be viewed here.

MATLAB – Calculate L2 Euclidean distance

Here’s how to calculate the L2 Euclidean distance between points in MATLAB.

The whole kicker is you can simply use the built-in MATLAB function, pdist2(p1, p2, ‘euclidean’) and be done with it. p1 is a matrix of points and p2 is another matrix of points (or they can be a single point).

However, initially I wasn’t really clear about what was going on. So if you are still a bit confused, let’s chat about it…

The scenario: You have one point, aPoint, that you wish to compare against a bunch of other points bunchOfPoints. For simplicity, let’s work in two-dimensional (2D) space. Note that 2D means each point is composed of two pieces of information (i.e. has 2 components).

We start by defining our points, then we calculate the L2 distance by hand, then we use the built-in pdist2() function to show we get the same result. And then finally, as a little bonus, we show how to get the minimum L2 Euclidean distance at the end.

Here’s how to calculate the equation by hand if you’re interested.
http://en.wikipedia.org/wiki/Norm_%28mathematics%29#Euclidean_norm

% disL2
 
% Define our points.
aPoint = [1,4]; % A single point with 2 components.
bunchOfPoints = [2,3; 1,4; 0,1]; % A bunch of other points.
 
% Make 'aPoint' the same size as a 'bunchOfPoints'.
aPointMatrix = repmat(aPoint,size(bunchOfPoints,1),1);
 
% Calculate by hand.
%% L2 Euclidean Norm.
% http://en.wikipedia.org/wiki/Norm_%28mathematics%29#Euclidean_norm
% 1) Take the difference between the two -> aPointMatrix-bunchOfPoints
% 2) Square the difference to get rid of positive/negative: 
%       (aPointMatrix-bunchOfPoints).^2
% 3) Sum this up along the rows.
%       (sum(((aPointMatrix-bunchOfPoints).^2), 2))
% 4) Take the square root of this.
%       (sum(((aPointMatrix-bunchOfPoints).^2), 2)).^0.5
pointsDifSquare = (sum(((aPointMatrix-bunchOfPoints).^2), 2)).^0.5
 
% output = 1.4142 0 3.1623
 
%% Or we can just use this handy built-in function...
d = pdist2(aPoint,bunchOfPoints,'euclidean') 
 
% same output! = 1.4142 0 3.1623
 
%% Bonus, how to find the min distance!
[theMinDistance, indexOftheMinDistance] = min(d)
 
% Yeah! As expected theMinDistance = 0 and indexOftheMinDistance = 2.

LaTeX – Detexify ~ a shout out

[latexpage]
Here’s how to easily find symbols in LaTeX.

Okay the most handiness (is that really a word?) little web app that I’ve come across since starting work in $\LaTeX$ is this little tool found here.
http://detexify.kirelabs.org/classify.html

$\text{detextify}^2$ has seriously saved me hours of my life.

This beautiful app lets you draw any symbol into a box, then it will return the corresponding $\LaTeX$ command. So freakin’ handy! And it works. That obscure symbol you saw in some paper $\zeta$, and you can’t think of the Greek name of the top of your head? Simple! Just draw it in and bam, $\text{detextify}^2$ throws the command back (\zeta in case you are wondering).

I use this all the time when I’m writing anything sort of equations in LaTeX. A definite must know about.

Hope this saves you countless sleepless nights.

MATLAB – how to make a movie of plots

Making a video of your moving graphs/charts is surprisingly easy to do in MATLAB. However most of my online searches gave me old outdated methods to do it. Here’s how to make a movie or a video in MATLAB.

I kept getting this freakin’ error using the old methods (i.e. the avifile() function):
“Windows Media Player cannot play the file. The Player might not support the file type or might not support the codec that was used to compress the file.”

So I found MATLAB recommends you use the VideoWriter() class…
http://www.mathworks.com/help/techdoc/ref/videowriterclass.html

They have a nice little example in the documentation, but for the impatient, here’s my quick and dirty implementation of it (with some modifications/additions of course).
Continue reading “MATLAB – how to make a movie of plots”

LaTeX – bold vectors and arrow vectors

Lately I’m writing a lot of papers in \LaTeX and every once and a while something comes up that drives me crazy trying to figure out.

Here’s how to easily switch between a bold vector \boldsymbol{x}  and an arrow vector \vec{x}  .

 
% Minimal latex example:
% Shows how to switch between bold and arrow vectors.
 
% Specifies the type of document you have.
\documentclass{article}
 
% Used for the boldsymbol.
\usepackage{amsmath}
 
% Comment this out to represent vectors with an arrow on top.
% Uncomment this to represent vectors as bold symbols.
\renewcommand{\vec}[1]{\boldsymbol{#1}}
 
% Start of the document.
\begin{document}
 
% Your content.
My lovely vector: $\vec{x}$
 
% End of the document.
\end{document}

Continue reading “LaTeX – bold vectors and arrow vectors”

MATLAB – How to scale/normalize values in a matrix to be between 0 and 1

I hate that I have to keep looking this up…

Here’s how to scale or normalize your numbers in MATLAB so they lie between 0 and 1.
Change the number of mins and maxs depending on the dimensionality of your matrix.

I = [ 1 2 3; 4 5 6]; % Some n x m matrix I that contains unscaled values.
scaledI = (I-min(I(:))) ./ (max(I(:)-min(I(:))));
min(scaledI(:)) % the min is 0
max(scaledI(:)) % the max 1

All of scaledI values are now between 0 and 1.

SQL vs ANSI ~ Why I now avoid the old SQL syntax like it has a bad disease

Old SQL syntax vs. New ANSI syntax.

The short: to answer the question of whether you should use SQL vs ANSI, use ANSI.

Why?

Let me tell you a story…
A young man is frantically programming a website. This is his first ever real site he has worked on so the learning curve is fairly steep. The website is to be used as an interface to a database storing a couple hundred thousand records.

Being quite used to desktop programming, this junior programmer is impatient and is coding using the old trial-and-error approach.

One query he is working on involves just a few tables being joined together. Pretty simple stuff… It went something like,

SELECT *
FROM specimens SP, box B
WHERE SP.box_fk = B.box_id

Then in a frenzied dash of debugging, he realizes he doesn’t want these tables joined so he comments out the last line,

SELECT *
FROM specimens SP, box B
-- WHERE SP.box_fk = B.box_id

He then faithfully tests his changes on his test database, notices no strange behavior and moves it over to the production side.

In his usual frantic pace, he executes the above statement three or four times from the webpage with no response. So he closes it, puzzled for a second, and then brushing it from his mind, he begins working on another project.

About 3 and a half hours later, his supervisor comes running in asking why he’s getting a million phone calls and emails from IT saying his queries are shutting down the production databases.

Oops…

And ever since this unfortunate incident, this young programmer used the ANSI syntax religiously.

So what happened?
1) A cross join.
Commenting out the last line resulted in each record in each table joining with each other to produce a enormous amount of records. Clearly not what he wanted.
2) The test database contained a small amount of records in it. Small enough in fact that the rows produced by cross join could be successfully returned. However the production database contained hundreds of thousands of records; this cross join probably was trying to pull millions of rows back.

By the way the new ANSI syntax (the good and holy way) of the same query would look like this,

SELECT *
FROM specimen SP LEFT OUTER JOIN box B ON SP.box_id = B.box_fk

Two things I learned.
1) Make sure to have a decent amount of test data in your development database.
2) Use the new ANSI syntax.

MATLAB – How to check if a file or a folder exists

So you want to check if a file or a folder exists in MATLAB? Here’s how to do it.

% The file/folder to find.
fPath = 'aLittleFile.m';
 
% To see what each of these "magic numbers" mean, go to, 
% http://www.mathworks.com/help/techdoc/ref/exist.html
if isequal(exist(fPath,'file'),2) % 2 means it's a file.
    % We have a file!
    display('a file!');
elseif isequal(exist(fPath, 'dir'),7) % 7 = directory.
    % We have a folder!
    display('a folder');
else
    % We have an invalid file or folder.
    display('an error!');
end

Note that we use the isequal bit to check if it is actually a file or a folder. If we take the isequal check out, then the first if statement would be true for a file and a folder (since exist(fPath,’file’) would return 7, thus the if statement would be true)!