For example, every image in the Frey face database is a two dimensional matrix, and all those pictures together can form a large three dimensional matrix. We will do 3D PCA on this matrix. This has the potential to preserve the spatial relation of the pixels in images.
Emphasize emphasize
Strong Strong
This blog gives a very thorough introduction to general issues.
From Tensorflow tutorial:
To create this model, we’re going to need to create a lot of weights and biases. One should generally initialize weights with a small amount of noise for symmetry breaking, and to prevent 0 gradients. Since we’re using
To create this model, we’re going to need to create a lot of weights and biases. One should generally initialize weights with a small amount of noise for symmetry breaking, and to prevent 0 gradients. Since we’re using ReLU neurons, it is also good practice to initialize them with a slightly positive initial bias to avoid “dead neurons”.
How to choose a neural network’s hyper-parameters?
Moving average from Tensorflow tutorial:
When training a model, it is often beneficial to maintain moving averages of the trained parameters. Evaluations that use averaged parameters sometimes produce significantly better results than the final trained values.
batch normalization, for gradient vanishing
stride convolution replace pooling, for fast computation
This page gives a very good example on how to implement stride convolution.
drop out
max out
An autoencoder has a lot of freedom and that usually means our AE can overfit the data because it has just too many ways to represent it. To constrain this we should use sparse autoencoders where a non-sparsity penalty is added to the cost function. In general when we talk about autoencoders we are really talking about sparse autoencoders.
Confusion matrix
Area Under Curve
and others
Order of singularity
polar transformation
rearrange integration points
analytical/numerical solution of singular terms
KNN
Kmeans
Bayes Net
Depth First Search
Breath First Search
Fast Fourier Transformation
Singular Value Decomposition
The most widely used optimization method in machine learning field is undoubtedly the Stochastic Gradient Descent (SGD) method. It is extremely straight forward and easy to implement in optimize neural networks. However, how to determine parameters of SGD can have a large impact on its convergence rate. In order to make SGD more robust and efficient, researchers have tried various ways such as momentum, weight decay and others.
This blog is to record my experience in using machine learning packages. A thorough comparison of those major machine learning packages could be found here.
With the development of computer hardware, it is more and more important to take hardware into consideration when trying to write super efficient code. The speed of a program is not only determined by the code, but also how well hardware supports. This blog is aimed at how to make the best use of your computer hardware. Continue reading “Accelerating computation – hardware aspect”
Saliency originally come from image analysis.
Based on similar idea, it could be expanded to 3D geometries as well.
Human computation based on shelling point.
Shape Analysis With Crowdsourced Data
We made this process into a game, hoping to extract human computation in this way.
We are thinking of incorporating this idea to boost CNN with attention.
Welcome 