Google Chrome Bug?

chrome-bug_thumb-5B2-5D

It seems there is a minor bug in Google Chrome. Apparently, you can’t click links that are in the same horizontal line as the status bar that pops up from the bottom left corner.

For example, clicking the link marked with a red circle in the image below does nothing.

chrome-bug

Well, this obviously isn’t a show stopper, but I think its a bug nevertheless.

Logistic Regression in C#

Logistic regression (sometimes called the logistic model or logit model) is used for prediction of the probability of occurrence of an event by fitting data to a logistic curve. Logistic regression is used extensively in the medical and social sciences as well as marketing applications such as prediction of a customer’s propensity to purchase a product or cease a subscription.

The code presented here is part of the Accord.NET Framework. The Accord.NET Framework is a framework for developing machine learning, computer vision, computer audition, statistics and math applications in .NET. To use the framework in your projects, install it by typing Install-Package Accord.MachineLearning in your IDE’s NuGet package manager.

Overview

The Logistic regression is a generalized linear model used for binomial regression. Like many forms of regression analysis, it makes use of several predictor variables that may be either numerical or categorical.

Logistic Sigmoid Function

The logistic sigmoid function is given by

g(z) = 1 / (1 + Exp(-z))

where in the context of logistical regression z is called the logit.

Logistic Regression Model

The logistic regression model is a generalized linear model. This means that it is just a linear regression model taken as input for a non-linear link function. The linear model has the form

z = c1x1 + c2x2 + … cnxn + i = ct x + i

where c is the coefficient vector, i is the intercept value and x is the observation vector for n variables and in the context of logistic regression is called the logit. The logit is then applied as input for the nonlinear logistic sigmoid function g(z), giving as result a probability.

So in a binomial problem where we are trying to determine whether a observation belongs to class C1 or class C2, the logistic model tell us that:

p(C1|x) = g(ct x + i)

p(C2|x) = 1 – p(C1|x)

where p(C1|x) denotes the probability of C1 being true when x is true.
In other words, denotes the probability of x belonging to class C1.

Coefficients

The coefficients for the logistic regression are the values which multiply each observation variable from a sample input vector. The intercept is analogous to the independent term in a polynomial linear regression model or the threshold or bias value for a neuron in a neural network model.

Odds Ratio

After computation of the logistic regression model, every coefficient will have an associated measure called the odds ratio. The odds ratio is a measure of effect size, describing the strength of association or non-independence between two binary data values and can be approximated by raising the coefficient value to the euler’s number.

Odds ratioc = ec

Standard Error

The standard error for the coefficients can be retrieved from the inverse Hessian matrix calculated during the model fitting phase and can be used to give confidence intervals for the odds ratio. The standard error for the i-th coefficient of the regression can be obtained as:

SEi = sqrt(diag(H-1)i)

Confidence Intervals

The confidence interval around the logistic regression coefficient is plus or minus 1.96*SEi, where SEi is the standard error for coefficient i. We can then define:

95% C.I.i = <lower, upper> = <coefficienti – 1.96 * SEi,  coefficienti + 1.96 * SEi>

Wald Statistic and Wald’s Test

The Wald statistic is the ratio of the logistic coefficient to its standard error. A Wald test is used to test the statistical significance of each coefficient in the model. A Wald test calculates a Z statistic, which is:

z = coefficient / standard error

This z value can be squared, yielding a Wald statistic with a chi-square distribution, or, alternatively, it can be taken as is and compared directly with a Normal distribution.

The Wald test outputs a p-value indicating the significance of individual independent variables. If the value is below a chosen significance threshold (typically 0.05), then the variable plays some role in determining the outcome variable that most likely is not result of chance alone. However, there are some problems with the use of the Wald statistic. The Likelihood-ratio test is a better alternative for the Wald test.

Likelihood-Ratio and Chi-Square Test

The likelihood-ratio is the ratio of the likelihood of the full model over the likelihood of a simpler nested model. When compared to the null model the likelihood-ratio test gives an overall model performance measure. When compared to nested models, each with one variable omitted, it tests the statistical significance of each coefficient in the model. These significance tests are considered to be more reliable than the Wald significance test.

The likelihood-ratio is a chi-square statistic given by:

D & = -2lnleft( frac{text{likelihood for first model}}{text{likelihood for second model}} right).

The model with more parameters will always fit at least as well (have a greater log-likelihood). Whether it fits significantly better and should thus be preferred can be determined by deriving the probability or p-value of the obtained difference D. In many cases, the probability distribution of the test statistic can be approximated by a chi-square distribution with (df1 − df2) degrees of freedom, where df1 and df2 are the degrees of freedom of models 1 and 2 respectively.

Regression to a Logistic Model

If we consider the mapping

φ(<x1, x2, …, xn>) = <x1, x2, … xn, 1>

The logistic regression model can also be rewritten as

p(C1|φ) = g(wt φ) = f(φ, w)

so that w contains all coefficients and the intercept value in a single weight vector. Doing so will allow the logistic regression model to be expressed as a common optimization model in the form f(φ, w) allowing many standard non-linear optimization algorithms to be directly applied in the search for the best parameters w that best fits the probability of a class C1 given a input vector φ.

Likelihood function

The likelihood function for the logistic model is given by:

p(t|w) = prod_{n=1}^N y_n^{t_n} { 1 - y_n }^{1-t_n}

but, as the log of products equals the sum of logs, taking the log of the likelihood function results in the Log-likelihood function in the form:

ln p(t|w) = sum_{n=1}^N { t_n ln y_n + (1-t_n) ln (1 - y_n) }

Furthermore, if we take the negative of the log-likelihood, we will have a error function called the cross-entropy error function:

E(w) = -ln p(t|w) = - sum_{n=1}^N { t_n ln y_n + (1-t_n) ln (1 - y_n) }

which gives both better numerical accuracy and enable us to write the error gradient in the same form as the gradient of the sum-of-squares error function for linear regression models (Bishop, 2006).

Another important detail is that the likelihood surface is convex, so it has no local maxima. Any local maxima will also be a global maxima, so one does not need to worry about getting trapped in a valley when walking down the error function.

Iterative Reweighted Least-Squares

The method of Iterative Reweighted Least-Squares is commonly used to find the maximum likelihood estimates of a generalized linear model. In most common applications, an iterative Newton-Raphson algorithm is used to calculate those maximum likelihood values for the parameters. This algorithm uses second order information, represented in the form of a Hessian matrix, to guide incremental coefficient changes. This is also the algorithm used in this implementation.

In the Accord.NET machine learning framework. the Iterative Reweighted Least-Squares is implemented in the IterativeReweightedLeastSquares class (source).

Source Code

Below is the main code segment for the logistic regression, performing one pass of the Iterative Reweighted Least-Squares algorithm.

Furthermore, as the likelihood function is convex, the Logistic Regression Analysis can perform regression without having to experiment different starting points. Below is the code to compute a logistic regression analysis. The algorithm iterates until the largest absolute parameter change between two iterations becomes less than a given limit, or the maximum number of iterations is reached.

Using the code

Code usage is very simple. To perform a Logistic Regression Analysis, simply create an object of the type LogisticRegressionAnalysis and pass the input and output data as constructor parameters. Optionally you can pass the variables names as well.

Then just call Compute() to compute the analysis.

After that, you can display information about the regression coefficients by binding the CoefficientCollection to a DataGridView.

Sample application

The sample application creates Logistic Regression Analyses by reading Excel workbooks. It can also perform standard Linear Regressions, although there aren’t many options available to specify linear models.

lr-1 lr-6

Example data set adapted from http://www.statsdirect.co.uk/help/regression_and_correlation/logi.htm.

 

lr-2 lr-3

The image on the left shows information about the analysis performed, such as coefficient values, standard errors, p-value for the Wald statistic, Odds ratio and 95% confidence intervals. It also reports the log-likelihood, deviance and likelihood-ratio chi-square test for the final model. The newest available version can also compute likelihood-ratio tests for each coefficient, although not shown in the image.

Final Considerations

The logistic regression model can be seen as the exact same as a one layer MLP with only one hidden neuron using a sigmoid activation function trained by a Newton’s method learning algorithm. Thus we can say the Logistic Regression is just a special case of Neural Networks. However, one possible (and maybe unique) advantage of logistic regression is that it allows simpler interpretation of its results in the form of odds ratios and statistical hypothesis testing. Statistical analysis using neural networks is also possible, but some may argue it is not as straightforward as using ordinary logistic regression.

Multilayer perceptron neural networks with sigmoid activation functions can be created using the Accord.Neuro package and namespace.

References

Kernel Discriminant Analysis in C#

kdaf16

Kernel (Fisher) discriminant analysis (kernel FDA) is a non-linear generalization of linear discriminant analysis (LDA) using techniques of kernel methods. Using a kernel, the originally linear operations of LDA are done in a reproducing kernel Hilbert space with a non-linear mapping.

This code has also been incorporated in Accord.NET Framework, which includes the latest version of this code plus many other statistics and machine learning tools.

Analysis Overview

KDA is an extension of LDA to non-linear distributions, just as KPCA is to PCA. For information about LDA, please check the previous post, Linear Discriminant Analysis in C#. The algorithm presented here is a multi-class generalization of the original algorithm by Mika et al. in Fisher discriminant analysis with kernels (1999).

The objective of KDA is to find a transformation maximizing the between-class variance and minimizing the within-class variance. It can be shown that, with Kernels, the original objective function can be expressed as:

 J(alpha) = frac{alpha^T S_B alpha}{alpha^T S_W alpha}

with:

S_B =  sum_{c=1}^C (mu_c - bar{x}) (mu_c - bar{x})^T S_W =  sum_{c=1}^C K_c (I - 1_{l_c}) K_c^T

where Kc is the Kernel matrix for class c, uc is the column means vector for Kc, I is the identity matrix, lc is the number of samples in class c and 1lc is a lc x lc matrix with all entries 1/lc.

The Kernel Trick

The Kernel trick transforms any algorithm that solely depends on the dot product between two vectors. Wherever a dot product is used, it is replaced with the kernel function.

K(x,y) = <φ(x),φ(y)>

Thus, a linear algorithm can easily be transformed into a non-linear algorithm. So the algorithm can be carried in a higher-dimension space without explicitly mapping the input points into this space. For more information and a cool video showing how the kernel trick works, please see the introductory text in the previous post, Kernel Principal Component Analysis in C#.

Adding regularization

As the problem proposed above is ill-posed (we are estimating l dimensional covariance structures from l samples), the Sw matrix may become singular, and computing the objective function may become problematic. To avoid singularities, we can add a multiple of the identity matrix to the Sw matrix:

S_W :=!, S_W + lambda I

The λ adds numerical stabilities as for large λ the matrix Sw will become positive definite. The λ term can also be seen as a regularization term, favoring solutions with small expansion coefficients [Mika et al, 1999].

 

Source Code

The code below implements a generalization of the original Kernel Discriminant Analysis algorithm by Mika et al. in Fisher discriminant analysis with kernels (1999).

After completing the analysis, we may wish to project new data into discriminant space. The following code projects a data matrix into this space using the basis found during the analysis.

Sample Application

The sample application shows the how Kernel Discriminant Analysis works. The application can load Excel worksheets containing tables in the same format as the included sample worksheet.

Kernel Discriminant Analysis (KDA) Kernel Discriminant Analysis (KDA) Kernel Discriminant Analysis (KDA) Kernel Discriminant Analysis (KDA)

The first image shows the Wikipedia example for Kernel Principal Component Analysis. The second image shows the Analysis carried out with a Gaussian kernel with sigma = 3.6. The third image shows the Kernel between class and within class equivalent matrices. The last image shows the subset spawned by each class and its Kernel scatter matrix.

Kernel Discriminant Analysis (KDA) Kernel Discriminant Analysis (KDA)

The picture on the left shows the projection of the original data set in the first two discriminant dimensions. Note that just one dimension would already be sufficient to linear separate the classes. On the right, the picture shows how a point in input space (given by the mouse cursor) gets mapped into feature space.

Linear Discriminant Analysis equivalent as a special case

Linear Discriminant Analysis as Kernel Discriminant Analysis spacial case Linear Discriminant Analysis as Kernel Discriminant Analysis spacial case

We can also check that a projection equivalent to Linear Discriminant Analysis can be obtained by using a Linear kernel in the Kernel Discriminant Analysis.

See also

References

Linear Discriminant Analysis in C#

cigars_thumb

Linear discriminant analysis (LDA) is a method used in statistics and machine learning to find a linear combination of features which best characterize or separates two or more classes of objects or events. The resulting combination may be used as a linear classifier, or, more commonly, for dimensionality reduction before later classification.

The code presented here is also part of the Accord.NET Framework. The Accord.NET Framework is a framework for developing machine learning, computer vision, computer audition, statistics and math applications. It is based on the already excellent AForge.NET Framework. Please see the starting guide for mode details. The latest version of the framework includes the latest version of this code plus many other statistics and machine learning tools.

Motivations

The goals of LDA are somewhat similar to those of PCA. But different from LDA, PCA is an unsupervised technique and as such does not include label information of the data, effectively ignoring this often useful information. For instance, consider two cigar-like clusters in 2 dimensions, one cigar having y = c and the other y = –c (with c being a arbitrary constant), as the image below suggests:

cigars

This example was adapted from the
note on Fisher Linear Discriminant Analysis by Max Welling.

 

The cigars are positioned in parallel and very closely together, such that the variance in the total dataset, ignoring the labels, is in the direction of the cigars. For classification, this would be a terrible projection, because all labels will get mixed and we will destroy any useful information:

cigars-pca-2 cigars-pca-1

 

A much more useful projection is orthogonal to the cigars, which would perfectly separate the data-cases:

cigars-lda-2 cigars-lda-1

 

The first row of images was obtained by performing PCA on the original data. The left image is the PCA projection using the first two components. However, as the analysis says the first component accounts for 96% of the information, one can infer all other components could be safely discarded. The result is the image on the right, clearly a not very useful projection.

The second row of images was obtained by performing LDA on the original data. The left image is the LDA projection using the first two dimensions. However, as the analysis says the first dimension accounts for 100% of the information, all other dimensions could be safely discarded. Well, this is actually true, as we can see on the result in the right image.

 

Analysis Overview

Linear Discriminant Analysis is closely related to principal component analysis (PCA) and factor analysis in that both look for linear combinations of variables which best explain the data. LDA explicitly attempts to model the difference between the classes of data. PCA on the other hand does not take into account any difference in class, and factor analysis builds the feature combinations based on differences rather than similarities. Linear Discriminant Analysis considers maximizing the following objective:

 J(w) = frac{w^T S_B w}{w^T S_W w}

where

S_B =  sum_{c=1}^C (mu_c - bar{x}) (mu_c - bar{x})^T S_W =  sum_{c=1}^C sum_{i in c} (x_i - mu_c) (x_i - mu_c)^T

are the Between-Class Scatter Matrix and Within-Class Scatter Matrix, respectively. The optimal solution can be found by computing the Eigenvalues of SBSW-1 and taking the Eigenvectors corresponding to the largest Eigen values to form a new basis for the data. Those can be easily obtained by computing the generalized eigenvalue decomposition of SB and SW.

 

Source Code

The source code below shows the main algorithm for Linear Discriminant Analysis.

 

 

Using The Code

Code usage is simple, as it follows the same object model in the previous PCA example.

 

Considerations

Besides being useful in many situations, in many practical cases linear discriminants are not suitable. A nice insight is that LDA can be extended for use in non-linear classification via the kernel trick. Using kernels, the original observations can be effectively mapped into a higher dimensional non-linear space. Linear classification in this non-linear space is then equivalent to non-linear classification in the original space.

While the code available here already contains code for Kernel Discriminant Analysis, this is something I’ll address in the next post. If you have any suggestions or questions, please leave me a comment.

Finally, as usual, I hope someone finds this information useful.

 

References:

Kernel Principal Component Analysis in C#

kpca-1_thumb-5B1-5D

Kernel principal component analysis (kernel PCA) is an extension of principal component analysis (PCA) using techniques of kernel methods. Using a kernel, the originally linear operations of PCA are done in a reproducing kernel Hilbert space with a non-linear mapping.

The code presented here is also part of the Accord.NET Framework. The Accord.NET Framework is a framework for developing machine learning, computer vision, computer audition, statistics and math applications. It is based on the already excellent AForge.NET Framework. Please see the starting guide for mode details. The latest version of the framework includes the latest version of this code plus many other statistics and machine learning tools.

Analysis Overview

KPCA is an extension of PCA to non-linear distributions. Instead of directly doing a PCA, the original data points are mapped into a higher-dimensional (possibly infinite-dimensional) feature space defined by a (usually nonlinear) function φ through a mathematical process called the “kernel trick”.

The Kernel trick

The kernel trick is a very interesting technique, able to transform any algorithm that solely depends on the dot product between two vectors. To apply the trick, we replace all dot products in an algorithm with a kernel function. Thus, a linear algorithm can easily be transformed into a non-linear algorithm. This non-linear algorithm is equivalent to the linear algorithm operating in the range space of φ. However, because kernels are used, the φ function is never explicitly computed. This is desirable [8], as the high-dimensional space can have an arbitrary large dimensionality, or can be even infinite-dimensional (such as in the case the chosen kernel function is a Gaussian) and thus infeasible to compute.

Interesting video by Udi Aharoni demonstrating how points that are not linearly separable in a 2D space can almost always be linearly separated in a higher dimensional (3D) space.

Principal Components in Kernel Space

Like in PCA, the overall idea is to perform a transformation that will maximize the variance of the captured variables while minimizing the overall covariance between those variables. Using the kernel trick, the covariance matrix is substituted by the Kernel matrix and the analysis is carried analogously in feature space. An Eigen value decomposition is performed and the eigenvectors are sorted in ascending order of Eigen values, so those vectors may form a basis in feature space that explain most of the variance in the data on its first dimensions.

However, because the principal components are in feature space, we will not be directly performing dimensionality reduction. Suppose that the number of observations m exceeds the input dimensionality n. In linear PCA, we can find at most n nonzero Eigen values. On the other hand, using Kernel PCA we can find up to m nonzero Eigen values because we will be operating on a m x m kernel matrix[2].

A more detailed discussion can be seen in the works of Mika et al. (Kernel PCA and De-Noising in Feature Spaces) and Scholkopf et al. (Nonlinear Component Analysis as a Kernel Eigenvalue Problem). A more accessible explanation of the topic is also available in the work from Ian Fasel.

The pre-image problem

In standard PCA it was possible to revert the transformation, retrieving the original data points and enabling the use of PCA for compression* and denoising. However, as the results provided by kernel PCA live in some high dimensional feature space,  they need not have pre-images in input space. In fact, even if it exists, it also need be not unique.

Thus, in KPCA, it is only possible to obtain an approximate pre-image for a given set of patterns projected on the feature space. This problem can (and has) been tackled by many approaches, some of them of iterative nature, but some closed-form solutions also exists. Typically those solutions make use of the fact that distances in feature space are related to distances in input space. Thus, those solutions try to achieve an optimal transformation that can embed those feature points in input space respecting those distances relationships.

One of the first solutions of this form were proposed by Kwok and Tsang in their paper “The Pre-Image Problem in Kernel Methods". A better approach is also described by Bakir on his thesis “Extensions to Kernel Dependency Estimation”, alongside with a nice comprehensive description on various other methods for handling the pre-image problem.

Note: actually the use of KPCA for compression is a bit limited, since one needs to store the original data set to compute transformations and its approximate reversion.

 

Code Overview

Below is the main code behind KPCA. This is a direct, naive implementation of the algorithm that may not scale very well for very large datasets. It is implemented on top of the previous standard PCA code and thus on top of the AForge.NET Framework.

 

The main code behind the reversion of the projection. It is a direct, naive implementation of the closed-form solution algorithm as proposed by Kwok and Tsang. Currently it is completely not optimized, but it gives an better overview of the algorithm on its essential form. The source code available to download may be better optimized in the future.

 

 

Using the Code

As with PCA, code usage is simple. First, choose a kernel implementing the IKernel interface then pass it to the KernelPrincipalComponentAnalysis constructor together with the data points.

Then, simply call Compute() to perform the analysis. Once done, data about the analysis will be available through the object properties and transformations on additional data can be computed by calling Transform().

 

Sample Application

To demonstrate the use of KPCA, I created a simple Windows Forms Application which performs simple statistical analysis and KPCA transformations. It can also perform projection reversion by solving the approximate pre-image problem.

Scholkopf's KPCA Toy Example Kernel principal components detected by the KPCA Two first components detected by the KPCA

On left: Scholkopf Toy Example. Middle: Kernel principal components detected by the KPCA. Right: The two first dimensions obtained by the two first Kernel principal components.

 

Wikipedia's Example on KPCA Projection using a non-centered Gaussian kernel Reconstruction of the original data

Left: Wikipedia example. Middle: Projection using a non-centered Gaussian kernel. Right: Reconstruction of the original data using all Kernel principal components, showing some denoising ability.

Linear PCA as a special case

Lindsay Smith's PCA example data Linear Principal Component Analysis

Additionally we may check that linear PCA is indeed a special case of kernel PCA by using a linear kernel on the example by Lindsay Smith on his A Tutorial On Principal Component Analysis.

 

See also

 

References

Principal Component Analysis in C#

Principal Component Analysis (PCA) is an exploratory tool designed by Karl Pearson in 1901 to identify unknown trends in a multidimensional data set. It involves a mathematical procedure that transforms a number of possibly correlated variables into a smaller number of uncorrelated variables called principal components.

Foreword

Before you read this article, please keep in mind that it was written before the Accord.NET Framework was created and became popular. As such, if you would like to do Principal Component Analysis in your projects, download the accord-net framework from NuGet and either follow the starting guide or download the PCA sample application from the sample gallery in order to get up and running quickly with the framework.

Introduction

PCA essentially rotates the set of points around their mean in order to align with the first few principal components. This moves as much of the variance as possible (using a linear transformation) into the first few dimensions. The values in the remaining dimensions, therefore, tend to be highly correlated and may be dropped with minimal loss of information. Please note that the signs of the columns of the rotation matrix are arbitrary, and so may differ between different programs for PCA.

For a more complete explanation for PCA, please visit Lindsay Smith excellent Tutorial On Principal Component Analysis (2002).

Accord.NET Framework

This new library, which I called Accord.NET, was initially intended to extend the AForge.NET Framework through the addition of new features such as Principal Component Analysis, numerical decompositions, and a few other mathematical transformations and tools. However, the library I created grew larger than the original framework I was trying to extend. In a few months, both libraries will merge under Accord.NET. (Update April 2015)

Design decisions

As people who want to use PCA in their projects usually already have their own Matrix classes definitions, I decided to avoid using custom Matrix and Vector classes in order to make the code more flexible. I also tried to avoid dependencies on other methods whenever possible, to make the code very independent. I think this also made the code simpler to understand.

The code is divided into two projects:

  • Accord.Math, which provides mathematical tools, decompositions and transformations, and
  • Accord.Statistics, which provides the statistical analysis, statistical tools and visualizations.

Both of them depends on the AForge.NET core. Also, their internal structure and organization tries to mimic AForge’s wherever possible.

The given source code doesn’t include the full source of the Accord Framework, which remains as a test bed for new features I’d like to see in AForge.NET. Rather, it includes only limited portions of the code to support PCA. It also contains code for Kernel Principal Component Analysis, as both share the same framework. Please be sure to look for the correct project when testing.

Code overview

Below is the main code behind PCA.

Using the code

To perform a simple analysis, you can simple instantiate a new PrincipalComponentAnalysis object passing your data and call its Compute method to compute the model. Then you can simply call the Transform method to project the data into the principal component space.

A sample sample code demonstrating its usage is presented below.

Example application

To demonstrate the use of PCA, I created a simple Windows Forms Application which performs simple statistical analysis and PCA transformations.

input_thumb-5B2-5D

The application can open Excel workbooks. Here we are loading some random Gaussian data, some random Poisson data, and a linear multiplication of the first variable (thus also being Gaussian).
 
univariate_thumb-5B4-5D
Simple descriptive analysis of the source data, with a histogram plot of the first variable. We can see it fits a Gaussian distribution with 1.0 mean and 1.0 standard deviation.
 
principal_thumb-5B9-5D
Here we perform PCA by using the Correlation method. Actually, the transformation uses SVD on the standardized data rather than on the correlation matrix, the effect being the same. As the third variable is a linear multiplication of the first, the analysis detected it as irrelevant, thus having a zero importance level.
 
projection_thumb-5B4-5D
Now we can make a projection of the source data using only the first two components.
 

 

Note: The principal components are not unique because the Singular Value Decomposition is not unique. Also the signs of the columns of the rotation matrix are arbitrary, and so may differ between different programs for PCA.

Together with the demo application comes an Excel spreadsheet containing several data examples. The first example is the same used by Lindsay on his Tutorial on Principal Component Analysis. The others include Gaussian data, uncorrelated data and linear combinations of Gaussian data to further exemplify the analysis.

I hope this code and example can be useful! If you have any comments about the code or the article, please let me know.

See also

A Tutorial On Principal Component Analysis with the Accord.NET Framework

This is a tutorial for those who are interested in learning how PCA works and how each step of Lindsay’s tutorial can be computed in the Accord.NET Framework, in C#.

Kernel Principal Component Analysis in C#

This is the non-linear extension of Principal Component Analysis. While linear PCA is restricted to rotating or scaling the data, kernel PCA can do arbitrary transformations (such as folding and twisting the data and the space that contains the data).

Instalando o Debian no Notebook HP DV5 1251

Este é um tutorial de como instalar o sistema operacional Debian GNU/Linux versões testing ou unstable em um notebook HP dv5 1251. Este notebook possui processador Intel Core 2 Duo e adaptador gráfico NVIDIA com memória dedicada. Este tutorial possivelmente se aplica a outros modelos da série dv5-1000.

Especificacões do notebook HP dv5 1251nr:

  • Processador: Intel Core 2 Duo Processor T6400
  • Sistema Operacional: Windows Vista Home Premium 64-bit
  • Tela: 15.4" (diagonal) high-definition widescreen
  • Memória: 4GB RAM
  • Disco Rígido: 320GB
  • Memória Gráfica: Memória dedicada de 512MB
  • Duracão da Bateria: Até 2 horas e 15 minutos
  • Dispositivos:
    • Leitor de Cartões 5-em-1
    • Webcam de 2.0 megapixels
  • Dimensões: 14.05" (W) x 10.2" (D) x 1.37" (min H)/1.65"(max H)
  • Cores: Onyx / Bronze

 

Índice

  1. Obtendo a mídia de instalação
  2. Instalando o Debian GNU/Linux
  3. Configurando o Gerenciador de Pacotes
  4. Configurando os Dispositivos
    1. Adaptador Wireless (Intel WiFi Link 5100)
    2. Adaptador Gráfico (GeForce 9200M GS)
    3. Adaptador de Áudio (Intel IHT High Definition Audio)
  5. Dispositivo de Toque (Synaptics Touchpad)
  6. Configurações Adicionais
    1. Acessando outras partições
    2. Configurando layout de teclado e idioma
  7. Sugestões Adicionais

 

 

Obtendo a mídia de instalacão

Baixe o cd de instalação da distribuição testing com KDE embutido. Este manual se refere à versão com KDE, porém outras versões também podem funcionar. O CD de instalação da distribuição testing é gerado automaticamente a cada semana, portanto não se preocupe em gravar um CD-R, grave num CD-RW para evitar o desperdício de discos.

Após gravado, deixe o CD na tray e reinicie seu notebook.

 

Instalando o Debian GNU/Linux

Após colocar o cd na bandeja, aguarde para cair na tela inicial de instalação. Selecione o modo texto, pois o gráfico não funcionou quando tentei. Na build particular do instalador que utilizei, a Wireless também não funcionou durante a instalação. Na verdade, o chipset wireless é reconhecido sem problemas, mas o botão de ligar não funciona, o que a deixa desabilitada e consequentemente nos deixa sem rede.

Adicionalmente, encontrei alguns problemas com o particionador pois, quando instalei o Windows 7, criei uma partição adicional no disco. Como o Windows 7 também criou uma partição especial extra, e como a HP coloca uma partição de restore em todos seus notebooks, isto me deixou com 4 partições primárias. Assim, por mais que houvesse espaço não alocado disponível no disco para se criar uma nova partição ext4, o instalador não conseguia criar uma partição a mais, pois este é o limite de partições primárias que um disco com particionamento DOS pode ter.

A solução foi remover a partição de restore da HP e criar uma nova partição extendida no espaço restante para acomodar duas novas partições lógicas, uma partição ext4 para servir de root e a swap.

Feito isso, a instalação seguiu sem mais problemas.

 

Configurando o Gerenciador de Pacotes

No meu caso, utilizei a distribuição unstable para pegar o restante dos pacotes. Mas você também pode continuar com a versão testing se preferir.

De qualquer forma, adicione as seguintes linhas a seu /etc/apt/sources.list:

deb http://http.us.debian.org/debian/ testing main contrib non-free
deb http://deb.opera.com/opera/ testing non-free
deb http://dl.google.com/linux/deb/ stable non-free main

A seguir prosseguiremos configurando cada componente do notebook em separado. Ao final deste tutorial, todos os dispositivos, como vídeo, webcam, áudio, microfone e touchpad estarão funcionando corretamente.

 

Configurando os Dispositivos

Adaptador Wireless (Intel WiFi Link 5100)

O chipset wireless Intel 5100 funciona bem no linux. Tanto que já deve ter sido instalado por padrão durante a instalação. No entanto, muito provavelmente ele ainda não deve estar funcionando.

O problema aqui é que, por alguma razão, o botão touch que habilita/desabilita a wireless (que fica azul e laranja em cima do teclado) não funciona corretamente. Se você executar o comando dmesg para ver as mensagens do kernel, certamente irá encontrar a linha "Radio disabled by HW RF Kill switch".

Para contornar este problema enquanto não temos uma solução definitiva, digite:

sudo modprobe -r iwlagn && sudo modprobe iwlagn

Isto deverá reabilitar a sua wireless. Contudo, tenha em mente que, toda vez a wireless for desligada através do touchbutton este comando terá de ser executado de novo.

 

Adaptador Gráfico (NVIDIA GeForce 9200M GS)

Instalar placas de vídeo nvidia geralmente é um procedimento simples no linux, ainda mais no Debian. Primeiro, instale os pacotes module-assistant e nvidia-kernel-common:

# apt-get install module-assistant nvidia-kernel-common nvidia-glx

Em seguida, execute o module-assistant digitando m-a no terminal como root:

# m-a

Basta então preparar, selecionar, compilar e instalar o modulo nvidia seguindo o assistente.

 

Após instalar o módulo nvidia e o pacote nvidia-glx, faltará pouco para concluir a instalação. O Debian não inclui mais o arquivo padrão /etc/X11/xorg.conf contendo as configurações básicas do xorg porque, ao menos teoricamente, agora o xorg seria capaz de identificar automaticamente qual driver deve ser utilizado com cada adaptador gráfico. O problema é que, como ele não tenta utilizar o driver oficial da nvidia, acaba falhando miseravelmente neste processo e nos deixando na mão.

Será necessário, então, criar este arquivo e inserir nele as seguintes linhas:

Para finalizar, salve o arquivo e reinicie o X. Recomendo também a instalação do pacote nvidia-settings para ajustar configurações adicionais de sua placa de vídeo, como o dual display e o twinview.

 

Adaptador de Áudio (Intel IDT High Definition Audio)

Para que o dispositivo de áudio funcione corretamente será necessário adicionar uma linha de configuração no final do arquivo alsabase.conf localizado no diretório /etc/modprobe.d/. Para isto, é possível abrir o arquivo com:

nano /etc/modprobe.d/alsabase.conf

Após abrir, navegue até sua ultima entrada e então insira a seguinte linha:

options snd-hda-intel model=hp-dv5 enable_msi=1

Após editar o arquivo, reinicie o sistema ou digite update-modules -f.

Habilitando o suporte a arquivos MP3

Para tocarmos arquivos mp3 é preciso instalar o decoder de áudio ffmpeg e seu respectivo plugin para a engine de áudio xine. Para isto, digite:

sudo apt-get install libxine1-ffmpeg ffmpeg

 

Dispositivo de Toque (Synaptics Touchpad)

Para configurarmos o touchpad, primeiro precisamos habilitar seu compartilhamento de configuração. Para isto, você precisa criar um arquivo texto de nome shmconfig.fdi no diretório /etc/hal/fdi/policy/ e inserir nele as configurações desejadas. Para isto, podemos criar este arquivo utilizando o nano:

# nano /etc/hal/fdi/policy/shmconfig.fdi

Para em seguida inserir, dentro do arquivo, o seguinte código:

Em seguida, baixe o pacote kcm_touchpad para obter um modulo do painel de controle do KDE capaz de gerenciar seu touchpad. No momento da elaboração deste tutorial o pacote ainda não estava disponível via apt-get, mas podia ser obtido em:

http://opendesktop.org/content/show.php/kcm_touchpad

Para instalar, baixe a versão para Ubuntu e a instale utilizando o comando dpkg –i <pacote.deb>.

 

Configurações Adicionais

Acessando outras partições do sistema

Para configurar rapidamente a montagem das outras partições de seu sistema, como por exemplo a partição do Windows, baixe os pacotes ntfs-3g e ntfs-config pelo apt-get:

apt-get install ntfs-3g ntfs-config

E então digite:

sudo ntfs-config

Basta então seguir as instruções na tela para configurar o acesso a cada partição detectada.

 

Configurando layout do teclado e idioma

Para o teclado funcionar corretamente deve-se utilizar o layout de teclado USA variante International (with dead keys). O comando resultante na janela do System Settings (KDE) deverá ser:

setxkbmap -model pc104 -layout us -variant intl

 

Sugestões Adicionais

Wine (para executar aplicativos windows)

apt-get install wine

Mono (para executar aplicativos .NET)

apt-get install mono-runtime

GTK-QT Engine (para aplicar estilos QT em aplicativos GTK)

apt-get install gtk-qt-engine system-config-gtk-kde

Sun Java 6 (máquina virtual java)

apt-get install sun-java6-bin sun-java6-plugin

Flash plugin (para visualizar sites em flash)

apt-get install flashplugin-nonfree

Splashy (para exibir uma splash screen durante o boot do sistema)

apt-get install splashy startupmanager

Voce se considera um bom programador?

bug

Você se considera um bom programador? Se respondeu sim a esta pergunta, então aposto que não vai deixar este bug passar despercebido:

E ai, vê alguma coisa?

 

Óbvio não?

 

 

Dica: olhe atentamente para a linha 6:

Ainda não percebeu? Então digamos, o que acontece, se, por acaso, low valer 10 e high valer 2.147.483.647 (ou seja, 2^31-1, o valor máximo que um inteiro de 32 bits pode possuir)?

Overflow!

Sim, de fato, a maioria das implementações de busca binária possui este erro. Não apenas livros, apostilas e materiais didáticos, mas sistemas críticos de produção também. Mas ai você fala: Ahh mas eu nunca vou empregar este tipo de busca em algo tão grande! – Bom, talvez não, mas não se esqueça que este é o mecanismo chave de muitos outros algoritmos de divisão-e-conquista, como, por exemplo, o mergesort.

 

Mais do que isto, esta observação serve para elucidar o quão é difícil garantir a corretude de um software. Demoraram 16 anos para detectar um erro no algoritmo que é, provavelmente, o mais popular de toda computação, presente em virtualmente todos os livros da área. Pois é: Planejamento cuidadoso é uma bênção, testar é uma maravilha. Utilizar métodos formais, melhor ainda. Revisão de código e análise estatística? Impossível ainda restar algum bug! Errado! Nenhuma destas coisas são suficientes para eliminarmos todos os bugs de um software. A verdade é: Eles sempre vão estar presentes e nos acompanhar por toda nossa vida :~

 

Ah, a correção? Simplesmente mude a linha 6 para:

Mágica.

 

[Fonte: Official Google Research Blog]

The weirdest Microsoft’s workaround ever

microsoft-solution-en-2_thumb-5B3-5D

Here is Microsoft’s solution for a problem with Excel 97 when using a Oracle data source. Now take a closer look on the second method suggested by Redmond:

microsoft-solution-en-2

If you move your mouse pointer continuously while the data is being returned to Microsoft Excel, the query may not fail. Do not stop moving the mouse until all the data has been returned to Microsoft Excel.

And I always thought improving performance by waving the mouse around was just silly superstition. Gee, I was wrong!

 

[Source: The CodeProject’s Lounge]

Introducing System.Console.Forms

terminal

Some days ago I started a new project just for fun. System.Console.Forms is a extension to .NET framework to “port” as much as possible of System.Windows.Forms to a Console environment.

 

I know a lot of people think there is absolutely no reason for developing console programs nowadays, but they are just plain wrong. It obviously isn’t mainstream, but if you use ssh a lot and cannot afford a X connection, then working with console interfaces makes a lot of sense. Also, I know there are other known, well-tested and proven solutions for console user interfaces like the ncurses library, but they are not very object oriented and not very user-friendly if you came from a .NET WinForms or Java SWT/Swing world.

Thus this project was started.

 

I know accomplishing this task would consume a lot of time and effort, so I can’t promise it will ever be complete, or even if it will not die a few months from now. But if are interested and want to help, feel free to contact me. Much of the base code is almost done (it is already possible to create Forms and UserControls) but most of the default controls, like TextBoxes, RadioButtons and ToolStripMenus are still incomplete or missing.

nano#nano# - 2

As you can see, it currently lacks some visual enhancements (such as drop-shadow borders and a better default form design), but this is something scheduled only for future releases. The first milestone is to replicate only the basic functionality and interface of nano, and the second is to fully reproduce MS-DOS edit.exe. When this becomes possible, the library will be almost feature complete.

I don’t think I’ll ever be able to finish this by myself alone, so I licensed the project under the LGPL if anyone else gets interested. The project is being hosted at Google Code, and the project page is http://console-forms.googlecode.com.