The list of accepted students is available at http://studies.aalto.fi/en/admissions/admissionresults/

Intelligent algorithms and machine learning perform tasks that cannot be solved by the human brain. Statistical modelling of vast information sources is a must in decision making in numerous fields of society, from medicine to traffic planning and from analyses of share markets to spam identification. However, to interpret and evaluate the functionality of the models, a critically-oriented human being is needed.

In their most complex form, statistical models are based up to many thousands of explanatory variables. The most important thing, therefore, is to find from the material the features that describe the target of investigation as precisely as possible. The task known as feature selection seems an obvious choice, but attempts have been made already over a half a century to solve this basic pattern recognition problem.

Juha Reunanen, in his doctoral dissertation, Overfitting in Feature Selection: Pitfalls and Solutions, at the Aalto University School of Science, shows that feature selection methods are often compared and evaluated on unjustified grounds. Generally, two wrong conclusions are made about pattern recognition: it is thought that computationally intensive and slow search algorithms as well as fine-grained feature selection will produce the most accurate results.

– Everyone wants to introduce a new, and the best, feature selection method. Nevertheless, comparisons and choice between these is not as easy as it is often believed during investigation, Reunanen summarizes a key problem in his field of science.

Results that are too good to be true and great little errors

According to Reunanen, the problem lies more on overfitting in models that use machine learning methods to classify statistical data than in simple algorithms or algorithms that have not been pruned.

A statistical model is overfitted when it is capable only for repeating information that has been fed to it but unable to describe and classify new data. The model learns about all the pikes swimming in the pond that the fish in question is a pike but not why it is a pike.

– This basic overfitting, of course, has been recognized and identified since the rudimentary statistical modelling, but it is less frequent that the "overfitting of the second kind" would become accounted for, Reunanen observes.

When it is believed that if a particular fish species swimming in a particular pond can be identified with 95 percent accuracy by a model having a certain variable set and that the model subsequently would perform just as well with different fish species, lakes, rivers and oceans, we come across with the interpretation error discovered by Reunanen. The mistake is not necessarily noted, because the results of overfitting models are often deceptively good.

– It is hard to draw any conclusions. It is far too easy to get excited over some good results obtained by statistical models and pattern recognition methods. The researcher should exercise self-criticism and take a hard look whenever a promising explanation emerges.

Thus, rather than having discovered an optimal set of variables or created a brilliant algorithm, we might fallen a victim of a statistical illusion.

– Accuracy and prediction capabilities of multiple variable models are especially important in fields where the difference between 85 and 95 percent probability matters. For example, if a model meant as a tool for a physician can diagnose a rare illness with 95 percent accuracy, it pays to use it.

However, we wouldn't like the tool to consist of variables that require biopsies, which are dangerous and painful to the patient, nor would we like exhausting wait for the results.

– To dramatize a bit, a model selected with correct methods should not need to include variables that require trepanation for the patient.

Additional information:

Juha Reunanen

juha.reunanen@iki.fi

tel. +358 50 375 4475

The workshop takes place in Mariehamn, the Åland Islands, on 23-26 May 2012, and is supported by the Aalto Science Institute, Nordita, the ACCESS Linneaus Centre in Sweden, and the graduate schools Hecse, FICS and NGSMP.

Further information and instructions for registration can be found on the workshop's web page at http://agenda.albanova.se/conferenceDisplay.py?confId=3181. Registration closes on 1 April.

• Kernels for sequences, trees and graphs
• Multi-view learning, multiple kernel learning
• Multi-task/Multi-label learning
• Structured output prediction

will be explored.

Applications are found in, e.g.

• Bioinformatics (e.q. sequence annotation, gene function prediction)
• Natural language processing and information retrieval (e.g. classification of documents)
• Image analysis

More information:  https://noppa.aalto.fi/noppa/kurssi/t-61.9910/