Bias and Variance Tradeoff

There is a great blog entry that describes this from a practical standpoint. If references the following lecture also.

Excellent List of Machine Learning Resources

A List of Data Science and Machine Learning Resources

PCA, SVD, LSA

Great links to learn and understand these concepts:
PCA Tutorial
SVD and PCA discussion
LSA Tutorial

Paper Summary - Short text classification in twitter to improve information filtering

Short text classification in twitter to improve information filtering, B. Sriram and D. Fuhry and E. Demir and H. Ferhatosmanoglu,2010
This paper describes research that classifies tweets using a reduced set of features. In this approach they try to classify text into the following set of classes "News, Events, Opinions, Deals, and Private Messages". The problem they present is the curse of dimensionality problem that results from trying to conquer the spareness issue related to classifying twitter messages. Other research typically uses external knowledge bases to support tweet classification. They argue this can be slow due to the need to excessively query the external knowledge base. Important points about this paper:
1. They provide a very useful discussion of Twitter and tweets
2. How they classify tweets is interesting worth another review

Data Mining Resources

This thread has a lot of useful links for resources to help one studying data mining. It is mainly to help one build the mathematical background.

Data Mining and Machine Learning

I wished to understand the distinction between data mining and machine learning. This presentation (Machine Learning and Data Mining: 01 Data Mining) is useful.

Canopy Clustering

"Efficient Clustering of High Dimensional Data Sets with Application to Reference Matching", McCallum,Nigam,Ungar,http://www.kamalnigam.com/papers/canopy-kdd00.pdf

This paper discusses a different type of clustering, titled canopy clustering. It is an interesting idea. There are basically two thresholds, using a 'cheap distance metric', we evaluate a list of points. Threshold 1 is > than Threshold 2. Pick one point to compare with all the other points in the list. When the distance between the two points falls within T1 put the points into a canopy. If the distance falls within T2 then remove point from list. We generate the canopies this way and work through the list until empty.

We can then apply our second level of clustering to each canopy and are pretty much guaranteed that if two points do not fall into the same canopy then they are likely not to be co-referent and therefore do not need to be evaluated.

This is efficient and elegant. Currently the only implementation that I found of canopy clustering is in Mahout. I am building my own implementation though to get a feel for how well it works.

Paper Summary - Data Mining: An Overview from a Database Perspective

M. Chen and J. Han and P. Yu, "Data Mining: An Overview from a Database Perspective", IEE Transactions on Knowledge and Data Engineering, 8(6): 866-883, 1996

This is a seminal paper about mining information from large databases. It is a survey of data mining techniques from a database researcher perspective.

The paper discusses key feature and challenges:

• Different types of data
  
• Efficiency and Scalability of algorithms
  
• Accuracy and usefulness of results
  
• How results are conveyed
  
• Multiple Abstraction Levels
  
• Mining different sources
  
• Privacy and security
  

They go on to classify different types of data mining schemes. They can be classified according to the data they are examining, according to the kind of knowledge they are mining, and according to the technique they implore. This paper focuses on the knowledge they are mining.

• Association rules
  
• Data Generalization and Summarization
  
• Classification huge amount of data
  
• Data Clustering
  
• Pattern based similarity
  
• Path traversal patterns
  

It describes each one of this items in great detail. The paper is a great paper to get a good foundation on this topic. It is quite long but detailed. I don't seem mention of confidence approach.