EM Tutorial

EM Tutorial

Using PageRank to Locally Partition A Graph

Using PageRank to Locally Partition A Graph

An Introduction to Conditional Random Fields

CRF Examples...

t-SNE

t-Distributed Stochastic Neighbor Embedding (t-SNE)

A Brief Introduction to Graphical Models and Bayesian Networks

http://www.cs.ubc.ca/~murphyk/Bayes/bnintro.html

Advanced Machine Learning with scikit-learn

Mid-Atlantic Student Colloquium on Speech, Language and Learning

UMBC is hosting the Mid-Atlantic Student Colloquium on Speech, Language and Learning.

Register online.

The schedule is now posted.

Snapshot of the schedule:

09:00-09:45 Registration, set up

09:45-10:00 Opening

10:00-11:20 Oral presentations I

Lushan Han, Abhay Kashyap, Tim Finin, James Mayfield and Jonathan Weese (UMBC & JHU). Semantic Textual Similarity Systems

Keith Levin, Aren Jansen and Ben Van Durme (JHU). Toward Faster Audio Search Using Context-Dependent Hashing

Shawn Squire, Monica Babes-Vroman, Marie Desjardins, Ruoyuan Gao, Michael Littman, James MacGlashan and Smaranda Muresan (UMBC & Brown). Learning to Interpret Natural Language Instructions

Viet-An Nguyen, Jordan Boyd-Graber and Philip Resnik (UMCP). Lexical and Hierarchical Topic Regression

11:20-12:10 Poster session I

Posters

12:10-12:40 Lunch

12:40-1:40 Panel

"How to be a successful PhD student and and transition to a great job"
Marie desJardins (UMBC)
Mark Dredze (JHU)
Claudia Pearce (DoD)
Ian Soboroff (NIST)
Hanna Wallach (UMass)


1:50-3:10 Oral presentations II

Qingqing Cai and Alexander Yates (Temple). Large-scale Semantic Parsing via Schema Matching and Lexicon Extension

William Yang Wang and William W. Cohen (CMU). Efficient First-Order Probabilistic Logic Programming for Natural Language Inference

Xuchen Yao, Benjamin Van Durme, Chris Callison-Burch, Peter Clark (JHU & UPenn & AI2). Semi-Markov Phrase-based Monolingual Alignment

Wei Xu, Alan Ritter and Ralph Grishman (NYU). Gathering and Generating Paraphrases from Twitter with Application to Normalization

3:10-4:00 Poster session II

Posters

4:00-5:00 Breakout sessions

NLP in low resource settings, Ann Irvine (JHU)

Dynamic Programming: Theory and Practice, Alexander Rush (Columbia/MIT)

NELL: Never Ending Language Learning, Partha Pratim Talukdar (CMU)

5:00 Closing

5:15 - 7:00 Wine down

Wine and beer at Flat Tuesdays, UMBC Commons

Never Ending Learning (NELL)

Attended a very interesting talk related to Nell. The work is from Carnegie Mellon. If you are interested in machine learning, particularly how to perform learning over time, you may find this work of interest.

UMBC ebiquity group

Our group is performing interesting research by blending semantic computing with security, mobile devices, medical informatics and big data problems. Some of this research overlaps quite a bit with natural language processing. Read our publications to see the latest research coming from our lab.

Bias and Variance Tradeoff

There is a great blog entry that describes this from a practical standpoint. If references the following lecture also.

LDA - Step by Step in R

Was able to get an example running very quickly with this tutorial. http://www.rtexttools.com/1/post/2011/08/getting-started-with-latent-dirichlet-allocation-using-rtexttools-topicmodels.html

Topic Modeling - Great Resource

Excellent resource! http://www.cs.princeton.edu/~blei/topicmodeling.html

SVMs - Good Description

http://nlp.stanford.edu/IR-book/html/htmledition/support-vector-machines-the-linearly-separable-case-1.html

Excellent List of Machine Learning Resources

A List of Data Science and Machine Learning Resources

PCA, SVD, LSA

Great links to learn and understand these concepts:
PCA Tutorial
SVD and PCA discussion
LSA Tutorial

Paper Summary - Twitter Sentiment Classification using Distant Supervision

Twitter Sentiment Classification using Distant Supervision, A. Go and R. Bhayani and L. Huang, 2009, Technical report, Stanford Digital Library Technologies Project
This paper relates to classifying sentiment found on Twitter. They use machine learning and are able to construct training data by using the Twitter API and emoticons present among tweets. The standard :) and :( are used to determine if a tweet contains positive or negative sentiment. The key points are 1.) they picked an efficient way to construct their training sets, 2.) Tweets are harder to classify because their length can not exceed 140 characters, and 3.) their results were promising for classifying the sentiment of the tweets.

Data Mining and Machine Learning

I wished to understand the distinction between data mining and machine learning. This presentation (Machine Learning and Data Mining: 01 Data Mining) is useful.

Canopy Clustering

"Efficient Clustering of High Dimensional Data Sets with Application to Reference Matching", McCallum,Nigam,Ungar,http://www.kamalnigam.com/papers/canopy-kdd00.pdf

This paper discusses a different type of clustering, titled canopy clustering. It is an interesting idea. There are basically two thresholds, using a 'cheap distance metric', we evaluate a list of points. Threshold 1 is > than Threshold 2. Pick one point to compare with all the other points in the list. When the distance between the two points falls within T1 put the points into a canopy. If the distance falls within T2 then remove point from list. We generate the canopies this way and work through the list until empty.

We can then apply our second level of clustering to each canopy and are pretty much guaranteed that if two points do not fall into the same canopy then they are likely not to be co-referent and therefore do not need to be evaluated.

This is efficient and elegant. Currently the only implementation that I found of canopy clustering is in Mahout. I am building my own implementation though to get a feel for how well it works.

Narrowing the Modeling Gap: A Cluster-Ranking Approach to Coreference Resolution

"Narrowing the Modeling Gap: A Cluster-Ranking Approach to Coreference Resolution"
Journal of Artificial Intelligence Research 40 (2011) 469–521 Submitted 06/10; published 02/11
Altaf Rahman altaf@hlt.utdallas.edu
Vincent Ng vince@hlt.utdallas.edu
Human Language Technology Research Institute
University of Texas at Dallas
800 West Campbell Road; Mail Station EC31
Richardson, TX 75080-3021 U.S.A.
http://www.jair.org/media/3120/live-3120-5478-jair.pdf

-Long paper, very thorough
-A lot of history, use the citations in this paper
-Learn more about "centering algorithms"
-Describes three different models, mention-pair,entity-mention and mention-ranking
-Outlines key features and deficiencies of each
-In particular the transitivity property is not addressed in the mention-pair model so clustering is used
-Mention-ranking outperforms mention-pair
-Describe a cluster-ranking approach combines both models
-Use lexicalization and knowledge of anaphoricity
-Used ACE for experiments

Interesting:

1. 
   "Specifically, a classifier that is trained on
   coreference-annotated data is used to determine whether a pair of mentions is co-referring
   or not. However, the pairwise classifications produced by this classifier (which is now commonly
   known as the mention-pair model) may not satisfy the transitivity property inherent
   in the coreference relation, since it is possible for the model to classify (A,B) as coreferent,
   (B,C) as coreferent, and (A,C) as not coreferent. As a result, a separate clustering mechanism
   is needed to coordinate the possibly contradictory pairwise classification decisions and
   construct a partition of the given mentions."
   
2. 
   Read about Lappin and Leass’s algorithm
   
3. 
   Read about centering algorithms
   
4. 
   "the distinction between
   classification and ranking applies to discriminative models but not generative models.
   Generative models try to capture the true conditional probability of some event. In the context
   of coreference resolution, this will be the probability of a mention having a particular
   antecedent or of it referring to a particular entity (i.e., preceding cluster). Since these probabilities
   have to normalize, this is similar to a ranking objective: the system is trying to raise
   the probability that a mention refers to the correct antecedent or entity at the expense of
   the probabilities that it refers to any other. Thus, the antecedent version of the generative
   coreference model as proposed by Ge et al. (1998) resembles the mention-ranking model,
   while the entity version as proposed by Haghighi and Klein (2010) is similar in spirit to the
   cluster-ranking model."
   

Paper Summary - Ontology matching: A machine learning approach

A. Doan and J. Madhaven and P. Domingos and A. Halevy, "Ontology matching: A machine learning approach", Handbook on Ontologies in Information Systems,2004,397-416, Springer-Verlag


This paper is about finding mappings between ontologies and discusses the system GLUE. Using learning techniques it semi-automatic generates mappings. This work attempts to address the issue of matching concept nodes.

It begins by discussing the meaning of similarity and a discussion around using the joint probability distribution of concepts.

It then discusses the complexities of computing the JPD for two different concepts.

It then discusses using machine learning to use instances of one concept to learn a classifier for that concept and the same for the second concept.

Rather than a single algorithm is uses multiple algorithms and then combines their predictions.

This paper is long. Their techniques are novel and interesting. This is a good paper to use for machine learning techniques for instance matching.