Final project for MIT 15.062 Data Mining class. Include a project report & R code written for the project
The labor condition application (LCA) is an application filed by prospective employers on behalf of employees applying for an employment-based, non-immigrant visa (H-1B, or variants of H-1B) in the United States and is necessary to get certified for H-1B visa petitions. In this project, I aim to predict the certification and denial decisions of the LCA (decision from the step 1 by using (1) k-nearest neighbors classifier, (2) classification trees, (3) logistic regression and (4) neural nets. I have processed, cleaned and explored the LCA raw data obtained from the DOLETA’s OFLC webpage: https://www.foreignlaborcert.doleta.gov/performancedata.cfm#dis. Then, I have built four prediction models on CERTIFIED/DENIED decisions with the balanced and unbalanced data sets using two different approaches in treating the categorical variables, and evaluated the performance of the classification models compared to two baseline models. From the data processing, cleaning and exploitation, we have learned that the units of wages need to be consistent with the amount of wage and that the proposed wage rate of pay needs to be at least the prevailing wage. From the prediction models, we have obtained higher accuracy using the four classification models with balanced data set than using the baseline models. The classification tree performed the best among the four models, probably because the U.S. DOL makes the CERTIFIED/DENIED decisions based on some rules and logics and the classification tree model can mimic the decision making process better than other models. From the classification tree model, I find the chance of CERTIFIED decisions increases, as H1B_DEPENDENT exists, AGENT_REPRESENTING_EMPLOYER exists, PW_SOURCE is from OES and the EMPLOYER_NAME files more applications. On top of the current four classification models that already improved the prediction accuracy form the baseline models, I believe that the prediction models can be further improved by working on several possible approaches such as implementing PCA, checking invalid and inaccurate information and analyzing text variables better.