Capital One: Optimize predictions using model benchmarking on high-feature synthetic data. Python, Scikit-learn, Pandas, Matplotlib.
The two synthetic datasets were generated using the same underlying data model. Your goal is to build a predictive model using the data in the training dataset to predict the withheld target values from the test set.
You may use any tools available to you for this task. Ultimately, we will assess predictive accuracy on the test set using the mean squared error metric. You should produce the following:
We have provided two tab-delimited files along with these instructions:
codetest_train.txt: 5,000 records x 254 features + 1 target (~7.8MB)codetest_test.txt : 1,000 records x 254 features (~1.5MB)The purpose of this test is to test your ability to write software to collect, normalize, store, analyze and visualize “real world” data. The test is designed to take about two hours.
You may also use any tools or software on your computer, or that are freely available on the Internet. We prefer that you use simpler tools to more complex ones and that you are “lazy” in the sense of using third party APIs and libraries as much as possible. We encourage the reuse of code when appropriate. If you include code directly in your submission that was written by someone else other than commonly imported modules, please be sure to provide proper attribution, including a URL, text, author, etc. or other available information in the code comments.
Do as much as you can, as well as you can. Prefer efficient, elegant solutions. Prefer scripted analysis to unrepeatable use of GUI tools. For data security and transfer time reasons, you have been given a relatively small data file. Prefer solutions that do not require the full data set to be stored in memory.
There is certainly no requirement that you have previous experience working on these kinds of problems. Rather, we are looking for an ability to research and select the appropriate tools for an open-ended problem and implement something meaningful. We are also interested in your ability to work on a team, which means considering how to package and deliver your results in a way that makes it easy for others to review them. Undocumented code and data dumps are virtually useless; commented code and a clear writeup with elegant visuals are ideal.