Capital One: Trace naming trends using temporal analysis and gender ambiguity metrics. Python, Pandas, Matplotlib, WordCloud.
A) Descriptive analysis
B) Onward to Insight!
What insight can you extract from this dataset? Feel free to combine the baby names data with other publicly available datasets or APIs, but be sure to include code for accessing any alternative data that you use.
This is an openended question and you are free to answer as you see fit. In fact, we would love it if you find an interesting way to look at the data that we haven't thought of!
Please provide us with both your code and an informative writeup of your results. The code should be in a runnable form. Do not assume that we have a copy of the data set or that we are familiar with the build procedures for your chosen language.
You are given state-specific data on the relative frequency of given names in the population of U.S. births where the individual has a Social Security Number (Tabulated based on Social Security records as of March 6, 2022)
For each of the 50 states and the District of Columbia we created a file called SC.txt, where SC is the state's postal abbreviation.
Each record in a file has the format: 2-digit state code, sex (M = male or F = female), 4-digit year of birth (starting with 1910), the 2-15 character name, and the number of occurrences of the name. Fields are delimited with a comma. Each file is sorted first on sex, then year of birth, and then on number of occurrences in descending order. When there is a tie on the number of occurrences names are listed in alphabetical order. This sorting makes it easy to determine a name's rank. The first record for each sex & year of birth has rank 1, the second record has rank 2, and so forth.
To safeguard privacy, we restrict our list of names to those with at least 5 occurrences. If a name has less than 5 occurrences for a year of birth in any state, the sum of the state counts for that year will be less than the national count.
The purpose of this test is to test your ability to write software to collect, normalize, store, analyze and visualize “real world” data. The test is designed to take about two hours.
You may also use any tools or software on your computer, or that are freely available on the Internet. We prefer that you use simpler tools to more complex ones and that you are “lazy” in the sense of using third party APIs and libraries as much as possible. We encourage the reuse of code when appropriate. If you include code directly in your submission that was written by someone else other than commonly imported modules, please be sure to provide proper attribution, including a URL, text, author, etc. or other available information in the code comments.
Do as much as you can, as well as you can. Prefer efficient, elegant solutions. Prefer scripted analysis to unrepeatable use of GUI tools. For data security and transfer time reasons, you have been given a relatively small data file. Prefer solutions that do not require the full data set to be stored in memory.
There is certainly no requirement that you have previous experience working on these kinds of problems. Rather, we are looking for an ability to research and select the appropriate tools for an open-ended problem and implement something meaningful. We are also interested in your ability to work on a team, which means considering how to package and deliver your results in a way that makes it easy for others to review them. Undocumented code and data dumps are virtually useless; commented code and a clear writeup with elegant visuals are ideal.