Homework 9: Final project: Design your own data analysis

So far, you have analyzed data from a variety of sources to solve realistic problems from science, engineering, and business. Now it's your turn to choose and analyze a problem. This is good practice for how you will use Python for the remainder of your career.

There are three parts to this assignment, due separately.
Part I is due on Thursday, March 7 at 11pm.
Part II is due on Friday, March 15 at 9pm.
Part III is due on Monday, March 18 at noon.
For examples, see the reports from Winter 2013.

For this assignment, you are permitted and encouraged to work with a partner; the two of you will submit one solution. You are not required to work with a partner, and groups may not be larger than two people. Only one of you will submit the assignment — do not submit duplicates. If you work with a partner, we will expect your project to be more substantial than a project done individually. If you wish to submit your assignment late, then all project partners must use a late day. If one of the partners does not have any late days left, then the whole group is required to submit the work on time (or take a grade of zero).

Part I: Propose an analysis and locate data

Propose a data analysis project to the course staff. This can be almost anything that you choose. You might select a project from your field of study, your extracurricular interests, government or public policy, or elsewhere. Another good source of ideas is repeating an analysis that your read about in the popular press or in a scientific paper — usually you will do a simplified version of the analysis. We are just looking for you to show that you have absorbed the lessons of CSE 140. Impress us!

In general, your data analysis proposal must clearly state the questions you will seek to answer, the existing dataset you will analyze, and the algorithm you will run on the dataset to answer your questions. More specifically, you should turn in a draft report, in the format required in Part II of this assignment. Your report should include all of the sections required in the final report, except for: the results in section 1 (Summary), all of section 5 (Results), and all of section 6 (Reproducing Your Results). (Later, in Part II, you will add these missing sections, and update other sections in response to feedback from the course staff.) Don't forget to include section 7 (Collaboration and Reflection) in your report, in both Part I and Part II.

You can think of your proposal as a pitch to a venture capitalist, a foundation, or a scientific review panel. Another way to think of it is as creating a new CSE 140 assignment.

Your proposal will probably be about 2-3 pages long, but it is acceptable for the proposal to be longer or shorter as long as it conveys the required information without irrelevant details. Submit your proposal as a PDF file. (In Microsoft Word, you can choose “save as” to save your document as a PDF file. Do not turn in a Word document or plain text.)

Your proposal will be graded based on the quality and clarity of the writing and on whether your proposal conforms to the requirements above. The staff may direct you to make changes to your proposal. For example, the staff may direct you to revise the scope of your proposal (we don't want you to proceed with a project that is too hard or too easy).

Your project will use Python to help you answer your research questions. You might use Python to obtain the data, to clean/format it, and/or to process it (perform computations on it). Any of these is acceptable; in particular, it's OK if the bulk of your code does the work of obtaining and cleaning the data, then the actual data processing is simple. (But some part of the projects must require non-trivial Python processing.)

You will not turn in any code for Part I. However, you might write code. Some reasons are:

• You might write code to help you download your data.
• You might write code to perform data cleaning.
• You might perform an experiment to validate your plans. For instance, maybe you plan to use a particular library. It would be wise to experiment with that library to make sure it has the functionality you need and that you understand how to use it. Such an experiment is called “prototyping”.
• You might want to get a head start on Part II.

Do not neglect Part I of the assignment. If you do not work hard on Part I, you will not be prepared for Part II (not to mention that each part is graded).

Submit your work (Part I)

One group member should submit your report via Catalyst CollectIt (a.k.a. Dropbox).

Finally, answer a survey about how much time you spent on Part I of this assignment. (Each group member should do this part individually.)

Part II: Perform the analysis and report the results

Implement your analysis, process your data, and interpret the results. Then, write a report that describes the results and conclusions of your analysis. It might include graphs, tables of numbers, or just a few key computations. Remember that plots and other visual representations of data are very useful in conveying your conclusions.

Submit your report in PDF (the same as in Part I). Your report will probably be about 4-6 pages of text long, but there are no fixed upper or lower bounds on its size. You should write at an appropriate length: neither so briefly that you omit information, nor so verbosely that you pad your report or bury the important information under irrelevant details.

Your report should contain at least the following parts. You are permitted to write additional sections as well.

0. Title and author(s). The title should describe your research questions. It should not be something like “CSE 140 homework 9” (though you could use that as a subtitle if you wish).
1. Summary of research questions and results. Give a numbered list of one or more research questions. Each one should be a specific question with a specific answer, not merely a general topic or area of investigation. In 1-3 sentences per research question, state what are you trying to compute, and why.
   After each research question, clearly state the answer you determined. Don't give details or justifications yet — just the answer.
2. Motivation and background. Explain the context and why the problem matters. This expands on the research questions that you already stated. Why are they worth computing? What difference would knowing the answers make? We require a problem with some kind of real-world motivation.
3. Dataset. Describe the real, existing dataset that you used, including exact URLs. You may not use a dataset that has been used in a previous CSE 140 assignment or demo. You may not repeat an analysis that you have performed in another class (though you might do something inspired by another class). The data must be real — neither you nor someone else may make up the data.

   You do not have to turn in the dataset itself (it may be quite large, or it might be available only via the web rather than as a single download). Your program might access the data directly via the web. If your program needs to access the data through the file system, then ideally, when your program is run, it should automatically check whether the required data files are present, and if not download them before doing any additional work. Alternately, your report must include simple, clear, unambiguous instructions that anyone can follow to download the data themselves.

   If the data is not currently publicly available (for example, it is data from some activity you are involved in), then you should make it publicly available. One way to do this is to host it on some public website (Dropbox is one possibility); then, your program and your report should reference that location. Do not use a dataset that cannot be shared with the course staff, such as one that contains confidential medical information or intellectual property.

   If the full dataset is too large to download, then provide a subset of it for the course staff to experiment with.

   Do not manually perform any data cleaning steps — all data cleaning should be done programmatically, by Python code that you write.

   Hints about datasets: The best approach is to start with a problem that interests you, and then look for data. Alternately, you can start with a dataset. Here are some possible data sources, but many more exist:

   • Baron Schwartz's list of datasets. Some of these are themselves rich lists of datasets, such as the Amazon AWS public data sets.
   • Data.gov, U.S. open government data
     data.wa.gov, Washington state open government data
   • SQLShare: public scientific datasets. Some require considerable knowledge to interpret, others are easier to understand. You can select “All datasets” and then filter by keyword, or you can select a tag from among those in the left column.
4. Methodology (algorithm or analysis). Write a complete, clear, unambiguous English description of the analysis you performed. This should be sufficient for someone else to write a Python program (or perform manual computations) that reproduces your results, without access to your source code, and without having to guess or make significant design choices. This description is also likely to be helpful to people who read your code later.
   This section explains how your analysis works on an abstract level, focusing on the problem domain and your algorithm. It should be written to be read by a scientist or engineer, not by a programmer. It should should say nothing about specific implementation choices, such as how your code is organized or implemented (such details belong in code comments), what data structures are iterated over, and the like.
   This section should tie your computations to your research questions, indicating exactly what results would lead you to what conclusions.
   You have learned some elementary statistics in CSE 140. One other concept you might find useful is correlation, a measure of whether two variables' values are related, such as when one variable depends on another. Correlation is a quantitative measure of how good a "best fit line" you can draw on a scatterplot. A concrete metric for correlation is the Pearson coefficient r. It is better to import and use SciPy's pearsonr function rather than implementing it yourself. After you have computed r, you will need to interpret it.
5. Results. Present and discuss your research results. Treat each of your research questions separately. Focus in particular on the results that are most interesting, surprising, or important. Discuss the consequences or implications. Interpret the results: if the answers are unexpected, then see whether you can find an explanation for them, such as an external factor that you analysis did not account for.
6. Reproducing your results. Give clear and explicit instructions for obtaining the data and for running the analysis — this is usually a set of command lines that can be typed to the operating system shell. Also explain how to interpret the results or to find the interesting parts in the output.
   These instructions must make it possible for the course staff to re-create, without any additional data entry or interaction, both (a) every number or figure that appears in your report and (b) any other results that support your argument but that you did not include in your report.
7. Collaboration. State which students or other people (besides the course staff and your project partner, if any) helped you with the assignment, or that no one did.
   Reflection. What did you learn from this assignment? What do you wish you had known before you started? What would you do differently? What advice would you offer to future students embarking on this project?
   In Part I, the Reflection part of section 7 should include a breakdown of the remainder of your project (that is, the work you will do for Part II) into at least 3 and no more than 6 parts, and an estimate of the time you will spend on each of the parts. In Part II, state how accurate your estimates were. Why were your estimates good or bad? Did you entirely forget any parts?

Your source code should be well-written and well-commented. It should be clear enough for another programmer, such as the course staff, to understand and modify if needed. Your source code documentation should assume that the programmer has already read your report — you do not need to repeat any of those details, but may wish to use concepts that it introduced. A typical final project might contain around 200 lines of well-structured code without duplicated functionality, though longer or shorter is possible.

It is acceptable for you to scale back, or to expand, the scope of your project if necessary. It's better to do a great job on a subset of your original proposal, than to do a bad job on a larger project. If you have to scale back, then explain why the task was more difficult than you estimated when you wrote your proposal. This will help you to make a better estimate for your next project. It will also convince the course staff that you have done an acceptable amount of work for CSE 140.

Submit your work (Part II)

One group member should submit the following via Catalyst CollectIt (a.k.a. Dropbox):

• Your report
• Your source code

Finally, answer a survey about how much time you spent on Part II of this assignment. (Each group member should do this part individually.)

Part III: Present your results

Prepare a slide deck of no more than 5 slides (including title slide), that can be presented in no more than 3 minutes. The slide deck should summarize the main points of your report, including motivation, research questions, and results. You will only have time to present the most important aspects of your project. This may include your algorithm, but it will probably not include implementation details of your Python code. The slide deck must be a single self-contained file, in PDF (preferred), PowerPoint, or Impress format. We will load all the documents on a single computer and all groups will present using it.

You will present your work to the rest of the class on Monday, March 18. It is strongly recommended that you practice your presentation ahead of time.

Submit your work (Part III)

One group member should submit your slide deck via Catalyst CollectIt (a.k.a. Dropbox):

It is not permitted to submit your slide deck late: you may not use late days, and the deadline is firm.

Finally, answer a survey about how much time you spent on Part III of this assignment. (Each group member should do this part individually.)