CS598CXZ Advanced Topics in Information Retrieval (Fall 2013)

Instructor: ChengXiang Zhai

| Home | Basic Information | Schedule |
| Readings | Assignments | Project | Resources |



Course Project


Introduction

The purpose of the course project is to provide students an opportunity to study and work on a topic in-depth so as to obtain experience with research in IR or application of advanced IR technologies to develop an interesting novel application system. Group projects with a team of up to 4 students are allowed.

General steps

  1. Pick a topic
  2. Read related work
  3. Write a project proposal
  4. Work on the project and submit a progress report
  5. Present the project
  6. Write a report

Grading criteria

Your project will be graded primarily based on the following weighting scheme: The factors to be considered in grading include (1) the overall quality of your project work (including novelty, soundness of approach, potential impact etc); (2) the relevance to the course; (3) the clarity of your project proposal, presentation and report. Refer to the guidelines on the proposal and report for what should be included in a report.

1. Pick a topic

You can either pick from a list of suggested topics (to be posted later), or choose your own topic. In general, a topic can fall into one or more of the following three categories:

When picking a topic, try to ask yourself the following questions:

A good topic is one that is novel, that you like to work on, that people care about, that you have some idea about how to carry out the proposed work.

2. Check related work

This is a very important step! While choosing a topic, you should also check to see whether there has already been work on the topic. If so, you may want to figure out where exactly your novelty is and whether the identified novelty is important. Your goal is to extend , rather than duplicate, the existing work. Novelty is one of the most important criteria for good research work. The instructor can also help you check related work. Please feel free to discuss your plan with the instructor at any time.

3. Write a proposal

You are required to write a one-page proposal before you actually go in depth on a topic. Please post your proposal on the class wiki by a deadline to be announced later.

To post your proposal, go to the Project Proposals page, and add your project to the page. Follow the same format as used by some existing project proposals there. Specifically, you need to create a separate page for your project, where you would put your proposal. It is up to you how to design such a page; it could directly contain your proposal or have your proposal attached to this page.

In the proposal, you should address the following questions and include the names and email addresses of all the team members. (As long as these questions are addressed, the proposal does not have to be very long. A couple of sentences for each question would be sufficient.)

If your project involves testing a hypothesis or original research, your proposal should state explicitly all the following:

If your project is to build a software tool, then your proposal should state explicitly all the following:

Proposal review:After the project proposals are posted, they will be grouped into clusters based on the topic areas. Each of you will be asked to review two other proposals in the same cluster as yours and post your review of each of them as a comment on the corresponding proposal wiki page. In almost all the cases, a cluster has 3 projects, so it just means that you need to review the other two projects in the same cluster. One cluster has 4 projects; for this cluster, please just choose two from the other three projects in the cluster. Please try to ensure that every project be reviewed. That is, try to find a project that has not yet been reviewed to review. The benefit of doing proposal reviews is two-fold: first, it allows you to know more and think more about what your classmates are proposing; second, it would allow you to get feedback about your projects from others. Your project review doesn't have to be long (a couple of short paragraphs would suffice), but should try to touch the following aspects in addition to any other aspects that you want to comment on: (1) novelty of the proposal; (2) quality of the technical approaches; (3) any concerns about the proposed project; (4) any suggestions. Try to understand how exactly the proposed systems/applications/methods are different from existing ones.

To post a review for a project proposal, first click on the link to the proposal from the proposal list page to open the specific proposal wiki page, and then click on "Add Comment" at the bottom of that page to post your review. Please post your reviews by Oct. 8, Tuesday.

4. Work on the project and progress report

Try to follow your proposal, but since research by nature has unpredictable outcome, feel free to adjust your plan and research hypothesis as needed; in such a case, make sure to discuss it with the instructor. Consider documenting your work regularly. This way, you will already have a lot of things written down by the end of the semester.

You will need to submit a progress report in the middle of semester (the exact due date will be posted later) to demonstrate clear progress toward your proposed project. This progress report will be graded mainly based on your effort.

5. Present the course project

At the end of the semester (on Dec. 17, Tuesday, at 1:30-4:30pm, in 1109 Siebel Center), each project group is expected to make an oral presentation of the project. Please submit a PowerPoint presentation file to the course wiki site on the presentation schedule page. If you have a URL for your demo, you may also add a link there to your demo. The purpose of this presentation is: (1) Let you know about others' projects easily. (2) Give you some opportunity to practice presentation skills, particularly how to convey a message effectively and efficiently, which are very important for a successful career. (3) Obtain feedback and suggestions from your peers about your project. There will be two sessions. In each session, one half of the project groups will present their projects and the other half will be the audience. So everyone should show up for both sessions. Please check the presentation schedule to see when you should present your project and when you should be the audience. There will be about 9~10 projects to be presented in each session, so each presentation is expected to last for no more than 8 minutes, including question answering. This means that your actual presentation probably should take just about 6 minutes.

To make efficient use of the very short time period you have for the presentation, I suggest that you do a demo of your system if you have developed one. This is often the best way to explain what you've done to others. If your system is not designed to run in real time (e.g., it may be an offline analysis tool), or if you have not developed a system, you should show some sample results in your presentation. It's up to each project group to decide how to organize the presentation. Given the short time allocated, I suggest that each group choose one person to present on behalf of the whole group, but it's also fine if you decide to have multiple group members to co-present the work (just watch the time so that your whole presentation won't run over the time limit).

Your presentation will be graded mainly based on the clarity of your slides and presentation, and whether you can manage to complete your presentation within the allocated time. You may find it convenient to prepare your presentation based on this template, but you are not required to use it. You can target at making about 6~8 content slides, but no more than 10 slides since it's unlikely you can cover so many slides with just about 6 minutes. While it's desirable to cover all the aspects of your research (i.e., problem statement, motivation, related work, technical approaches, experiment results, conclusion and future work), with only about 6 minutes, you won't be able to do that. So I suggest that you focus on explaining the problem you want to solve and the goal of your project, briefly explain your approach, and show sample results with a demo or screen shots. Think about how you can best present your work so as to make it as easy as possible for your audience to understand your main messages. Pictures, illustrations, and examples are generally better than pure text descriptions. Try to avoid including too much small-font text.

6. Write a project report (due Dec. 19, 2013, Thursday, 11:59pm)

You should write your report as if you were writing a conference paper. You can think of it as an expanded written description of your proposal with more substantial research results and details. Thus the same general guideline for your project proposal applies, i.e., you should (1) explain your problem clearly; (2) provide sufficient motivation for your work and explain how your work is connected with the existing/previous work; (3) explain your methods with sufficient details; (4) discuss the research results; (5) summarize your work, draw conclusions if possible, and discuss how you think the work can be further improved/extended. There is no strict length requirement. If you would like to set a target length, you may target at anywhere between 6 pages (font 10, single column) or 10 pages (font 11, single column) without counting any necessary appendices that you'd like to include. A good report is not just a straightforward description of what you did; it should also include a discussion of the challenges that you had to solve and how you solved them. It is especially important to clearly state the novelty of your work (e.g., a novel research question that wasn't previously answered, or a new method that hasn't been tried for an existing problem, or a novel system function that has never been supported by any existing system), and explan why we should care about this novelty. In general, it is important that you think very clearly about what are the major points you want to make and include arguments and empirical evidence that support your points. For example, whenever you make a choice of some method or design an algorithm in a particular way, always justify it, at least, try to discuss what are the possible choices and explain why you have made this choice. Also, you may want to summarize or plot your experiment results in a particular way to best illustrate your findings. Always keep in mind what exactly you expect your readers to learn from your report, including both positive and negtive findings. As a research report, it is especially important to clarify what new knowledge you have discovered through working on the project. In other words, focus on describing what your readers can learn from reading your report that they would not be able to learn from any other published literature.

Each project team only needs to submit one report. Please make sure to include, for each member of your team, a couple of sentences to describe what he/she did exactly for the project. .

Grading of a project report will be based on three factors: (1) [20%] clarity and completeness of the report itself (i.e., whether you have clearly described what you have done and addressed all the questions that you are supposed to address); (2) [30%] amount of work that you have done; and (3) [50%] the quality of your project as reflected in the importance of problem being addressed, the quality of solution, and the impact of your project. In general, all the members of a team will get the same grade for the project unless the report indicates that some member(s) only superficially participated in the project without doing much actual work; in that case, I will discount the grade.

The project report must be posted in the presentation schedule wiki page at the end of your project proposal entry (beside your presentation slides) by 11:59pm, Dec. 19, Thursday. If you have a URL for your demo, you may also add a link there to your demo.