Evaluation

Class Participation Tips

The goal of this class is to facilitate in-depth discussions of research issues and opportunities related to self-managing systems. The success of the class and a portion of your final grade depend on your class participation.

In practice, every student will be an instructor with the following responsibilities:

• Read the assigned papers before each class meeting
• Participate in the discussion in every class meeting
• Lead the discussion during one or more class meetings
• Keep notes for one or more class meetings
• Review one final project report

The following sections provide some further advice on each activity related to class participation. These tips have been taken primarily from Christos Kozyrakis's EE392C class at Stanford.

Reading and Summarizing Papers

Our in-class discussions will be of little use to you if you are not prepared for them. Read the required paper(s) in advance. If the specific topic is new to you, drop by during office hours for an informal chat to get some background information on the topic. Also, Shivnath may be able to suggest introductory papers for that topic. If you have extra time and really like a topic, contact Shivnath to learn about papers for further reading.

When reading and summarizing a paper, you should look for the following:

1. The issue or problem addressed by the paper
2. The original contributions (real or claimed, you have to check)
3. Critique: what are the major strengths and weaknesses of the papers? Look carefully at the claims and assumptions, the experimental methodology, the analysis of data, and the presentation style
4. Future work: what are the natural extensions or improvements to this work? Can we apply a similar methodology to other problems of interest?

If you are the discussion leader for a topic, you will need to write down these points in a summary that does not exceed 1 page per paper.

The following techniques may be helpful while studying papers:

• Read the Abstract, Introduction, and Conclusions sections first. This step should give you a good idea of the problem/issue addressed, the basic insight of the authors, and the results achieved.
• Read the paper twice. The first pass should be quick; you should try to get a rough idea of the flow of information, development of the main idea, and analysis of data. Once these seem to make sense, read the paper again and focus on the details.
• Underline/highlight the important parts of the paper. You can take a quick look at these points right before the class.
• Keep notes on the paper margins about comments or questions that come to your mind as you read the paper: important insights, questionable claims or techniques, relevance to other topics, ways to improve some technique, etc. These notes will help you a lot with the class discussion.   
• Look up references that seem to be important or missing. In some cases, you may also want to check who and how references this paper. The CiteSeer and Google Scholar citation engines are great sources of backwards and forward citations (citation + text).

  Participating in the Class Discussion

This class is structured around round-table discussions on advanced research topics. We will all be actively teaching each other. You are expected to come to all the class meetings and be well prepared. Study the required papers for each topic in advance. Your contribution in the discussion will be in the form of comments or questions.

Here are some examples of the kind of participation we are looking for:

• Ask a question on an issue, idea, or technique that is unclear; if you cannot understand something, there is a good chance others don’t understand it either
• Explain an issue that seems unclear at the moment or deserves further discussion
• Propose new ideas on the topic and attempt to develop them into concrete techniques
• Identify important issues that we are missing at the time
• Indicate useful or damaging interactions with other fields; use your experience in other areas of science and engineering for this purpose and keep in mind the discussions from previous class meetings  
• Provide arguments to support or oppose a proposed idea; use qualitative or quantitative arguments to support your claims

In general, you must be active in the classroom. No comment or idea is by default a bad one, so do speak up. However, be polite to your classmates and stick to the point. We want discussions, not competitions.

Leading a Class Discussion

Each student is expected to lead one or more class discussions. The schedule will be arranged as the class progresses.

Before the class meeting:

1. Carefully read the required papers for the class. Discussion leaders are also encouraged to study background and further readings.
2. Prepare a short summary for each required paper. After the class meeting, the summary will be integrated with discussion notes.
3. Prepare a short presentation to initiate the discussion. Your presentation should cover the following:
1. The problem/issue addressed by the presented papers and how it relates to the class theme
2. Why is this problem/issue hard?
3. The summary of contributions/solutions in the paper: ideas, achieved results, conclusions. Avoid details. If the details are interesting, bring them up in the followup discussion
4. 5 major strengths of the paper
5. 5 major weaknesses of the paper

The presentation should not exceed 30 minutes. We want to leave plenty of time for discussion. You can use graphs from the papers or other visual aids. You can use a laptop or write/draw on the blackboard, whichever you are comfortable with.

4. Be prepared to explain some of the detailed issues in the paper. You can assume that your classmates have read the papers, so there is no need to include details in the presentation. However, you should be able to clarify or explain detailed issues, if necessary, during the class discussion. You don’t need to prepare graphs or visual aids for this purpose. 

During the class meeting:

1. Give your introductory presentation
2. Lead the discussion. You can do this by preparing a set of interesting questions. Here are some examples:
• Does someone in the audience want to support/rebut the strengths/weaknesses you found in the paper?
• Are the assumptions in the paper realistic?
• Are there better ways to solve the same problem?
• What are the implications of the results/conclusions of the paper for self-managing systems? What is the take-away message?  
• What is the future work on this topic (unanswered questions in the paper)? Are there techniques that we can improve on? How?
• How do the ideas on this topic interact with the issues discussed in the previous papers and class meetings?

Remember, your goal is to lead the discussion, not to dominate it. Ask your questions or use your influence when the discussion is stuck, or when you believe we are missing a very important issue. Don’t expect the discussion to always go as you planned. We will hopefully generate some unexpected yet interesting ideas.

3. In some cases, I may intervene to steer the discussion towards a specific direction or make sure that we don’t miss an important point. Don’t consider this a failure on your part.

After the class meeting:

Meet with the student who kept notes during the discussion. Help her merge your paper summaries and presentation material with her notes to generate a single PDF document. Send the document within one week to Shivnath who will post it on the class web-site.

Feel free to drop by during office hours for any type of help with the presentation.

Keeping Discussion Notes

In each class one student will be assigned the responsibility of taking notes during the discussion. We will work out the "scribing schedule" as the class progresses. The notes must capture the essence of the discussion: significant ideas, major arguments (pros/cons), unresolved issues, conclusions, etc. You don’t have to rewrite the papers in your notes. The discussion leader will provide summaries which you can merge into your notes. However, you should keep track of the comments (for our discussion) that target each paper.

After the class discussion, you should meet with the discussion leader to produce the final discussion document. It should include the summaries of the required papers (up to 1 page per paper), notes from the presentation material used in class, and your notes from the class discussion. Use the following structure for the document (more or less):

1. Short introduction to the topic
2. Paper #1

Summary

Comments from our discussion

3. Paper #2

Summary

Comments from our discussion

4. Notes on other issues we discussed
5. Conclusions & open issues

You should send Shivnath an email with the notes as a PDF attachment within 1 week from the class meeting. If you want to use Latex for document preparation, start with file lect00.tex and add your notes to it. You will also need file handout.cls to compile your notes.

Reviewing Project Papers

As active members of the research community, you will often be asked to review papers submitted for publication at conferences or journals. In this class, you will get some practice with peer reviews. Each student will review another student's or group's final project report.

The reviews will be anonymous: the project groups will not be told who their reviewers were.  Your goal as a reviewer is to provide an impartial review and constructive criticism. Hence, you need to make a fair assessment of the strengths and weaknesses of the paper and provide suggestions for improvement. Avoid harsh comments in the review (remember someone else is reviewing your report at the same time). Fair and constructive are the keywords to remember here.  

I strongly suggest that you read the paper “ The Task of a Referee ” by A.J. Smith (IEEE Computer, 1990).

Use this ASCII form for your review.