Edited by: William Newman
with contributions from Kate Ehrlich, Robin Jeffries, Jock Mackinlay, William Newman and John Tang

Introduction

Considerable effort can go into preparing a paper for the CHI conference. Nobody likes to see this effort go to waste, especially if it involves a potentially valuable contribution that could bring credit to authors and fresh insights to readers. However, CHI is very popular as a place to present work, and this inevitably means that a large proportion of submitted papers – typically 80 percent – get rejected. For authors such as you, this means taking special care to submit your work in a manner that guarantees the best chance of gaining acceptance. This Guide has been prepared to help you.

Like most high-profile conferences, CHI relies on a panel of reviewers to assist the selection of papers. Your paper will be subjected to thorough reviewing by five or more experts in the relevant field. The process by which these reviewers are selected, and by which their assessments then influence the final decision, has been reworked and refined over the years in order to ensure that papers are treated as fairly as possible. Reviewers receive guidance too on how to prepare their reviews. Particular care is made to ensure that the instructions issued to them are consistent with the instructions to authors in the Call for Papers and in this Guide.

A major step towards guaranteeing authors fair treatment was made prior to the CHI 99 conference. That year and ever since, reviewers have been asked to assess submitted papers in terms of the contribution made to HCI, and the benefit this gained by the target audience. This year some minor changes have been introduced to the review process, to ensure that papers offering strong contributions are indeed more likely to be accepted than those that do not.

The Review Form

We start this guide with a brief overview of the on-line form used by reviewers to submit their reviews of your paper. You may have been asked to review papers for CHI 2003, in which case you will need to become very familiar with this form! That's not why we describe it here, however. Rather, we suggest that familiarity with the questions on the review form will help you to decide what to include or emphasize in your paper.

The questions on the review form (reproduced in full in Note 3 at the end of this Guide) ask the reviewer for the following:

1. A rating of the reviewer's own expertise in the topic area of the paper, from 4 (high) to 1 (low); this helps members of the Papers Committee to resolve conflicting views on papers.
2. Statement of the paper's contribution to the field of HCI; this provides the Papers Committee with a basis for assessing the significance of the contribution, and for judging whether all the reviewers agree on what the contribution is.
3. A review of the paper in terms of the criteria laid out in the Call for papers; more on this below.
4. Any aspects of the paper's written presentation that need improvement.
5. A rating of the paper's acceptability, from 5 (high) down to 1 (low). Papers whose contribution is judged significant are rated 4 or 5, depending on whether or not they contain any non-serious flaws.

Reviewers can also add further comments that they want you or the Papers Committee to see. But for you, the main concerns to keep in mind are those numbered 2, 3 and 4: you need to offer the reviewers a strong, well-presented contribution to HCI that meets the criteria in the Call. If you do this, your paper should get a high rating.

Making a significant contribution

Your paper's reviewers will be asked to focus on the significance of the paper's contribution, the benefit others can gain from its results, the validity of the work, and its originality. Of these, the contribution and its significance are the most important. CHI papers are expected to offer contributions that clearly and significantly advance the field of HCI. Your paper should offer CHI attendees, and readers of the proceedings, something that adds to what they could have learned from existing publications.

HCI is a very broad field, and it opens up a correspondingly broad range of possible contributions. The complete list contribution types is given in Note 1 at the end of this Guide and in the Papers section of the CHI 2003 Call for Participation. Thus your paper's contribution may, as the Call points out, be a design briefing, a new design for an interactive technology or system, a new method or tool, an analysis of an HCI issue, results from fieldwork and ethnography, results from laboratory studies, or a theory or model. This list is not exhaustive, but it encompasses the vast majority of past CHI paper contributions.

CHI asks for just one such contribution. You may be tempted to offer more than one, e.g., a new design and a new method by which you evaluated it. You are advised to resist this temptation, and focus on presenting one strong contribution really well. If you offer several contributions of different types, reviewers may be confused as to which is the paper's primary focus. Furthermore you will probably find it difficult to do justice to more than one contribution in eight pages. Should you find you have several strong contributions to offer – and you have time to spare – consider submitting them as separate papers.

Offering benefit to the reader

If you are in a position to make a contribution to HCI, there will be people who stand to benefit from it. For example, if you are presenting a new interaction technique for small screens, it will be of potential use to people involved in developing handheld computers. A case study describing how you developed a new interactive product will probably catch the eye of HCI educators looking for teaching materials. The benefit thus to be gained from your paper will be one consideration that reviewers take into account.

As you write your paper, therefore, keep in mind the kinds of people you think might benefit from reading it. Think also about how this might happen – what kinds of problems might readers be facing, to which your paper could provide the solution. Try to make sure that the paper explains the contribution in sufficient detail for the full benefit to be extracted.

This is an appropriate point in our guide to mention an item you will need to provide when you submit your paper: a statement of contribution and benefit, in 30 words or less. This statement is not part of your paper, and is not seen by reviewers. If the paper is accepted, however, a statement of contribution and benefit will be included in its entry in the proceedings' Table of Contents. It will be based on the statement you provide with your submission. We suggest it may be useful to draft this statement before you begin writing your paper, to help keep the contribution and benefit in sharp focus. Hints on writing the statement will be found in Note 2 at the end of this Guide.

Ensuring results are valid

The validity of your paper's contribution needs to be adequately supported by appropriate arguments, analyses, evaluations and/or data. Otherwise readers will find it hard to judge whether they can confidently take up your ideas, and thus gain the benefits you are claiming to offer. Reviewers are therefore asked to assess the validity of the results you are presenting.

Demonstrating validity is one of the most troublesome aspects of writing CHI papers. Reviewers often cite problems with validity, rather than with the contribution per se, as the reason to reject a paper. For this reason it is risky to leave validity issues (e.g., evaluating a design) until the last minute. Instead you should consider, when planning the work and certainly before embarking on the paper, how you will demonstrate your contribution's validity. There are many ways to do this; they mostly fall under two headings:

• showing that you used sufficiently sound methods to arrive at your contribution;
• conducting an evaluation of your contribution that provides sufficient confirmation of your claims about it.

Your choice of how to demonstrate validity will depend on what kind of contribution you're offering. Is it some kind of empirical result – a finding or some guidelines, for example, drawn from analyzing experimental data? If so, you will almost certainly want to demonstrate "sound methods" validity, by showing that the experiment was sufficiently well designed, and the data sufficiently carefully analyzed using appropriate methods, to back up the results you're presenting. Alternatively, does your contribution take the form of a design for a new interface, interaction technique or design tool? If so, you will probably want to demonstrate "evaluation" validity, by subjecting your design to tests that demonstrate its effectiveness.

You must also decide how far to go towards demonstrating validity. How rigorous and painstaking should you be? Of course it usually pays to err on the side of thoroughness, but this can lead you into short-changing other aspects of the work, because all your time is going into gathering extensive data under carefully controlled conditions and subjecting it to thorough analysis. Where should you draw the line?

A guiding principle here is to consider the benefits that lie in your contribution, and confirm to your own satisfaction, and your colleagues', that these benefits are really there. Again, the steps you take depend on the contribution. For example, if you have developed a new type of menu, capable of reducing errors in selecting menu items, you will probably run a careful experiment to measure error rates with this and other types of menu. If you have come up with an innovative system to support collaborative writing, you may try to evaluate it in real-world conditions, offering it to a group of co-authors for use in a joint writing task, and conducting studies to determine how the system helps them – and what problems it introduces. If you have developed a new system design methodology, you may find it hard to compare with other methodologies, but you will probably want to report on your experiences in using it. Any evaluations like these, conducted to convince yourself that you've got something of benefit to the HCI community, can appropriately be adopted and extended to convince your reviewers and readers.

Bear in mind that reviewers of papers often mention issues, of an obvious or important nature, that have not been addressed by the authors. They often criticize authors for conducting studies without adequate theoretical basis, or for not providing enough evidence or sound reasoning for claims. A further concern is lack of justification for design choices and not explaining why certain design features have been included. In summary, you should explain not only what you did, but also why you did it, so that readers (including reviewers) can be convinced that you made appropriate choices. Explaining your choices can also stimulate more research by helping others see alternative approaches.

Finally, remember that where you have collected data for analysis, appropriate methods should have been selected and correctly applied to support the work. You should provide sufficient data and/or well-supported arguments, explain what analyses were made and why, cite relevant work, and cover the important issues at the appropriate level of detail.

Gaining credit for originality

Originality in your paper will help it get accepted in two ways. First, it is not just helpful but essential that paper's contribution be original, going beyond any work already reported in other journals or conference proceedings. Second, reviewers will often give credit for original approaches adopted in conducting the work, particularly if these contributed strongly to the work's success.

To demonstrate the originality of your contribution you should make sure to cite prior work (including your own) in the relevant area. If possible, explain the limitations in this work that your contribution has overcome. Make sure also to cite publications that have had a major influence on your own work. Lack of references to prior work is a frequent cause for complaint – and low rating – by reviewers. Note that reviewers are being asked, this year, to set the context for their review by identifying relevant past work; you can help them do this. You can also make it easier for them to check your references by concentrating on papers in easy-to-find publications. Allow adequate time for this part of your paper's preparation.

As regards originality in conducting your work, remember that acceptance of your paper doesn't depend on this. If the paper's contribution is a strong one, it should gain acceptance however you arrived at it. However, reviewers do appreciate novelty and elegance in conducting the work, particularly if they can see how it simplified the work, or could help others conduct work of a similar kind, or both. Thus they will probably give credit for an original way of collecting data during a study, or of choosing a means of evaluating a design, or of overcoming a weakness found in a new design. A few examples of such originality in your work will probably strengthen your paper; however, a plethora of them could drown out the central contribution – so go easy!

Describing the work clearly and concisely

You would be surprised at the number of reviewer complaints about written presentation. Describing your work involves not only writing good prose, but also providing a good structure that helps the reader follow the explanation. The text should be supported with figures, tables and even videos where appropriate; these should be clear and easy to understand. In summary, try to write clearly and concisely, avoid jargon, organize the paper to flow logically and smoothly, provide the right level of detail, and make good use of figures to support the text.

Although all presentations at CHI are made in English, CHI is a conference with an international audience – and an international panel of reviewers. Papers need to be written in a language that effectively communicates across national and cultural boundaries. When authors are not native speakers of English, reviewers try to assess the quality of the work independent of language issues, but good English always helps. If you are not a native English speaker but have access to those who are, it is a good idea to ask them to proof-read your paper before you submit it.

Even if your first language is English, keep in mind that non-native English speakers will be reading and reviewing the paper. Avoid long, complex sentences as well as regional colloquialisms, jokes or puns that could be difficult for someone outside your culture to understand.

A couple of final and important points: first, CHI papers are reviewed on an 'as-is' basis, and cannot be accepted conditionally upon making changes. This is unavoidable given the tight schedule of the reviewing process: there is no time for a second review after the author has made changes, so reviewers must make a decision whether the submission in its current form is acceptable for CHI. Note, however, that reviewers often do make suggestions for further improvement, and authors of accepted papers are encouraged to make minor revisions to their work before a final draft is required.

Also, when writing your paper you should resist the temptation to describe future work, or work expected to be completed before the final submission or conference. Although these planned activities are often interesting, you cannot rely on them to get your paper accepted. On the contrary, they may be seen by reviewers as evidence that the submission is premature, and you may be advised to resubmit when more of the work has been completed.

Summary

With its large number of submissions, CHI's review process is bound to be highly competitive. The intent of the review process is to provide the conference with a program of papers offering significant contributions of high potential benefit to attendees and readers. Writing such a paper for CHI is a lot of work, but it is rewarded with the visibility and influence that only high-profile publications like the CHI Letters series (in which CHI Proceedings are included) can offer. We hope this document has helped give you some clear and concrete guidance on how to write a successful CHI submission. If not, feel free to send feedback and questions to chi2003-papers@acm.org.

Best wishes, and we look forward to seeing a good collection of successful submissions this year.

Further information:

1. The types of contribution your paper can offer

   Review criteria are explained in the Papers section of the Call for Participation. They hinge on the contribution that the paper offers. This contribution should be made clear in the abstract as well as the paper, outlining its direct significance to the field of HCI. The contribution you present may be one of the following:
   1. Design Briefings. Accounts of the design (rationale, process, outcomes and evaluation) of an innovative application or system.
   2. Interaction Technologies. A new technique, device, or other component of the user interface.
   3. Interactive Systems: Descriptions of the architecture, interface and evaluation of a new interactive system.
   4. Methodologies and Tools. New methods, processes, techniques, and tools for use in interactive system design, development and deployment.
   5. Reflective Analyses: Thought-provoking, well substantiated analyses of HCI issues.
   6. Results from Fieldwork and Ethnography. Findings, guidelines, etc. from studies of real world settings, or of technology use in such settings, with clear relevance to the design and deployment of interactive systems.
   7. Results from Laboratory Studies: Findings, techniques, methods, etc. from controlled studies of systems, techniques, and other phenomena relevant to HCI.
   8. Theories and Models: Descriptions and evaluation of HCI theories, models and other formal approaches.

   As explained in the Guide, you should focus primarily on one kind of contribution. Papers making one clear, significant contribution are more likely to be accepted than papers making several lesser contributions.

2. Questions on the review form

   This year reviewers will be asked to answer the following questions about each paper they review:
   1. Rate your expertise in the topic area of this paper:
      4. Expert: First-hand experience of the topic and thorough knowledge of literature
      3. Near-expert: Work on a related topic and generally keep up with the literature
      2. General interest: Have read some of literature on this topic
      1. Outsider: Have little or no background in this area
   2. Briefly state in your own words the paper's contribution, if any, to the field of HCI.
   3. Write your review of the paper here.
      Please identify relevant past work in this area and, with this as context, state the:
      • significance of the paper's contribution to HCI – what new insights does the paper offer?
      • benefit that others can gain from its results, and why do they matter?
      • validity of the work presented – how confidently can researchers and practitioners take up the results?
      • originality: have new ideas or approaches been employed where needed?
   4. Identify aspects of the paper's written presentation that need to be improved prior to publication:
      • any statements/passages that could be expressed more clearly and concisely
      • any redundant figures, or additional ones needed
      • any missing or incomplete references
      • any aspects that could make the paper more understandable to an international readership
   5. Provide a rating of the paper's acceptability:
      5. Definitely Accept – significant contribution and benefits
      4. Probably Accept – significant contribution/benefits, some non-serious flaws
      3. Borderline – question marks over significance and flaws
      2. Probably Reject – serious problems identified with significance and flaws
      1. Definitely Reject – no identifiable contribution or benefits
   6. Additional comments that you want forwarded to the author(s).
   7. Additional comments that you do not want forwarded to the author(s).

Writing a Statement of Contribution and Benefit:

Short statements of contribution and benefit will be included in the Table of Contents of the CHI 2003 Proceedings, as they were for CHI 2002, with a view to emphasizing CHI's commitment to publishing papers that make strong contributions to HCI. The specific gains envisaged from including these statements are:

• For the CHI Proceedings reader, they can quickly give an overall idea of what will be got out of reading a paper.
• For those searching online, they can make it easier to find papers of interest.
• For prospective CHI authors, they can explain and reaffirm the Call for Papers' message about the importance of contribution and benefit.
• For future CHI reviewers, they can clarify what is meant by contribution and benefit, and can help appreciate what the authors have achieved.
• For CHI organizers, they can offer a basis for assessing the content and quality of the papers program.

At CHI 2002, three out of four users of the Proceedings reported finding the statements useful. But in order to serve the above purposes while staying within space constraints, statements need to be crafted with some care. The rest of this note offers some advice on how to do this.

The structure of the statement

A contribution/benefit statement describes the contribution made by the paper to HCI and the benefit that readers can gain from it. These are stated in two sentences, as shown in the following CHI 2002 examples:

1. Describes an adaptive technique for improving focus-targeting in distortion-based visualizations, that flattens the view based on pointer speed. The technique can significantly reduce targeting times and targeting errors (p. 267).
2. Describes a sound-enhanced system based on Instant Messaging, supporting presence awareness and opportunistic interactions among mobile, distributed groups. Can help mobile people stay connected in a lightweight, enjoyable way (p. 179).
3. Presents guidelines for designers of learner-centered tools based on a case study of a scaffolded software environment. Can assist in developing effective scaffolded tools (p. 81).
4. Presents findings concerning males' outperformance of females in 3D navigation tasks. Explains how large displays and wide fields of view can be employed to improve females' performance dramatically (p. 195).
5. Extends the Cognitive Walkthrough method to group situations by treating individual and collaborative tasks separately. Presents practitioners with a clear method for evaluating groupware usability (p. 455).
6. Case study describing development of a physical environment that allows young children to program stories. Can assist designers in understanding how to involve users, especially in formative design stages (p. 299).

For the purposes of this guide, the two parts of the complete statement are called the contribution statement and the benefit statement. The next two sections offer advice on writing each part. For CHI 2003, the two parts must amount to 30 words or less.

Stating the contribution

Contribution statements should specify what is being contributed to HCI, in a manner that not only identifies the general nature of the contribution (the 'generic' contribution), but also explains how the contribution is unique. It may be easiest to start by describing, in a dozen words or less, the generic contribution, e.g.:

1. an adaptive technique for improving focus-targeting
2. a sound-enhanced system based on Instant Messaging
3. guidelines for designers of learner-centered tools
4. findings concerning males' outperformance of females in 3D navigation tasks
5. an extended Cognitive Walkthrough method for group situations
6. a case study describing development of a physical environment

Then add a few words that make it clear how it's unique ­ a technique that flattens the view based on pointer speed, a sound-enhanced system supporting presence awareness and opportunistic interactions, guidelines based on a case study of a scaffolded software environment, etc.

Note also that the initial few words should identify the type of contribution, as in our examples: (a) [interaction] technique, (b) system, (c) guidelines, (d) findings, (e) method and (f) case study.

Identifying the benefit

The second part of the statement identifies the benefit to the reader. The contribution may, of course, offer a number of potential benefits. However, it's most unlikely that more than one can be accommodated within the overall 30-word limit. If it's really important to indicate that there are multiple benefits, start the 'benefits' section with wording such as "Benefits include..."

In thinking about the nature of the benefit, bear in mind that it's very likely that an improvement of some sort can be generated by the contribution, e.g.:

1. targeting times and targeting errors are reduced
2. mobile people are helped to stay connected
3. developing effective scaffolded tools is assisted
4. females' performance of navigation tasks is improved
5. practitioners are given a clear groupware evaluation method
6. designers are assisted in involving users

If this improvement can be identified, then writing the benefit statement is largely a matter of finding a few words that express it clearly. If no such improvement can be expressed, then look for a beneficial outcome of some other kind, as in the following example:

Presents a system for creating non-planar 3D curves of automotive designs using 2D tape drawing. Provides new insights on inputting geometry on large-scale displays.

After paper acceptance

If your paper is accepted, your contribution/benefit statement will appear in the CHI 2003 Proceedings' Table of Contents. By this time it will have been seen by your paper's reviewers and associate chair, and they may have made recommendations on how to improve its wording. You will receive back your revised statement shortly after receiving news of your paper's acceptance. Before the statement is finalized you will have an opportunity to discuss changes with a member of the Papers Committee.

In summary

Go about writing the statement as follows:

1. Describe the 'generic' contribution
2. Add words that express what makes the contribution unique
3. If necessary, add words at the start to specify the type of contribution
4. Identify an improvement that the contribution can bring about, or some beneficial outcome
5. Express the benefit in terms of this improvement or outcome.

All that remains is then to pare the statement down to 30 words or less. If your paper is accepted, work with the Papers Committee on the final wording.

Created August 6, 1998
Last updated June 28, 2002