The central concern of a research university is the fund of human knowledge: its creation, transmission, and eventual application toward the progress of civilization.

Accordingly, faculty are judged on their contributions toward this central concern and toward the functioning of the university. This leads to 4 dimensions of evaluation:

1. contribution to the fund of human knowledge;
2. contribution to the transmission of knowledge (mainly to students but also to others);
3. contribution to the eventual application of knowledge toward the progress of civilization;
4. service to the university and, secondarily, to other institutions having missions complementary to the university's (scholarly societies, journals, research institutes and agencies, etc.).

The evaluation of research can pose difficult questions as to what kinds of activities should qualify as "research", a term that I take to mean "worthy contribution to the fund of human knowledge". Is a sample survey with no interpretation and analysis "research"? Is the suggestion of a not-yet-validated hypothesis "research"? Is an essay expressing an inspired qualitative insight "research"? It is tempting to say No, but one must allow that the data, hypothesis, or insight might ultimately prove to be of the greatest importance.

What confuses the issue is that there are several kinds of activities which may not be bona fide "contributions" per se to the fund of human knowledge, but rather are among the generative processes associated with the creation of such contributions. Contributions do not appear spontaneously out of nowhere; they emerge from the turbulent world of intellectual processes. Examples of such processes are:

• concept formation, the process by which one learns to classify specific experiences or instances in useful ways (based on their attributes);
• cueing, the process by which relationships are established between current problems (or other matters of concern) and the available fund of past experiences, information, and problem-solving approaches;
• question formation (good questions direct attention to promising avenues of inquiry)
• exploratory data analysis;
• hypothesis and conjecture formation.

The results of such processes are frequently written down as discussion or working papers, research proposals, journalistic commentaries, essays, reviews, and the like. Their ultimate value to anyone other than the author is often in doubt. Nevertheless, a person who is insightful and creative in these respects is showing the smoke likely to be accompanied sooner or later by the fire of scholarly contribution.

There are many different types of research activities. One enumeration of types appears below. It is arranged in a rough intellectual pecking order, from lowest to highest, based on the common view that contributions of greater generality or domain should rank higher than those of lesser generality or domain. This order is not the same as one would derive from considerations of chronological sequence.

1. A new and reasonably reliable but essentially ad hoc description of reality that is instructive in some way -- e.g., a bit of evidence for or against some hypothesis or conventional wisdom, perhaps even a clear instance of agreement or denial; data worthy of banking for some possible future use; evidence suggestive of new hypotheses
2. Measurement and data collection or routine descriptive statistical analysis carried out according to generally accepted scholarly methods -- quantitative rather than qualitative, all available evidence reported whether unfavorable or favorable, measurement error assessed, etc., as in a sample survey or controlled experiment
3. Generative processes of the type described previously
4. Hypothesis evaluation by statistical analysis, theoretical analysis, or reasoned argument
5. Develop a technical, provably valid tool, technique, or method to facilitate the performance of some class of tasks; methodological studies
6. Establish a reasonably credible theory elucidating some aspect of reality; contributing to such a theory

Activities related to the transmission of knowledge can be of two main types:

1. the development of instructive materials (textbooks, tutorial papers, videotaped lectures, etc.);
2. direct interaction with others (classroom teaching and working with thesis students).

It is easier to evaluate activities of the first type, which are tangible, than those of the second type, which are not. The dilemma faced in trying to evaluate the second type of activities is that the only evidence commonly available relates to "consumer satisfaction" rather than to "consumer benefit".
 
Professional Activity

It is not expected that every university faculty member will necessarily participate personally in the practical application of knowledge. However, it is not unreasonable to expect some effort at least to facilitate such applications. A professional school would likely expect even more.

Typical professional activities include:

1. authoring plain-language materials aimed at improving the standards of professional practice (e.g., articles in the non-academic press);
2. participating in meetings, short courses, and seminars catering to non-academic professionals;
3. consulting engagements.

There is work to be done to maintain and improve the university and kindred institutions. This is not to be taken lightly, as these institutions provide the host environments within which individuals and groups of individuals nurture the fund of human knowledge.

At least a modest amount of conscientious service is expected of every faculty member. Beyond verifying that, the principal challenge to evaluating service is to discern those services that genuinely contribute to an institution's "ends" rather than merely to its bureaucracy.