1
Joint Standpoint of Art Universities on the draft Methodology for
Evaluating Research Organisations in the Universities Sector
We highly appreciate the idea of the Methodology for Evaluating Research
Organisations which will replace the current “coffee grinder” assessment of the
quantity of outcomes with the evaluation of the quality of an entire research
organization, its results and its internal scholarly environment. This undoubtedly is
a significant improvement which may have a considerable benefit for the
development of the Czech research environment and enhancement of its quality.
At the same time, however, we have to point out that the presented proposal of the
Methodology for Evaluating Research Organisations in the Universities Sector may
be functional and beneficial for large research universities, but for art universities it
is usable to a limited extent only and, in addition, there is the risk that its
consequences will be quite the opposite than those intended: instead of giving
motivation for improvement of quality it will lead to the destruction of the existing
functioning models.
Characteristic of art universities is the fact that artistic creative work (reported and
assessed in the Registry of Artistic Results – RUV) is the primary creative domain
for them and research is a supplementary and minor activity, but no less significant
or relevant. In our opinion it is very much needed and essential for the development
of our disciplines that we also develop, in addition to the actual artistic creative
work, its critical reflection and analysis, which is often performed by the artists
themselves who are able to reflect art in a unique manner, taking into account their
experience with their own art production as well as art in general. It is in this area,
in the domain of artistic research, where we see a great potential for our schools,
which fact is also confirmed by our colleagues from abroad where this field has
been developed on a much larger scale and where its contribution to the
development of knowledge has already been proven.
Another specific feature of research at art schools is that it usually does not take
the form of large research centres and teams but is carried out by individual
scholars, it does not require almost any special technical equipment and the fact
2
that it often focuses on national topic cannot be seen as a shortcoming because by
reflecting the Czech arts and culture it contributes to their development and brings
Czech art to the world.
However, the proposed evaluation criteria and their rating clearly show that these
specific research workplaces of art academies, which have had a long tradition in
this country and which have always been able to perform highly and deliver quality
(see the numbers of our successful projects with the Czech Science Foundation, the
Technology Agency of the Czech Republic and in the NAKI project, the numbers of
publications in proportion to the size of institutional support and the number of
awards given to our outputs by the Czech as well as foreign professional
community), will not be able to prove their quality in the proposed system of
evaluation because instead of respecting their specifics, they will be evaluated on
the basis of their ability to fulfil criteria that for them are irrelevant or even totally
unusable, and only applicable to the different environment of large research
universities where disciplines other than humanities prevail.
This fact is most visible in the proposed criteria of the module M4 (table 4, p. 30)
particularly in relation to the criteria 4.4 (Strategy for the establishing, financing and
long-term development and sustainability of research centres and large research
infrastructures), 4.5 (Training system for intellectual property protection and
technology transfer), 4.22 (Conditions for setting up new teams and introducing new
research topics (start-up strategy)), 4.24 (System for acquiring and renewing
instruments and equipment for R&D&I) or 4.25 (System for sharing instruments and
equipment for R&D&I) which can only be evaluated on the scale from
“Unsatisfactory” to “Excellent” with great difficulty in our schools because those
activities do not exist at our schools and considering the specifics of our research
their existence is not even desirable, or take place on a scale which is entirely
sufficient for our disciplines, regardless of how trivial this may seem in comparison
to large research centres.
For instance, if we state that we do not find it desirable in our fields to set up large
research centres and research infrastructure, will that be evaluated as
"Unsatisfactory“ or “Excellent”, or if we write that the system of renewing
3
instruments and equipment for R&D&I in our situation means the updating of
standard computer equipment only, will that be evaluated as “Unsatisfactory” or
“Excellent”? The five above-mentioned criteria can in total make up as much as 25
points and in the event that the evaluator concludes that our fulfilment of these
criteria is “Unsatisfactory”, we would lose almost one fifth of the achievable score
on the basis of what for us is totally insignificant or inapplicable criteria. In
consequence of that we would lose confidence in the fairness of the defined rules
and instead of being motivated to work on a further development and improvement
of the quality of our research we would be forced to resign on the development of
our uniqueness. Actually in this point, the research activities at our schools are
identical to those of the majority of other humanities where these criteria are also
totally irrelevant.
These criteria would only be relevant for us if we were able to take into account the
interconnection between artistic creative work and research: for us the large
research infrastructure is our art laboratories (theatres and concert halls, film and
art studios, record studios, etc.) where works of art are created in the first place but
where research projects are performed as well; for our creative activities we also
necessarily need highly demanding, in terms of technology and finance, devices and
tools (such as camera, lighting and sound systems, as well as musical instruments
or post production devices) which, however, are primarily acquired from the
contributions or subsidies for the artistic and educational area, not in connection
with research. But the proposed methodology forces us to view the wide and highly
diverse spectrum of our creative activities through a very reduced perspective of
narrowly defined research activities.
Therefore we suggest that the criteria for the module M4 should respect the
specifics of our field (just like in the module M3) and that for artistic disciplines (or
possibly also for the entire FORD Humanities and the Arts) these criteria are ruled
out as “Not applicable” and that the overall evaluation or, more precisely, the rating
for the fields is altered accordingly. Another option is that art academies are
allowed for self-evaluation purposes to reflect all of their creative activities in their
complexity and reciprocal synergies, not only the activities strictly classifying as
4
research and without considering their artistic and art research context, which
domains are inextricably linked together at our schools.
We appreciate that the specifics of the individual scientific areas (indicative
relevance) have already been applied in the evaluation of the module M3,
nevertheless we consider the proposed rating in this category insufficient as well.
Especially the criteria 3.3 (Contract research), 3.5 (Applied research results with an
economic impact on society), 3.8 (System and support for technology transfer and
intellectual property protection) and 3.9 (Strategy for setting up and supporting
spin-off firms or other forms of commercialising R&D&I results) are not only
“Marginally relevant” for us but in fact “Not applicable” because they too are
unfeasible or contrary to the specifics of our fields. Thus in the proposed tabular
rating of humanities certain schools cannot achieve ranking that is better than
“Good” even though in all the relevant criteria they would have earned full scores. In
addition, the outputs of applied research in humanities or art disciplines are totally
different from natural sciences or technical disciplines (for example in artistic
environment an art exhibition or a theatre performance with the participation of
disabled persons) and the proposed rating is absolutely not suitable for them and
therefore this rating is also totally not applicable to this segment. We also find
absolutely inappropriate the decision that this segment will be evaluated strictly by
faculties or other smaller units, which is going to significantly disadvantage smaller
research teams that operate at our small schools.
We also have reservations as to the excessive complexity and demands (in terms of
time and funds) of the proposed evaluation methodology which are totally
disproportionate to the size of not only our institutions but any smaller research
organizations in general. For example, in some of our schools/faculties there will
have to be more members of the International Evaluation Panel than the research
personnel actually employed. The very preparation of the self-evaluation report and
the evaluation process itself will require a non-negligible (compared to the total
amount) amount of our time and financial capacities—which could otherwise be
expended on actual research. We would be willing to accept all this if only the
results of the evaluation had a significant benefit for our schools and contributed to
5
their further development. However, we are afraid that the information value of the
results for our schools will be absolutely marginal because, as has already been
mentioned, the results will totally disregard the interconnectedness and the mutual
synergies of our creative, that is, primarily artistic and research, activities. The
evaluation of research activities only, without taking into account our artistic
production and the interconnection of these domains, will always be limiting and
almost useless for you.
We understand the efforts of the Research, Development and Innovations Council
and the Ministry of Education, Youth and Sports to enhance the quality of Czech
science and support its development, nevertheless we would strongly welcome if all
the forms of Czech research environment were taken into account and if the efforts
to find a unified and universally valid methodology were moderated—if the specifics
of art disciplines and humanities were given more respect.
Although research of art and art research cannot bring about direct economic
effects and do not allow for the creation of large research infrastructure of
international importance, we still are convinced that it is in the interest of the Czech
state and the Czech society to ensure their further support and enable their further
development. The objective of our artistic creative work and the research activities
thereto related is to enrich the Czech (as well as world) culture and contribute to
the cultivation and development of our society and to the enhancement of the lives
of all people—we are convinced that the quality of our performance in fulfilling this
objective is already very high. We will strongly appreciate when the resultant
Methodology for Evaluating Research Organisations in the Universities Segment
gives us the opportunity to continue our efforts.
doc. Mgr. Jan HANČIL, Rector of the Academy of Performing Arts in Prague
prof. Mgr. Petr OSLZLÝ, Rector of Janáček Academy of Music and Performing Arts in
Brno
doc. MgA. Tomáš VANĚK, Rector of the Academy of Fine Arts in Prague
6
prof. PhDr. PaedDr. Jindřich VYBÍRAL, CSc., Rector of the Academy of Arts,
Architecture and Design in Prague
In Brno and Prague on 29 May 2019