New Indicators for Crown Research Institutes

From 1 July 2006, a new set of indicators will be used to measure the performance of the CRIs in applying their research results and technologies.  These activities are in line with section 5(1) of the Crown Research Institutes Act 1992 which states that the CRIs should promote and facilitate the application of the results of their research and technological developments.

To date, the CRIs have been reporting against a set of indicators developed in 1998.  Over time, however, the CRIs’ activities in applying their research results have become more sophisticated.  They range from traditional forms of transfer such as scientific papers and presentations to licensing agreements, joint ventures and subsidiaries.  There is now a need for a new set of indicators to take account of this range of activities and to provide a more comprehensive understanding of the CRIs’ overall performance.

Structure of indicators

The new indicators are structured as follows.

1. Research application metrics – to demonstrate the CRIs’ activities in transferring their research results and technologies.  These indicators are generic to all CRIs.

2. Relationships/influencing role – to demonstrate the role that the CRIs play in their target sectors and how effectively they influence and are engaged with those sectors.  These indicators are specific to each CRI.

3. Measure of impact – to understand the actual impact of selected research results/technologies that the CRIs have applied or transferred over the previous five years.  These indicators are specific to each CRI.

Generic indicators

All CRIs will set targets for the relevant indicators in their Statements of Corporate Intent.  They will report actual results in their annual reports.

Indicator	Definition
Commissioned reports to users	Reports for users commissioned under contract or other formal arrangement.
Presentations on technical information and research results	Papers, oral presentations, static displays and web presentations presented at a venue where users are present, eg a conference, workshop, training course, seminar, fielday or hui.
Publications on technical information and research results	These publications are not peer-reviewed and aim to disseminate technical information and research results to users, such as: §               papers in trade journals, magazines, series or books §               conference papers and abstracts §               research monographs or books §               popular books/articles §               web-based publications.
Peer-reviewed articles	A peer-reviewed journal article that requires a rigorous quality assurance by peers before publication.
Keynote and plenary presentations	A written or oral presentation delivered at a recognised forum where the CRI representative is invited and costs are paid in full or in part.
New or improved products, processes and services	‘Products’ are articles of substance produced by manufacture or other means (eg new plant cultivars, improved instrumentation, or new materials), or objects which otherwise embed information (e.g. software, plans, blueprints). ‘Processes’ comprise operational systems for supplying or realising products (eg frozen storage processes for food products, tests for diseases, substance extraction processes). ‘Services’ include the provision of technical support (eg advice, opinion) which should result in some substantive outcome (eg a problem resolved, decision-making criteria implemented).
Patents granted -         in New Zealand -         overseas	Registration for the same item that covers more than one overseas country is counted once only.
Requests for information from databases and collections	This covers both ‘nationally significant’ databases and collections and others that a CRI may consider is significant.
Licensing arrangements entered into	Formal agreements whereby an outside party will further develop or commercialise a CRI’s intellectual property (product, process, service etc.) and, in return, a CRI receives royalty income for units sold.
Joint ventures or formal associations	Incorporated or unincorporated joint ventures or associations with any other organisation for the explicit purpose of developing intellectual property or disseminating results.  Evidence would include memoranda of understanding, contracts, formal written agreements etc.
Spin-out companies formed	Subsidiaries that a CRI establishes, wholly owned by the CRI, for the purposes of further development or commercialisation of intellectual property.  The subsidiary is still dependent on the assignment of the CRI’s technology.
Spin-off companies formed	Companies formed in which a CRI retains some degree of ownership for the purposes of further development or commercialisation of intellectual property or for continued access to market knowledge

It is likely that certain indicators will be more important or relevant for some CRIs than others.  This could reflect the fact that some forms of transfer may be more suitable depending on each CRI’s client base and the actual knowledge, technology or research result that is to be transferred.  For example, the dissemination of research results on better cultivation practices to groups of users such as farmers and growers may be best delivered through seminars and workshops.  On the other hand, a CRI might consider it appropriate to deliver a new product to end users through a licensing arrangement with a third party.

Specific indicators

Relationships with clients

These indicators aim to collect information on each CRI’s engagement with its target client sectors during the financial year.  Unless a CRI is actively engaged, it is unlikely that it would be able to understand client needs and to apply effectively the results of its research.  ‘Clients’ are specific to each CRI and can include, for example, private sector firms, local and central government, and M?ori organisations.

Most CRIs will set targets for the number of strategic relationships or interactions with clients.  Strategic relationships can be demonstrated through, for example, memoranda of understanding, staff secondments, preferred supplier status etc.  Some CRIs may carry out customer surveys to collect information on how their strategies are aligned with those of their clients and/or how they are influencing the clients’ strategies. 

Impact of technology transfer

The CRIs will present case studies on the impact of research results and technologies transferred over the last five years.  Each case study will report on the underlying research effort, the means of transfer and, as accurately as possible, the quantifiable impact.  The impact can affect, for example:

·        intermediaries such as firms that produce or supply technologies developed by a CRI and benefit in the form of revenue growth and the upskilling of their workforce

·        end users such as farmers, growers, regional councils, and firms that benefit in the form of entry into new markets, cost savings, better work practices

·        New Zealanders as a whole who benefit in terms of improved environments, economic growth and protection against the entry of new pests. 

Where relevant, the CRIs will set targets for the above relationship and impact indicators at the beginning of each financial year.  They will report results in the first instance to shareholding Ministers at the end of each financial year.  The results will then be made public.