Review of The National Survey of Research Commercialisation Discussion Paper
Review of The National Survey of Research Commercialisation Discussion Paper
December 2014
1
Access
Unless specifically requested otherwise, submissions made to this discussion paper will be made
publicly available on the Department of Industry’s website. A request made under the Freedom of
Information Act 1982 for access to a submission marked confidential will be determined in accordance
with that Act.
Copyright
With the exception of the Commonwealth Coat of Arms and cover image, all material presented in this
document is provided under a Creative Commons Attribution 3.0 Australia
(http://creativecommons.org/licenses/by/3.0/au/ ) licence. To the extent that copyright subsists in
a third party, permission will be required from the third party to reuse the material.
The document must be attributed as the:
Commonwealth of Australia, Department of Industry, Review of the National Survey of Research
Commercialisation Discussion Paper, December 2014
Use of the Coat of Arms
The terms under which the Coat of Arms can be used are detailed on the following
website: http://www.itsanhonour.gov.au/coat-arms/.
2
Contents
December 2014 ................................................................................................................................... 1
Introduction......................................................................................................................................... 4
Purpose........................................................................................................................................ 4
Context ........................................................................................................................................ 4
Overview of survey .............................................................................................................................. 5
History ......................................................................................................................................... 5
Current Survey ............................................................................................................................. 6
Current Scope and Collections ...................................................................................................... 6
Stakeholder views................................................................................................................................ 7
A new approach ................................................................................................................................... 8
Repurposed.................................................................................................................................. 8
Refocused .................................................................................................................................... 9
Streamlined ................................................................................................................................10
Robust ........................................................................................................................................11
Aligned........................................................................................................................................11
Accessible ...................................................................................................................................12
Providing Feedback .........................................................................................................................12
Appendix A .........................................................................................................................................13
3
Introduction
Purpose
The National Survey of Research Commercialisation (NSRC) is being reviewed to help strengthen and
streamline the data collection regarding commercialisation of publicly funded research in Australia,
and to ensure alignment with current and emerging policy objectives for research commercialisation
here and abroad.
The review is also addressing administrative matters to make certain future collections and data
management processes are accessible and efficient for survey respondents, administrators and data
users. An underlying focus is how data might be better organised and presented to maximise
relevance and usage and maintain alignment with data collection abroad.
Context
Australia’s publicly funded research community includes universities, publicly funded research
agencies, medical research institutes and other research organisations. By international standards
Australia performs well in terms of research excellence and output, but poorly in translating publicly
funded research into commercial outcomes 1.
A key reason for this is the insufficient transfer of knowledge between researchers and business.
Australia ranks 29th and 30th out of 30 OECD countries on the proportion of large businesses and
small to medium enterprises (SMEs) collaborating with higher education and public research
institutions on innovation. 2
The Australian Government is actively implementing policy incentives that will improve the translation
of publicly funded research into commercial and broader public benefits. This includes 2014 budget
measures such as the Entrepreneurs Infrastructure Programme3 and proposals announced as part of
the Industry Innovation and Competitiveness Agenda4 and the Boosting the Commercial Returns from
Research Discussion Paper5.
Consistent with the policy objective to improve research industry collaboration and commercialisation
and thereby lift Australia’s productivity, prosperity and international competitiveness, the
Government will refocus the NSRC. This includes capturing new and robust data that will provide a
comprehensive picture of research commercialisation in Australia including pathways to
commercialisation.
1
Office of Australia’s Chief Scientist, 2014 and Cornell University, INSEAD, and WIPO, The Global Innovation Index, 2014.
2
OECD, based on Eurostat (CIS-2010) and national data sources, June 2013: http://dx.doi.org/10.1787/888932891359.
3
http://www.business.gov.au/advice-and-support/EIP/Pages/default.aspx
4
http://www.industry.gov.au/industry/Pages/Industry-Growth-Centres.aspx
5
http://www.education.gov.au/current-reviews-and-consultations
4
The review dovetails with other work undertaken by Government and the research community on
measuring the impact and industry engagement of the publicly funded research agencies. This work
includes:
• Impact and Engagement for Australia (Australian Academy of Technological Sciences and
Engineering, 2014) 6; and
• Excellence in Innovation for Australia trial (Australian Technology Network of Universities, 2012) 7
It also involves work undertaken by or associated with the former Department of Industry, Innovation,
Climate Change, Science, Research and Tertiary Education’s 2012 feasibility study on assessing the
public benefits arising from publicly funded research. This includes the 2013 discussion paper on
Assessing the wider benefits arising from university research8 and the following reports commissioned
by the department:
• Qualitative Research Practice: Implications for the Design and Implementation of a Research
Impact Assessment Exercise in Australia (Charles Sturt University, 2014) 9
• University Engagement Metrics (NewSouth Innovations, 2013) 10
• Research Performance of University Patenting in Australia (IP Australia, 2013) 11
• Australian Science, Technology and Research Assessment (ASTRA) Feasibility Study: final report
(Melbourne Institute of Applied Economic and Social Research, 2013) 12
The department has welcomed, considered and where relevant drawn on these bodies of work to
develop a new vision for the NSRC.
Overview of survey
History
The NSRC was launched in 2000 based on the United States/Canada Association of University
Technology Managers (AUTM) Licencing Survey. Its original aim was to establish a research
commercialisation framework to monitor performance in Australia that was comparable
internationally.
In its first iteration, the survey was conducted with universities, medical research institutes and the
CSIRO to capture data on formal intellectual property activity including the volume and value of
patents, licences, options, assignments and start-up companies. It comprised 13 main questions and a
small number of disaggregated questions, a combination of which now make up 16 time series
questions.
The NSRC was broadened after the first few collections with the inclusion of extra questions on
research contracts and consultancies, and skills development. Additional publicly funded research
organisations have also been included in the survey over time.
6
http://www.atse.org.au/atse/content/activity/innovation-content/developing-impact-engagement-australia-metric.aspx
7
https://www.atn.edu.au/Policy/Research/Excellence-in-Innovation-for-Australia-EIA/
8
http://docs.education.gov.au/node/34415
9
http://www.csu.edu.au/__data/assets/pdf_file/0008/994553/Impact-Report.pdf
10
http://www.nsinnovations.com.au/sites/all/files/uploads/Engagement%20Metrics%20Report%20October%202013%20-%20UNSW.pdf
11
http://www.ipaustralia.gov.au/uploaded-files/publications/Research_Performance_of_University_Patenting_in_Australia.pdf
12
http://melbourneinstitute.com/downloads/reports/ASTRA_Feasibility%20Study_FinalReport.pdf
5
An option to share success stories or case studies was included in the first three survey iterations.
Cooperative Research Centre (CRC) activity was also reported in the early surveys, with data now
sourced separately from the CRC Management Data Questionnaire.
In the 2008-09 collection, a number of disaggregated questions were added to the time series
questions, and other questions were introduced or expanded. This increase in the volume and
granularity of questions pushed average completion times up and drove data quality down as
respondents struggled to provide reliable data.
Current Survey
The NSRC operates as a voluntary, biennial survey of Australian universities, publicly funded research
organisations and medical research institutes. It is conducted every two years by the Department of
Industry, using a SmartForm, with the collection typically occurring over June and July. To date the
NSRC has enjoyed a high response rate – in recent collections over 85% of respondents have
completed the survey.
The NSRC is not tied to funding or legislation. Data is predominantly used for information purposes
including contributing to Commonwealth and state/territory government policy development and
benchmarking by institutions. System level data is also benchmarked with data from the United States
(US) and Canadian AUTM Licensing Survey and the United Kingdom’s (UK) Higher Education Business
and Community Interaction Survey.
Traditionally, NSRC data has been published every two years in a hard copy report and online 13 which
includes raw data down to institution level with the exception of data for 2012/13 which will be
released in electronic format only.
Knowledge Commercialisation Australasia (KCA) has used the NSRC instrument to survey its members
in alternate survey years.
• Comparison of current data – the most recent data collection is compared to the previous years’
collection;
• Time series data– based on 16 original survey questions and respondents with greater than or
equal to 70% response rate for those metrics in the dataset. Once the 2012/13 data is finalised
this data set will span 2000-2013;
• International comparison - six time series questions are used for benchmarking with the US,
Canada, the UK and when data is available, the European Union (excluding UK institutions); and
13 http://www.industry.gov.au/innovation/reportsandstudies/Pages/NationalSurveyofResearchCommercialisation.aspx
6
• Short data sets using various survey metrics.
CRC data from the CRC Management Data Questionnaire is reported with NSRC data to provide a
broad picture of research commercialisation activity in Australia.
Stakeholder views
Initial review consultations have been held with more than 45 stakeholders including Knowledge
Commercialisation Australasia (KCA), the Association of Australian Medical Research Institutes
(AAMRI), the National Health and Medical Research Council (NHMRC), universities (Deputy Vice
Chancellors-Research and commercialisation managers), university and industry peak bodies, medical
research institutes, overseas survey managers, and State and Territory governments.
Feedback on the survey has also been elicited from the survey questionnaire and through a targeted
email to survey managers. KCA has also provided separate feedback.
• There is overwhelming support for the survey to continue in a refocused and streamlined format;
• There is a lack of clarity about the purpose of the survey and what the data is used for;
• Some of the current questions are difficult to provide verifiable data for, particularly the
disaggregated and third-party survey questions and those that require fractional reporting;
• Where data verification is poor data reliability and usage is undermined;
• Quality data supports benchmarking by institution, cohort, region/jurisdiction and system and
provides a vehicle for recognising and rewarding good performance;
• Time series questions are easier to answer and generate better quality data;
• Accessing and aligning with other national collections will help streamline the survey;
• The biennial collection cycle is not supporting integration of the survey into respondents’ core
business;
• Reporting two years’ worth of data is problematic, especially for smaller institutions;
• Some definitions in the explanatory notes are unclear and need refinement; and
• For ease and efficiency, the collection instrument should be upgraded.
7
A new approach
A number of changes to the current survey are proposed in light of government priorities and sector
feedback. The review team welcomes stakeholder views on the issues outlined in this discussion
paper.
• Streamlining the survey to eliminate unnecessary and unverifiable questions to minimise the
administrative burden on respondents;
• Broadening the survey and adopting measures to capture the breadth of commercialisation
activity in Australia. This includes introducing new metrics on pathways to research
commercialisation while maintaining key intellectual property measures. Optional case studies to
support metric data may be included;
• Moving from a single-source survey collection to a multi-source data collection by accessing data
not only from survey respondents but various national data collections including IP Australia and
the Department of Education;
• Introducing new metrics that are already collected by respondent organisations or that are
available through other national data collections. In cases where data is not available in the
respondents’ information systems or collections, metrics will be phased in over time to allow
organisations time to prepare for the new survey requirements;
• Improving data quality by introducing a data verification requirement;
• Reporting data, including at institution level, using a digital platform that enables dynamic analysis
and visualisations, and providing regular communications with data highlights; and
• Implementing administrative and methodological changes to support these new arrangements
and achieve efficiencies for all parties.
Repurposed
Research-industry collaboration and research commercialisation is vital to supporting the health and
wealth of the nation. As noted, Australia needs to strengthen its performance in this field.
With a broader focus, the NSRC could be used to report on how Australia’s publicly funded research
sector is faring in transferring knowledge for economic and societal benefit. Broadening the survey
and adopting measures to capture the breadth of commercialisation activity in Australia may inform
an annual national statement on the research system showing activity at the sector and institutional
levels. Repurposing the NSRC in this way could provide useful information on Australia’s research
system, and enable robust benchmarking, policy and planning processes across industry, government,
and research sectors.
To ensure the future data collection provides a complete picture of Australia’s research system, the
scope of data could also be expanded to include other players in the public research space including
Rural Research and Development Corporations (RDC).
8
Key Questions
• What are your views on including new measures to capture the pathways to
commercialisation activity in Australia?
• Would expanding the data collection to include RDCs provide useful information on the
broader research system?
• Are there other entities in receipt of public research funding that should be included in a
repurposed survey?
Refocused
As noted above, the NSRC could be broadened to include new metrics on industry engagement and
pathways to research commercialisation. This would ensure a suite of measures are available to track
and report activity recognising commercialisation occurs through a range of processes and
relationships, and can result in a variety of products, systems and services that benefit industry and
the community.
New measures under consideration include income derived from and/or number of:
The option to reintroduce case studies is also under consideration. Success stories were included in
the first few survey collections providing respondents the opportunity to highlight in narrative form
successful examples of commercialisation product sales. If case studies are re-introduced to the new
collection, they could be optional with drafting instructions provided to ensure information is clear,
consistent, succinct and targeted.
To ensure the survey integrates with core business and supports easier access to information systems,
the NSRC will move to an annual collection cycle over July/August.
9
Key Questions
• Of the new metrics on research engagement being considered, what measures are most
useful and/or feasible to collect? Why?
• Of the new metrics on research engagement being considered, what measures pose the most
concerns in terms of feasibility of collection or issues with the data?
• If new metrics are introduced that are not available in respondents’ information systems or
external collections, what matters should be considered prior to implementation?
• What is your view on the inclusion of case studies in a new data collection?
Streamlined
Consistent with the Government’s commitment to deregulation, the current survey could be
significantly streamlined by reducing the volume and granularity of questions. This could be achieved
by:
• Assessing the quality and usefulness of existing questions and discarding questions where
appropriate including disaggregated questions as appropriate. This exercise would consider:
o whether questions are part of the time series dataset;
o quality and quantity of responses provided over recent surveys;
o relevance to current and emerging policy objectives;
o usefulness in benchmarking; and
o availability of data in institutions’ collections.
• Drawing on data in existing collections including IP Australia’s Analytics Hub, the Department of
Education Higher Education Research Data Collection and the Australia New Zealand Clinical Trials
Register.
In addition, the department is investigating opportunities provided by initiatives such as ORCID14 and
Researchfish15 that over time will offer other data on research impact and pathways to
commercialisation.
14
http://orcid.org/
15
https://www.researchfish.com/
10
Key Questions
• What administrative and methodological changes would help make the survey easier to
complete?
Robust
To ensure the collection is robust and reliable, in future data provided by respondents may need to be
verified as correct by either an auditor or a senior official such as a chief executive or operating officer
or vice or deputy vice chancellor of a university. This is important if benchmarking and policy and
planning processes are to be based on accurate information. While this new requirement may
increase administration responsibilities for some respondents, this would be offset by the reduction in
the number of questions.
As a number of respondents have concerns about the clarity of some existing questions, the
explanatory notes will be revised to tighten definitions of questions that carry over into the new
survey. Wherever possible, definitions will be aligned with other data collections. New explanatory
notes will be included as part of the testing process for the new survey scheduled in 2015.
Key Questions
Aligned
The current NSRC benchmarks six of the 16 time series questions against US, Canada, UK and when
available, European Union data. These questions will be maintained in the new survey.
To ensure Australia keeps pace in the broader global community and the region, international
benchmarking may be expanded to include additional Asian countries with comparable innovation
systems and surveys including South Korea. Consideration will also be given to expanding the metrics
used for international comparisons to capture and benchmark new data.
To ensure data remains relevant and targeted, the design of the collection along with reporting
arrangements may be adjusted over time.
Key Questions
11
Accessible
Options for secure web-based survey tools to replace the current SmartForm are being explored. A
new collection instrument must improve functionality by maintaining the good features of the current
collection instrument while addressing its errors and limitations. It should be compatible with all
standard browsers, operating systems and PC devices and interface successfully with data
management systems in the department.
The new instrument will be based on the revised survey design and tested in 2015. It is envisaged the
new survey tool be used to collect 2014 data in late 2015.
Options to publish and report data are also under consideration. In addition to producing static data
sets, alternatives are being explored on how sector and institutional based information can be
presented using benchmarking and analysis tools.
A new communications strategy for the collection will also be developed to ensure the data remains
dynamic. This could include regular highlights and special communiques noting trends and themes
relevant to current policy issues and emerging challenges.
Key Questions
• How can data be presented in the future to support access and usability?
• How useful would you find a digital platform that enables dynamic analysis and data
visualisation?
Providing Feedback
This discussion paper is the formal consultation component of the NSRC review. The department
welcomes input from industry, research, government sectors as well as the general community, about
the proposed changes to the survey. Submissions will form part of the evidence base used to guide the
redevelopment of the NSRC over the short and longer term. If you wish to be involved with testing of
the new survey tool please note this in your submission.
Input on questions outlined in the paper, along with general comments, should be submitted either
directly to the website at https://consult.industry.gov.au/ or by emailing
NSRCreview@industry.gov.au.
In order to facilitate thorough consideration of this paper, an eight week response period has been
allocated. Your feedback is sought by close of business on Friday 27 February 2015.
For more information please contact:
Mr Mark Wigley, Manager
Commercialisation & Knowledge Transfer Section, Commercialisation Policy Branch
Phone: (02) 6213 6400
12
Appendix A
National Survey of Research Commercialisation
Section No Question Proposed Handling Notes
Handling
16
16
In = retain metric in future survey. Out = remove metric from future survey. Ext = keep metric but source data from external collection.
13
National Survey of Research Commercialisation
Section No Question Proposed Handling Notes
Handling
16
14
National Survey of Research Commercialisation
Section No Question Proposed Handling Notes
Handling
16
Patent & 15 How many patents and/or plant breeder rights Ext
Plant Breeder were culled or allowed to lapse from your
Rights institution's holdings
Holdings
Licenses/Opti 16a How many Material Transfer Agreements did your In
ons/Assignm institution enter into where your institution Good response rate. Review definition
ents (LOAs) provided the materials
16b What income did you derive from the agreements Out
executed Difficult to collect data?
LOAs 17 How many LOAs did your institution A) execute and Maintain time series question and
b) have active: total.
ai - How many licences executed Out
aii - How many options executed Out
aiii - How many assignments executed Out
aiv - Total number executed In Time series question, International
comparability
bi - How many licences active Out
bii - How many options active Out
biii - How many assignments active Out
biv - Total number active In
LOAs 18 What was the location/ownership profile of the
Difficult to collect data?
organisations with which LOAs were executed:
a - Majority Australian owned and based Out
companies/organisations
b - Majority Australian owned and foreign based Out
companies/organisations
c - Foreign owned and Australian based Out
companies/organisations
d - Foreign owned and foreign based Out
companies/organisations
e - Unknown Out
f Total Out
LOAs 19 How many active LOAs yielded income in [date] In Maintain time series question,
International comparability
LOAs 20 For active LOAs that yielded income in [date], how
many LOA incomes can be attributed to the Total active LOAs income is sufficient.
following income categories? What is the value of Poor response rate
income derived from each income category?
ai - Number of Running royalties Out
aii - Income of Running royalties Out
bi - Number of Cashed-in-equity Out
bii - Income of Cashed-in-equity Out
ci - Number of Other Out
cii - Income of Other Out
di - Total Number Out Duplicated from Q17biv
dii Total Income Out Duplicated from Q19
LOAs 21 For those active LOAs that yielded income in [date],
Total captured in Q19. Breakdown of
how many LOA incomes can be placed into each of
income thresholds not necessary.
the following income ranges?
a - Between $0 & $10,000 Out
b - Between $10,001 & $50,000 Out
c - Between $50,001 & $200,000 Out
d - Between $200,001 & $500,000 Out
e - Between $500,001 and over Out
f Total Number Out
15
National Survey of Research Commercialisation
Section No Question Proposed Handling Notes
Handling
16
b - What was the total value of equity received from all Out
research commercialisation equity holdings that
were fully or partially exited
Start-up 26 For all start-up companies your institution was In
Companies formally involved with and were operational as of Maintain time series questions
[date]:
a - How many were dependent upon the In
licensing/assignment of your institution's
technology for initiation
b - In how many of the companies identified did your In
institution hold equity
Start-up 27 What was the number of start-up companies that Out
Companies were dependent upon the licensing of your
Poor response rate
institution's technologies for initiation that ceased
operations in [date]
Start-up 28 How many start-up companies did your institution In
Maintain, time series question.
Companies launch in [date]:
Review question/definition.
[provide details for each start-up company]
Research 29 For research consultancies and contracts your Ext/ In
Contracts, institution entered into in [date] please identify: Could take out of direct survey for
Consultancies universities and source from HERDC.
and Direct Maintain for other respondents.
Sales
a - Number of Consultancies Ext/In
International comparability
b - Total gross contracted value of consultancies Ext/In
Good response rate.
c - Number of Contracts Ext/In
29a and 29c could be collapsed
d - Total gross contracted value of contracts Ext/In
e - Number of direct sale transactions Out
f - Total gross value of direct sales Out Poor response rate. Value of data?
Research 30 Of those research consultancies and contracts in Ext/In/Out Could take out of direct survey for
Contracts, Q29, identify: universities and source from HERDC.
Consultancies a) The number of research consultancies and Could remove or maintain with fewer
and Direct contracts according to total gross contracted value: thresholds for other respondents
16
National Survey of Research Commercialisation
Section No Question Proposed Handling Notes
Handling
16
Sales
18