Asee peer logo

Qualtiy And Consistency In Idea Pitch, Research Proposal And Business Plan Competition Judging

Download Paper |


2010 Annual Conference & Exposition


Louisville, Kentucky

Publication Date

June 20, 2010

Start Date

June 20, 2010

End Date

June 23, 2010



Conference Session

Improving Student Entrepreneurial Skills

Tagged Division

Entrepreneurship & Engineering Innovation

Page Count


Page Numbers

15.1009.1 - 15.1009.17



Permanent URL

Download Count


Request a correction

Paper Authors


Daniel Ferguson Ohio Northern University

visit author page

Daniel Ferguson is Assistant Professor of Entrepreneurship at Ohio Northern University.
Prior to coming to Ohio Northern University he was Associate Director of the Inter-professional Studies Program at Illinois Institute of Technology and Co-PI on multiple National Science Foundation grants relating to assessment processes and interventions aimed at improving learning objective attainment. Prior to his University assignments he was the Founder and CEO of the The EDI Group, Ltd., an independent professional services company specializing in B2B electronic commerce and electronic data interchange; and a Vice President at the First National Bank of Chicago, where he founded and managed the bank’s market leading professional Cash Management Consulting Group and initiated the bank’s non credit service product management organization and profit center profitability programs.

visit author page


Michele Govekar Ohio Northern University

visit author page

Michele A. Govekar, Ph.D.

Michele is a Professor of Management at Ohio Northern University.

Her academic research and publications focus on the management and history of US firms’ international operations, corporate- nonprofit interactions, nonprofit organizations, and the process, outcomes and assessment of management education.

She currently teaches undergraduate courses in strategy, corporate citizenship, small business and project management. She is the co-program chair of the North American Management Society 2009 conference and past Division Chair of the Management History Division of the Academy of Management for 2005-06.

visit author page


Amanda Stype Ohio Northern University

visit author page

Amanda Stype is a 2009 graduate of the James F. Dicke College of Business Administration at Ohio Northern University with an Honors degree in International Business & Economics and also in applied mathematics. She currently is a graduate student at Bowling Green State University in their Master of Economics program. She plans to pursue a Ph.D. in applied microeconomics in the near future.

visit author page

Download Paper |

NOTE: The first page of text has been automatically extracted and included below in lieu of an abstract


If a student or innovator has a business plan that is judged to be viable and competitive, this can be seen as a

predictor of future success. However, the process to determine viability of plans is uncertain and often

perceived as overly subjective. This past year we collected data from Idea Pitch Competitions at Business,

Engineering, Pharmacy and Arts and Sciences colleges. We identified interventions or conditions that, either

cited in the literature or from practical experience, were present or applied in these judging situations. We used

the aWG statistical measurement techniques identified by Brown and Hauenstein (2005) to calculate consensus

among the judges in initial and final rounds. We identify interventions that appear to improve consensus over

time across assessment or competition situations. Finally we invite collaborative efforts to test individual or

combinations of interventions that offer the highest levels of consensus and continuous improvement in aWG .

Lack of a high level of interrater agreement can indicate poor judging and a weak judging process, a

professional issue of some importance. Very often judges volunteer to be part of the judging process but possess

varying degrees of knowledge or expertise regarding the outcomes or knowledge being judged. When

organizing assessments or competitions, effectively executing the event is the main goal and there may be no

formal attempt to measure interrater agreement or to improve the level of consensus, ie. interrater agreement, in

the assessments or competitions over time. Our goal is to provide evidence supporting the use of interventions

that do, in fact, improve interrater agreement as well as the perceived fairness of the event.

In competition and proposal assessment situations, it is important to measure consensus between judges.

Measuring consensus between judges can expose problems within the judging process such as a difference in

expectations between judges. Because many times the outcome of these assessments and competitions greatly

affects the participant, whether it is in the form of determining someone’s grade, a person who receives an

award, or a person who receives further funding, it is important that those organizing and heading these

competitions or assessments seek to make the situation as fair as possible. In most competitions, each presenter

or group is rated by a different group or panel of judges, which makes a high level of evaluation consensus

3 Submitted by Daniel M. Ferguson, Michele A. Govekar, and Amanda C. Stype Thursday, April 01, 2010

Ferguson, D., & Govekar, M., & Stype, A. (2010, June), Qualtiy And Consistency In Idea Pitch, Research Proposal And Business Plan Competition Judging Paper presented at 2010 Annual Conference & Exposition, Louisville, Kentucky. 10.18260/1-2--16654

ASEE holds the copyright on this document. It may be read by the public free of charge. Authors may archive their work on personal websites or in institutional repositories with the following citation: © 2010 American Society for Engineering Education. Other scholars may excerpt or quote from these materials with the same citation. When excerpting or quoting from Conference Proceedings, authors should, in addition to noting the ASEE copyright, list all the original authors and their institutions and name the host city of the conference. - Last updated April 1, 2015