Issue link: https://www.e-digitaleditions.com/i/302559
44 May/June 2014 BizEd Ten executive coaches, mostly working practitioners or retired executives, mentor and evaluate the teams. They each handle two sec- tions of the course and work with students approximately eight to 12 hours per week. We pay them US$1,800 per section. The simulation takes place over eight financial quarters, with each quarter lasting approximately one week. After four quarters of play, teams pause for a couple of weeks to reflect on their business perfor- mance, tactics, and strategy, as well as prepare detailed business plans for future growth. At the end of this time, they participate in a ven- ture capital fair where they present and defend their business plans to outside investors. And at the end of the course, teams must face these same investors to account for their performance by presenting stock- holder reports. At the end of every quarter, each team gives a 25-minute executive briefing to its coach, who assesses the students' business acumen. The coach's role is to challenge students' analyses, plans, tactics, and projections. When students are having trouble with difficult concepts during the briefings, our coaches give "chalk talks"—brief lectures tailored to the problem the students are facing. Coaches lay out the components of the prob- lem on a whiteboard—whether it's creating a marketing plan or deter- mining an investment strategy— and help students work through it. When we created Marketplace nine years ago, our coaches made no formal assessments; they gave students only overall grades for the quality of their team briefings at the end of the course. However, as we refined our assessment process, we discovered new skills we wanted to help students improve. That's why we decided to train our executive coaches to evaluate stu- dents with three different rubrics, one used for the executive briefings, one for the business plans, and one for the stockholder reports. As the coaches complete the rubrics, they evaluate the thought process, skill sets, and critical thinking that went into students' decisions. The rubrics are shared with the students in advance to set expectations and provide guidance. Before taking responsibility for mentoring student teams, our executive coaches attend a two-day workshop where they are intro- duced to the rubric framework. (See the rubric at www.bized magazine.com/features/assessment- rubric/.) They also work through the simulation themselves and practice the "chalk talk" format. For the first semester of their ser- vice, they work side-by-side with a veteran coach to learn how to eval- uate students effectively. Finally, they attend "norming" meetings, where coaches and faculty come together to ensure everyone is applying the rubrics consistently. Assessment, Before & After In the first years of the course, we based our objective assessment of students' management skills on market share and profits. No matter how hard we pushed the students to use data to support their decisions and link their decisions across func- tions, we did not see the results we wanted. As a result, their executive briefings, business plans, and stock- holder reports suffered. Our stu- dents could identify areas of neces- sary investment, but they struggled to apply principles of marginal cost and marginal revenue, decide which investments promised the best return, or determine where to focus resources when capital was low. They could identify problems but didn't always know what solu- tions to apply. We often were dismayed that students could conduct a SWOT (strengths, weaknesses, opportuni- ties, and threats) analysis, but then would ignore it when they adjusted their strategic decisions. They also were challenged when asked to view their firms in an integrated way. They would give each other "high fives" when they increased market share but frown when we asked them about employee morale or the profitability of a segment. In chess, the best players think several steps ahead of the game, but too many of our students could not coordinate moves or anticipate how the board could or should look two or three financial quarters in the future. Now, each quarter, the executive coaches score each student with a rubric that measures depth of understanding, breadth of under- standing, and management by the An executive coach conducts an executive briefing with a student team in "Marketplace," a startup course at the University of Tennessee, Knoxville.