A Year of Collaboration: MobLab in Kellogg School of Management

“Welcome,” beckons Donald Dale, waving me into his classroom. “It’s so good to see you.”

We continue to exchange cheerful greetings. It’s been a year since the Kellogg School of Management at Northwestern University adopted MobLab. We’ve been working closely with Donald and other professors at Kellogg during the adoption to integrate MobLab and create new meaningful games for MBA students.

As I sit down with a cup of Donald’s self-brewed coffee, I am swept with a wave of excitement—and caffeine. Today, I am a Kellogg Weekend MBA student. First, Microeconomic Analysis followed by a Strategy and Organization course in the afternoon.

For a Saturday morning at 9:00 a.m., the feeling in the room is electric. This is the last class of the quarter. For 10 weeks these students have committed to learning Microeconomic Analysis, many traveling from different states to attend.
Today’s lesson plan includes asymmetric information. To kick off the topic, we begin with the MobLab Market for Lemons Game after a brief discussion of the instructions and clue that the buyer’s valuation is 150% of the seller valuation. Boom! A quick game created an asymmetric market and a great discussion.

Donald Dale Market for Lemons
A market suffering from adverse selection will see purchases (blue circles) clustered on the left-hand side of the graph (i.e., low quality cars). You will also see the right-hand side of the graph (i.e., high-value cars) populated with rejected offers (red diamonds) as buyers learn to fear that high-priced cars are not in fact high-quality cars.
The area between the two reference lines represents the possibility for gains from trade. Only two sellers, out of a class of 40 offers were accepted to split the gains from trade.

After lunch, I join George Georgiadis in his Strategy and Organization class. Thanks to the help of George, Michael Powell & Niko Matouschek, MobLab collaborated with Kellogg to create a MobLab Double Marginalization Game. Previously, Kellogg ran this experiment using paper and pencil. Now with MobLab, students can quickly explore two potential solutions to this double marginalization problem: acquisition and franchise fees.

One of the benefits of adopting MobLab is our customer service and excellent support. We look forward to working together to provide a smooth transition and beneficial learning experience. A large thank you to the Northwestern Digital Learning Team for making this vision possible.

CTREE & ME: A Reflection of Growth and Love

Dearest CTREE,

I am reflecting back on the years at CTREE and getting wistful. Is it the romance of wall-to-wall books? After all, I am writing this from a bookstore and we are both bibliophiles. Is it the hum of baristas pulling shots and cool coffee shop music? I could go on. Whatever the reason, nostalgia has its grip. Just permit me to walk down memory lane for a moment.

When MobLab was little more than a gleam in our founder’s eyes we attended CTREE. There were a number of presentations on classroom experiments, including Vernon Smith’s keynote address. The audience reception was warm. That was the moment…the moment we knew this venture called MobLab might actually work. Depending on what stats you read, somewhere around 90% of startups fail. But, we believed – against the odds – that we would succeed. Instructors wanted to bring interactive experiments and surveys into their classes. What a ride it’s been! We’ve been going 6 years now and continue to grow.

A big part of our success was inspiration at CTREE. The word inspire means to breathe (spire) into (in). Instructors at CTREE continue to inspire us. These instructors don’t take the easy road, but continue to search for different ways to teach students. Growing in stride with CTREE, we happily pause for reflection, for in our journey together, there is much yet to come.

From the bottom of our heart: Thank you CTREE.


Why should you do classroom experiments?

Why should you do classroom experiments? Here I will invoke Aristotle. He postulated three important components to a good argument: logic, credibility, and emotional appeal. Economics models are logical but struggle to make an emotional connection with students and may not be viewed as credible. Classroom experiments can help on both fronts.

  • Consider the competitive market. The “invisible hand” suggests individuals following their own self-interest leads to an efficient allocation of resources. The invisible hand may be considered voodoo, superstition, or hocus-pocus. However, the double auction helps to show the prediction of convergence to equilibrium price and quantity is empirically valid.
  • Consider the prisoner’s dilemma. Students experience being the “sucker” that chose Cooperate when their partner chose Defect. Getting burned helps cement the idea of a “best response” and equilibrium.

There is a growing literature showing that classroom experiments improve learning outcomes. Sam Allgood and others wrote in their Journal of Economic Literature survey, “The evidence on the use of classroom experiments and cooperative learning shows consistent positive effects on student achievement …”. In my opinion we are only scratching the surface. Once we better understand the mechanism for how classroom experiments improve learning outcomes it will be even better. But, I think a big part of it is that experiments appeal to students in so many different and memorable ways.

In short, here are the benefits of classroom experiments as I see them:

  • Builds an emotional connection with the material
  • Shows an empirical basis for the claims we make (or where our models can be better)
  • Encourages students to think like scientists
  • Sparks the joy of discovery
  • Instructors have more fun too!


CTREE 2017: Come see us!

Ladies and Gentlemen! Educators of all ages! CTREE 2017 is almost here. It’s going to be great to reunite with old friends and meet new friends too. At the conference MobLab is unveiling our Instructor and Student Workbooks. These workbooks will make it easier than ever to use MobLab. Let’s get (a little) more specific:

  • Instructor Workbooks include step-by-step guide to running the gamesresults walk-through videos, and other content that help instructors teach with MobLab games.
  • Student Workbooks include warm up questions, thinking through theory, space to record decisions, and further questions. For example, instructors can post video instructions for the classroom experiment into their LMS. Students can complete warm up questions by being active listeners.

If you’re interested in hearing about these workbooks and other MobLab initiatives schedule a meeting with us. It would be great to hear from you! We would also love to give you a MobLab USB drive that contains the Instructor and Student Workbooks for Principles of Micro, Principles of Macro, and Intermediate Micro. It will also contain links to a number of helpful support videos.

If you can’t schedule a meeting with us, please come by our booth!


Learning Outcomes for Economists: Part 2

I wanted to follow up on the previous post that discussed how learning outcomes help guide student effort and make students better economists. In Part 2 I re-state the learning outcomes and discuss how I see these relating to MobLab:

LO 1. Explain how economists use the scientific process to expand understanding of individual decision-making, market outcomes, and government policies, and apply the process by practicing curiosity and hypothesis testing.

The key word here is how economists use the scientific method. The 2002 and 2012 Nobel Prizes are recognition that experiments are an important method in economic science. Moreover, even if an economic question is better answered with natural data sources it is useful to ask, “What would the best experimental test of this be?” One of the strengths of MobLab is showing students how a theory can be tested in a hands-on way.

LO 2. Choose and use appropriate concepts and models to analyze and evaluate choices, outcomes, and policies in diverse settings.

Here I expound on diverse settings. Experiments are micro-economies governed by institutions or “rules of the game”. Institutions have two features. First, they structure incentives that affect strategic interaction. Second, as Khaneman and Tversky show, the presentation of a decision situation can also have powerful framing effects. Different models of human behavior may be operable in different institutions or frames. For example, self-interested behavior in a market transaction versus other-regarding behavior in a public goods setting. Experiments help students to discover what models of human behavior are operable in diverse settings.

LO 3. Develop quantitative reasoning skills by working with equations and graphs and by explaining the need for empirical methods that distinguish causation from correlation.

Quantitative reasoning. Getting students to participate in data exercises can be challenging. Big understatement! But, with experiments students are familiar with the data generating process and often relish the opportunity to evaluate their own decisions and those of their peers. Because of their simplicity experiments also provide a great foundation for hypothesis testing.

LO 4. Identify the assumptions underlying models, and connect the assumptions to particular theoretical results and/or observed conditions.

Connect the assumptions. This is crucial. For an experiment to be a good test of theory we need to make sure the constructed microeconomic environment mirrors theory. For example, “If the market is perfectly competitive then, there is an equilibrium price P for which quantity demanded equals quantity supplied.” What do we need to test this hypothesis? We need suppliers and demanders. We need to give demanders a value for the commodity being traded. We need to give suppliers a cost to produce the commodity being traded. Those values and costs might be governed by different behavioral assumptions, for example, demander’s values might follow diminishing marginal utility. Finally, we need rules for how demanders and suppliers interact with each other. These rules should be a close approximation to the perfectly competitive model. Then, run the experiment!

LO 5. Discuss economic issues in ways that promote mutual understanding and inquiry, demonstrate fluency in basic economic terminology and tools, and explain economic reasoning to and incorporate insights from noneconomists.

Promote mutual understanding and inquiry. The modules and the student workbooks MobLab economists are developing are heavy on reflection. Students communicate their thought process in the experiment and how it evolved. Students articulate differences in behavior across institutions and become fluent in the interplay between theory and outcomes. Sometimes there are systematic divergences between theory and outcomes which leads to new questions and opportunities to incorporate insights from psychology, sociology, and more.

Learning Outcomes for Economists: Part 1

The current issue of the American Economic Review contains an article titled, “Learning Outcomes for Economists“. This is a brief article with a simple premise to use “Learning Outcomes” (LO) to guide student efforts. According to Allgood and Bayer a good LO should focus on (1) What students should know, but also on (2) What they should be able to do with that knowledge. In their paper they present five LOs that were, “the culmination of a project organized by the Social Science Research Council (SSRC) involving a dozen economists thinking carefully about what it means to learn economics.”:

Microeconomics provides a set of tools for understanding, predicting, and evaluating individual choices and collective resource allocation. By achieving the following learning outcomes, you will be able to make better decisions with the resources you control, be better equipped to explore policy issues, be prepared for further study in economics, and discover how the study of economics can enhance your intellectual exploration of the world. By the end of the course, you should be able to:

LO 1. Explain how economists use the scientific process to expand understanding of individual decision-making, market outcomes, and government policies, and apply the process by practicing curiosity and hypothesis testing.

LO 2. Choose and use appropriate concepts and models to analyze and evaluate choices, outcomes, and policies in diverse settings.

LO 3. Develop quantitative reasoning skills by working with equations and graphs and by explaining the need for empirical methods that distinguish causation from correlation.

LO 4. Identify the assumptions underlying models, and connect the assumptions to particular theoretical results and/or observed conditions.

LO 5. Discuss economic issues in ways that promote mutual understanding and inquiry, demonstrate fluency in basic economic terminology and tools, and explain economic reasoning to and incorporate insights from noneconomists.

I think these dovetail well with what we’re doing at MobLab. In a follow up post I will write material that matches our platform content and student workbooks with the learning objectives offered in Allgood and Bayer (2017).

Sam Lepler MobLab

Conversations with Educators: Sam Lepler

They offer game theory courses in High School? (YES!) Here to tell you more about it is Sam Lepler, Economics Instructor at Harker Prep School in San Jose, CA.

Q: First, please tell everyone a little bit about yourself and how you became interested in economics.

A: I am a native Californian, studied college at the University of Pennsylvania undergrad, went to live in Japan and taught English for two years subsequently, and then boomeranged back to California to begin my career as a teacher. I am married with two kids, and I am also a rabid San Jose Sharks fan (I play hockey, too) as well as SF Giants baseball.

I first became interested in Econ in high school through my own AP Economics class. My teacher, Michael Brody, remains a good friend 18 years later to this day, and just made Econ come alive. I ended up majoring in Econ at U of Penn, and went on to get my masters in Econ at San Jose State University. I feel like Econ, albeit pretty much just glorified logic, is an incredibly powerful tool for modeling and understanding human behavior as well as predicting unintended consequences of often well-intentioned, yet counterproductive policies. It’s applicable to so many areas of real life from corporate marketing and pricing strategies to border walls, tariffs, and other current ongoing political debates.

Q: You are teaching game theory at the high school level. Tell us more about that and how MobLab fits in.

A: I am indeed teaching what we call “Honors Advanced Topics in Economics: Game Theory.” The kids like to call it HATE Game Theory :-). We just launched it this past year for seniors who took AP Econ as juniors and were clamoring for more in the economics space. It’s a super applied class that mixes in various aspects of strategic interaction from simple sequential games and simultaneous dominance solvable games, to much more complicated calculus based continuous strategies and oligopoly models, and even strategic moves, voting, auctions, and bargaining. I try to mix theory with mathematics to create a guided applied economics class. I also try to add in behavioral and experimental elements to show the weaknesses of classical game theory as well.

MobLab is the main tool I use for the applied and experimental components. The games that I use range from the standard market model to simple games like the ultimatum game, various matrix games, bidding/auction games, and many more. The games take seconds to set up, a few minutes to play, and then instantaneous feedback about trends and how the theory and practice either converged or diverged and why. This entire section of the class takes minimal prep time and can be effectively executed and impactful in 15 minutes or less. The efficiency and ease of use is remarkable and the reputation of the class as being challenging yet really fun is largely due to MobLab. I actually tabulate game points throughout the semester and assign 10% of the grade based on class ranking on those games to give real incentive for students to try. It works great!

Q: For your strategic moves unit you ran a 10 round-repeated ultimatum game. Tell us a little more about the setup and how you used the game to illustrate threats, promises, commitment strategies, establishing credibility, and countering others’ moves against you.
Ultimatum game experiment
A: Indeed, I did use that game and others to demonstrate the power of strategic moves. By playing many times against the same person, the player who moves second in that ultimatum game can use threats of rejection, which are of course mutually harmful, to get better splits over time. Naturally, their main credibility mechanism is reputation, so many rejected on round one. I was looking to see whether any of them threatened to reject above a 50-50 split, as they could ask for a 70-30 or 90-10 split in their favor. However, none asked for more than 50-50 and pairs where both players were stubborn ended up pretty bad. The chat feature is what allows the threats to be observable so that is essential, but at times students would lie about who they are and pretend to be one of the more irrational students in the class to increase the likely success of a threat. It was awesome to watch the students get creative in figuring out ways to establish credibility, and to see how the first movers tried to counter those threats. One student immediately wrote in chat that he was hiding the chat box and would not read anything anymore, thus cutting off communication and making subsequent threats by player 2 unobservable and therefore ineffective. Very clever!


Q: How do your students react when you announce that you are playing a MobLab game in class? To encourage thoughtful and strategic game play, do you incentivize winners or mark participation?

A: They ask for MobLab every day–literally! They usually want to finish lectures and going over HW as soon as possible to maximize game time. Of course, those at the top of the leaderboard lead tend to try to stall to minimize the games at times, but even that is a fun game within a game. The reactions when a game goes well or when a partner cheats on a collusive agreement in a prisoner’s dilemma is so fun to watch. They are instantly engaged and incentivized to really think and apply our concepts in a meaningful way since the points rankings directly correspond to grades. Of course, since the points are in rankings, the incentives are a little different compared to if points were dollars, and it might be better to reject in an ultimatum game if the point spread between you and your partner is too large.


Q: What advice would you give a new instructor who wants to use games in their classroom?

Sam Lepler MobLab A: My advice is to try the game out with at least one or two practice rounds first. Sometimes it takes some time to understand the settings of a game or to make sure the numbers and payoffs are accurate for the intended game. Also, I would recommend starting with simple games like the ultimatum game or prisoner’s dilemma to make sure the students know how to use the program well. Furthermore, it’s crucial that the points have real meaning be it attached to grades, M&M’s or whatever. Lastly, explore all of the features and games as much as you can. MobLab can be used efficiently and easily to teach nearly every economic topic there is, especially in micro from S+D, price controls, taxes, elasticity, surpluses, externalities, all market structures, and so much more. For larger classes (30+ students), it allows for effective market simulations in a few minutes, so the time savings is crucial compared to other more traditional analog simulations, and the results are instantaneous. I cannot think of another tool out there that allows a teacher to show the effect of so many economic variables (like S+D shifts, etc.) in so little time to make graphs and models meaningful.

What Advice Would You Give New Instructors?

In our recent end-of-term instructor survey we asked the question, “What advice would you give a new instructor who wants to use MobLab in their class?” Below are a sample of responses:

  • Take a moment to connect with the Moblab staff to get started. Once you pay the start-up costs (which are not very high), it is easy to use. The students enjoy it, and the system is quite stable.
  • Try out the game ahead of time, and watch the student instruction video.
  • Take the time to really figure out how the games will incorporate into the overall course.  Also, to make it required.  Using it for extra credit leaves too few players.
  • Just use it. You will love it.
  • Don’t freak out when you don’t get the results you “want.” These are teaching moments. Have students help you figure out why things didn’t go as expected.
  • Start slow and don’t expect it to do *too much* at first.
  • just do it — it’s easy in F2F.

Much of the advice regarded instructor preparation. We are happy to report at CTREE we will showcase Instructor and Student Workbooks. We are also producing more support videos this summer for our instructors. It will be easier than ever to integrate MobLab with your class. And, of course, you can still take advantage of our excellent support.




NFL Free Agency: You Win, You Lose, You Curse

Some pairs survive the test of time: Tom and Jerry, milk and cookies, and Han Solo and Chewie. In economics one consistent pair is the common value auction and the winner’s curse. Here I will briefly describe the common value auction, winner’s curse, and apply it to sports free-agency.

A “common value auction” is one where bidders have the same (read: common) value for the item. A classic example used in classrooms is having students bid on a jar of pennies. In these settings bidders are assumed to have different “signals” about the value of an item — like people have different ideas on the number of pennies. From their signal each bidder forms an estimate and submits a bid.

If the actual value of the item tends to be the average bid then the winner of the object (highest bid) will tend to overpay. This is the winner’s curse and gives rise to the quip: You win, you lose, you curse. The winner’s curse seems to have first been acknowledged in the oil industry (see here) and our game screen at MobLab (below) pays homage to that origin.

The logic of the winner’s curse can also be applied to sports free agency. So your team is thinking of signing Cuter, Kap, Mangold, or others on the market. While it’s a rough approximation (players can have different values to different teams) the logic of the winner’s curse will still hold. The most optimistic team will win the service of the free agent and often they will frequently be sad they did. Especially win there are lots of teams bidding.

This has been discussed a number of times in baseball. Here is a great quote from a NYT article on the subject,

“Most times when you sign a player, you’re the only team in the game that was willing to pay him that money … There’s a rare occasion someone slips, but for the most part, you could almost argue he’s an untradeable player because no one else was willing to pay him; otherwise, they would have had him.” -Alex Anthopoulos, former GM of Toronto Blue Jays

So if your favorite NFL team doesn’t get the coveted free agent don’t fret. They probably would have overpaid a lot. While it might hurt your chances next season, teams that are disciplined and don’t overpay often do better. Just consider how much these bad signings set teams back.