Teaching Tip: Sequential Presentation of Data

You all know that in MobLab graphs are available immediately after game play. But, how much have you thought about how to present the information those graphs contain? In this blog post I argue that presenting information sequentially is a good way to talk students through theory and data. Let me show you what I mean with a quick example from the Competitive Market game.

When you click the “Results” button you see a graph with all lines present. However, you can use the legend below the graphs to add and subtract pieces of information as shown below. This is the crux of this blog post.



Let’s start with the Competitive Market Game:

  • Using the checkboxes in the legends for these two graphs, hide all of the lines.
  • Reveal the market demand curve. Ask students what it represents and ask them to provide reasons why it slopes downward.
  • Reveal the market supply curve. Ask students what it represents and ask them to provide reasons why it slopes upward.
  • Reveal the equilibrium price line. Ask students to explain why it is an equilibrium. If students simply give a textbook response probe them about the game. Often buyers start out submitting stingy bids and sellers hold out for high prices. Across time sellers must undercut each other to find willing buyers. Buyers must bid up each other to find willing sellers.
  • Is the equilibrium price-quantity pair a good prediction?
  • Reveal the transaction markers.

This sequential presentation of the data helps review content and walks students to the prediction. Then showcasing the data emphasizes economics as an empirical science.



Tragedy of the Commons at Penn State

After the recent RMU Teaching Workshop I road tripped over to Penn State with the amiable and generous Jadrian Wooten. I had a blast. We talked economics and life and he introduced me to Primanti Brothers which was delicious (the off-menu “When Pigs Fly” sandwich was my kind of blue collar food).

On the following Monday we ran three versions of the “Commons: Fishery” game which went really well. Note, I’m only presenting a subset of results as there were many students.

Version 1: N=1. The initial stock of fish is 40 and the fish stock doubles at the end of each round. The goal of this parameterization was two-fold. First, I wanted students to be familiar with the replacement rate and what that implied about sustainable catch each round. Second, I wanted a baseline for comparison when property rights are not so clean. From the results below you can see many students tracked well with the sustainable rate of resource consumption.

Screenshot 2017-03-15 13.52.51

Version 2: N=4. The initial stock of fish is 160 and the fish stock doubles at the end of each round. While students have now been familiar with the replacement rate the incentives are clear, “I drink your milkshake!“:

Screenshot 2017-03-15 13.56.33

After they participated in Version 1 and Version 2 I asked the following question on our free-text response, “Compare the second game, when you fished a common sea (with three others), to the first game when you fished alone. Explain any differences in incentives across those two games and whether those differences affected how you played.” Here are some student responses:

“The first game I fished sustainably, in the second – I tried sustainability at first but soon found I was competing to be able to get as many fish as I wanted and supply quickly ran out.”

“Property Rights, first one you have property rights over your fish so they are protected no one else is going to take them. Second time there are no property rights so you have to guarantee that if you try to conserve fish that they will not be taken by another fisher.”

“When you fish in your own private pool, you’re the only one setting the limit as to how much you want to catch. Whereas when you’re in a pool with 3 other players, there is a lack of communication in the sense that some people have no limit and just go wild. When there is no communication, strategy just becomes irrelevant, and you’re praying to god that the people you’re paired with aren’t idiots and consume the whole pool in a frenzy to get the top score.”


Version 3: Same as version 2. This time with chat and indefinite repetition to allow them to experience a situation with more realism: neighbors can talk to each other, there are long time horizons and reputation effects, etc.

Screenshot 2017-03-15 13.59.19

Asking the students about about why communication mattered, students responded that they could coordinate with each other. This kind of coordination on a set of rules for fishing matters even if communication is cheap talk. Also, students got a laugh when I asked them to recall, “What’s the nastiest thing someone said to you over chat?” As noted in the classic paper, “Covenants with and without a Sword: Self-Governance is Possible”,

“A striking aspect of the discussion rounds was how rapidly the subjects, who had not had an opportunity to establish a well defined community with strong internal norms, were able to devise their own agreements and verbal punishments for those who broke those agreements.” (Ostrom, Walker, and Gardner, 1992, p. 410).

Jadrian noted that 61% of students found MobLab added value to the lecture, 30% were neutral, and 9% didn’t like it. But, what I found really interesting was his comments after talking to his good students, “they enjoyed getting to see the results of what we talked about rather than just thinking about how it would happen.” This is an important point: MobLab isn’t just a useful tool for engaging students that are difficult to engage in standard lecture. MobLab engages good students because it showcases economics as an empirical science and not just a bunch of theories. 

I am hopeful the game made the incentive problems clearer to them but also gave them a sense that incentives are not destiny. Cheap talk can help construct rules and verbal reprisals can act as a price on the externality we impose on others. Economics is a rich science that can help them explore whatever issues they’re passionate about when they “look out the window.” Many thanks to Jadrian for entrusting his class to me.


A Personal Note on Match Day and our KR Matching Game

My wife is a pediatrician. In mid-March 2013 we experienced “match day”: the day where senior medical students match to a specialty-hospital pair for their residencies. For a moment stroll with me down memory lane: All the soon-to-be doctors were dressed up with family and friends in tow. There was so much chatter, excitement, hugs —the whole place was electric. The ballroom was decorated in a “Match Madness” theme and medical school faculty dawned basketball and referee uniforms with whistles and headbands. Each senior medical student was handed an envelope with their match location and speciality inside. And though I wanted to tear the thing open we were told not to open it until the appointed time.


When the envelopes were opened it is difficult to communicate the emotion. So much time invested in medical school and so many great and terrible experiences in the process of becoming a doctor. And, the hope after visiting so many different residencies, that you would get your preferred choice. We were very happy when we found out we were headed to Greenville, SC. The program there is stellar with committed attendings and a 100 percent pass rate on the pediatric boards for the last several years. That’s better than any pediatric residency east of the Mississippi.

So how did “match day” come about? In 1952 John Stalknaker and F. J. Mullen designed the National Residency Matching Program (NRMP) to alleviate problems in the labor market for physicians. The problem (described here) was,

“…competition for scarce medical students forced hospitals to offer residencies (internships) increasingly early, sometimes several years before graduation. Matches were made before the students could produce evidence of their qualifications, and even before they knew which branch of medicine they would like to practice. When an offer was rejected, it was often too late to make offers to other candidates …”

The NRMP required both students (hospitals) to submit a ranking of their most desired hospitals (students). Then an algorithim would match students and hospitals. In 1984, economist Al Roth showed that this algorithim was strategy-proof and produced “stable” matches. A match is stable if nobody perceives that a swap between residents and hospitals would be mutually beneficial. Later Al Roth would make modifications to the NRMP to accommodate married doctors. This is part of the work that eventually won Al Roth the 2012 Nobel Prize, “for the theory of stable allocations and the practice of market design”.


Speaking of Al Roth, he also happens to be a Senior Scientific Advisor at MobLab. In 2016, after a long day of job interviews and climbing hills in San Francisco I sat down for dinner at Le Colonial with the folks at MobLab. When he sat down at the table I was starstruck. Not knowing what to say, I leaned over and shook his hand,  “Thank you, Professor Roth. My wife and I are very happy with where she matched.” He laughed and smiled. He was kind and gracious in conversation and told me, “call me Al”. That is a dinner I won’t forget.

It pleases me greatly to announce that soon MobLab will be releasing its KR Matching Game (the KR stands for Kagel and Roth and refers to this paper in the Quarterly Journal of Economics). In this paper Kagel and Roth study the decentralized matching system that “unraveled” (with early matches and miscoordinations) as well as centralized systems like the NRMP. Their experiments provided a greater degree of confidence that what they had observed in a handful of cases were robust.

In our new game we will allow our instructors to run students through different matching markets to allow students to experience market unraveling and the different solutions to that problem. Keep a look out for the game in the “Market” tab in your instructor console.


Finally, it’s worth noting that the problem of matching is ubiquitous and far beyond NRMP. As Al Roth states in his excellent Freakonomics interview,

“Matching markets are markets where money, prices don’t do all the work. And some of the markets I’ve studied, we don’t let prices do any of the work. And I like to think of matching markets as markets where you can’t just choose what you want even if you can afford it — you also have to be chosen. So job markets are like that; getting into college is like that. Those things cost money, but money doesn’t decide who gets into Stanford. Stanford doesn’t raise the tuition until supply equals demand and just enough freshmen want to come to fill the seats. Stanford is expensive but it’s cheap enough that a lot of people would like to come to Stanford, and so Stanford has this whole other set of market institutions. Applications and admissions and you can’t just come to Stanford, you have to be admitted.”

Thinking through how different institutions or “rules of the game” affect market outcomes is really important. Behind all that math there is a human component. My wife and I are almost certainly happier with our outcome than we would have been before 1952.


Livin’ the Mug Life: Introducing the Beer Game

Ever go to the store to pick up the “beer essentials” and wondered how they got there? There are vast networks of exchange in place to get the simplest of items to our grocery store. It’s really amazing when you think of it.


Our “beer game” helps students understand how supply chains operate and to illustrate a common problem called the “bull whip effect” that comes from information asymmetries. This effect is characterized by how sudden changes in customer demand generate larger and larger changes in order submissions upstream through the supply chain.


In this game each student represents one of four roles in a supply chain: retailer, wholesaler, distributor, and producer. As depicted in the picture above, orders flow upstream between firms and shipments flow downstream between firms. Each round proceeds in the following way:

    • You receive incoming deliveries from upstream firms
    • Update inventory
    • You receive incoming orders from downstream firm
    • Send delivery to downstream firm from available inventory
    • Place order with upstream firm
      • Note: There is a time delay for orders to be filled. An order placed in the current round (week) takes one week to travel upstream between each node in the supply chain. Deliveries travel downstream and are subject to shipping delays between each node in the supply chain.


The goal of each firm is to minimize costs from holding inventory ($1/unit) and from unmet demand or backlog ($2/unit). But, minimizing cost gets harder and harder as you move upward through the supply chain since order submissions become larger as firms seek to avoid the penalty from unmet demand. Then order submissions nose dive as firms begin to feel the pain of high inventory costs. This kind of behavior is a huge drag on supply chain efficiency.

Speaking of efficiency, while most implementations of the beer game and run by hand, game board, etc.. This game has a smooth interface that can be played on any internet connected device. Plus, graphs and summary tables with relevant data are available immediately following game play. This will help drive home the bull whip effect but also to show how logistical improvements that reduce shipping delays can attenuate the bull whip effect.

Happy Playing!



Teaching Tip: Reflection Questions Before Results

Many instructors want to increase student engagement and make their class more interactive. One way to increase student interaction with our MobLab results is to have students write a reflection on the game immediately after playing and before presenting the results. For example, before presenting the results on the public goods game ask students some questions:

  • What did you contribute in the first round? What factors did you consider when choosing how much to contribute in subsequent rounds?
  • According to an economist, how do you choose how many donuts you buy? According to an economist, how would you choose how much to contribute to the water project? How does the water purification differ from a donut?

Screenshot 2017-03-10 19.34.26

Give students a couple minutes to answer these questions. Having thought through the ideas and organized their thoughts students will be more willing to jump into the fray. In addition, there is reason to believe that many of the positive benefits from classroom experiments come from reflection questions.

If you’re interested in reflection questions the economists from MobLab have dreamed up, check out our Modules section. Happy Playing!


Conversations with Educators: Andrew Kashdan


We are continuing our Conversations with Educators series. While we primarily work with college educators there are a number of high school educators who have shown enthusiasm about MobLab. One of our earliest high school adopters was Andrew Kashdan. Below is our interview with him:

Q: First, please tell everyone a little bit about yourself and how you became interested in economics.

A: Although I had graduated with a degree in Applied Economics, my interest in understanding the subject at a deeper level really began “on the job” when I worked in the financial industry in various capacities. Economics really allows you to better understand what is going on in the real world. I ended up going back to school and earned a master’s degree in economics.

Q: When did you know you wanted to become a teacher?

A: I definitely didn’t predict my career path early on. As a graduate student, I taught some courses and found the teaching aspect more to my liking than focusing on research. It was a lucky accident that I ended up teaching at an independent school where I can specialize in economics. I have the opportunity to teach introductory (AP) Micro and Macro, as well as a Post-AP seminar in political economy that I created.

Q: You are one of the first high school teachers to have adopted MobLab. How has the platform evolved since you first started using us?

A: The main thing, which is what really adds value, is the addition of more games. I remember seeing a documentary on behavioral economics that talked about how Vernon Smith ran experiments on asset market bubbles and thought, “that would be a great game to have in class.” I was excited when Moblab created a version of it, and I think it’s one of the better games because you often get a non-obvious result.

Q: How has the experience with MobLab been from your perspective? What have student reactions been?

A:If my students had their way, we’d use MobLab a lot more. It’s a good way to break up a traditional lesson that is both fun and productive.

Q: Do you have a favorite MobLab game to play with students?

A: My personal favorite is a simple one, playing a tournament of a repeated prisoner’s dilemma. As anyone knows who has studied a little bit of game theory, it allows for some interesting experiments in trying out different strategies with slight variations, and students will discover what characteristics a successful strategy will have. Another popular one is the public goods game with punishment — students relish the opportunity to punish their classmates for not cooperating.

Public Goods: Punishments and Rewards game screen

Q: What advice would you give a new instructor who wants to use games in their classroom?

A: If their situation allows them to get access to MobLab, it seems like a no-brainer to incorporate it into an economics class. It’s fun for students, and allows them to see some of the theories they are learning in a different way. It’s one thing to show an equilibrium on a graph; it’s another to see it emerge in a game when players have limited information. I recommend going through the video instructions yourself, and experimenting with test accounts, so you can give a concise overview for your students.


Monty Hall and Bayes’ Rule

In her 1990 article in Parade, columnist Marilyn vos Savant is asked about the following decision situation:

Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?

If you’re unfamiliar with the problem (called the “Monty Hall Problem”) take a moment to consider what you would do. Got it? Marilyn responds, “Yes; you should switch. The first door has a 1/3 chance of winning, but the second door has a 2/3 chance.” Many wrote in to tell her how wrong she was, below is a good representative comment:

You blew it, and you blew it big! Since you seem to have difficulty grasping the basic principle at work here, I’ll explain. After the host reveals a goat, you now have a one-in-two chance of being correct. Whether you change your selection or not, the odds are the same. There is enough mathematical illiteracy in this country, and we don’t need the world’s highest IQ propagating more. Shame!
Scott Smith, Ph.D.
University of Florida

In fact, Marilyn vos Savant was right. If you don’t believe me go ahead and play MobLab’s new Monty Hall game a hundred times, a thousand times, to your heart’s content! You’ll see that 2/3 of the time you’re better off switching.

Screenshot 2017-03-03 07.54.06

To help build intuition for why you are more likely to win by switching, what if there were one million doors? Then the host opened 999,998 doors. The car is behind one of the two remaining doors. Should you switch? It was very unlikely you picked the correct door! To help students see this intuition MobLab has a setting where there is an option for 20 doors (hard to fit 1 million doors on an iPhone screen).

Screenshot 2017-03-03 07.55.36

In the process of designing this game friends said, “That’s a fun parlor game, good for bar talk, but, what importance does it have?” It helps build intuition for Bayes’ Rule which is of tremendous consequence. Here is one memorable story conveyed in a pedagogical paper on Bayes’ Rule from Charlie Holt and Lisa Anderson,

“Of particular relevance to college students is a (true) story about a man who received a positive outcome on a first-stage test for the virus that causes AIDS. The test that was used had a 4 percent rate of false positives, and for simplicity, it is assumed that there were virtually no false negatives. The person committed suicide before follow-up examinations, presumably not realizing that the low incidence of the virus in the male population (about 1 in 250 at that time) resulted in a posterior probability of having the virus of only about 10 percent.”

Even though the test is good with only a 4 percent chance of detecting something that didn’t exist (false positive) the virus was relatively rare (about 1 in 250). That means that even with a good test it is more likely the person is in the other 249 out of 250 set. Bayes’ Rule takes the probabilities of AIDS in the population, rates of false positives, and the current test results to update the chance that a given person has AIDS. After using Bayes’ Rule the person only had a 10 percent chance of having the virus that causes AIDS and killed themselves anyway.

Going through a memorable example like this and using our Monty Hall game may help build intuition for Bayes’ Rule, help it stick, and provide a good segway into games of incomplete information which so many of our instructors are using. We are hoping to provide you more!

Finally, for your more savvy students who might already know about the Monty Hall Problem go ahead and trip them up by “re-skinning” the game with space explorers seeking to buy a working (not broken) teleporter.

Screenshot 2017-03-03 08.14.29

Results of our game show the average rate of “Switch” and “No Switch” as well as rate of switching in each round. We hope you enjoy this new game!


People Chatting. Vector illustration of a communication concept, relating to feedback, reviews and discussion.

Encourage Your Students To Talk

A couple weeks ago I dawned my best undergraduate garb and wandered into a college course using MobLab. The instructor knew I would be there. But, I didn’t want an introduction. I wanted to be an anthropologist and get first hand observations. It was a useful exercise and one thing became blatantly clear to me: Let the students talk.

Probably that seed was planted several weeks ago when Doug McKee, an instructor at Cornell, noted in his excellent teaching blog Teach Better (you can see his whole post here),

The morning class was good, but very quiet while the students were choosing production levels. In the afternoon I encouraged students to talk to each other about their strategies. They didn’t know who was in their particular market, so it wasn’t an opportunity for collusion. I think they got more out of the exercise and it was definitely more fun.

He’s absolutely right. There are several advantages to letting the students talk:

  1. Information Exchange: There is information exchange as students confer on strategies. Yes, some might talk about tonight’s party or ball game, but, there is some real peer-to-peer learning that happens when students talk.
  2. Fun: It’s more fun. The room has a different vibe: In addition to learning the games are more energizing. I would hazard to guess that students talking during games gives them a productive and more refreshing break from standard content delivery.
  3. Contamination Unlikely: They’re unlikely to be in the same group and therefore talking doesn’t serve the purpose of coordination. Sure it might change their strategy and the way they think about the game but you want that!
  4. Network Formation: Students form networks. Getting students to talk about a shared experience like the game is an excellent ice breaker. Perhaps the people they end up having conversations with become study buddies.

From where I sat in the classroom the silence was deafening. It’s an anecdote, but, once the students were allowed to talk in the next section of this class the energy changed and I’m hopeful their learning did too. Count me in with Doug McKee, let the students talk.


Takeaways From the End of Year Instructor Feedback Survey

First and foremost, thank you friends for constantly providing great feedback and supporting our product. We believe collaboration with our current users plays a crucial role in both new game and feature development. Not to mention, we have some of the brightest minds in the business as part of our ‘Mob.’

In my new role as Product Manager, one of my goals was to take a step back and assess how our platform and team is serving current users. As a team, we launched an End of Year Instructor Feedback Survey as a litmus test to quantify how we were doing and direct our team development priorities.

To properly complete the communication feedback loop, I wanted to share a top-line report of the feedback we received along with some of our initiatives currently underway.


Our instructors rated their experience with MobLab to be a 4/5, with the largest request being to add more content. Thus an early initiative of the year was creating Modules to layer different treatments games with pre and post game questions to hone in on key teaching points. Plus more modules are on the way!

Does our Support Matter?
There is a positive relationship between overall assessment and how well an instructor feels supported. Which was definitely reassuring for our team, because we love working with you all!

support relationship

Stay with the Mob

81.5% responded saying they would use MobLab again with 16% sharing they were unsure and may use us again. We understand teaching rotations are often up in the air, but first time users rest assured, once you master the system it’s a lot of fun!


New and Upcoming Games:
Monty Hall, Beer Supply Chain, KR Matching, Voter Turnout, Display Advertising Auction, and Market Exchange + Wealth Distribution are all planned for this spring.

Monty Hall Preview



Instructors are using surveys in a multitude of ways we never could have even imagined — even running quizzes outside of class! We’re adding some new survey features and making data downloads more intuitive this spring.


“MobLab is Awesome.”

Thanks folks. We’re glad you agree!