Kevin Bryan on A User's Guide to GPT and LLMs for Economics Research

Video & Timestamps

On Thursday, May 11, Kevin Bryan joined Markus’ Academy for a lecture on A User’s Guide to GPT and LLMs for Economics Research. Bryan is an Associate Professor, Strategic Management Area, Rotman School of Management at the University of Toronto.

Watch the full presentation below. You can watch all Markus’ Academy webinars on the Markus’ Academy YouTube channel.

Timestamps:

[4:45] Basics of LLMs

[16:25] Main research uses

[19:50] How to control the output of an LLM

[32:06] Examples of LLM use cases

[45:27] Practical fixes

Summary

• A summary in four bullets
  • LLMs have been shown to increase productivity in sales and programming in the past; academia should capitalize on this technology
  • 6 takeaways: (1) Controlling the output of LLMs is difficult, (2) the “Raw” ChatGPT online is far from state of the art, (3) hallucinations are mostly fixable, (4) the technology’ rate of improvements is fast, (5) most use cases for economists require using API+code (this will give you much more control on the output), (6) it is cheap to do so
  • The main uses for economists are: (1) cleaning data, (2) programming/making graphs, (3) spelling checks, (4) summarizing literature
  • Some best practices: (1) provide the model with as many examples as you can of what you need to be done, (2) do not use LLMs to do math, (3) use an “ensemble approach” to find the optimal prompt to ask the model
  • Download the full summary
  • Download Prof. Bryan’s User’s Guide to GPT & LLMs for Economic Research