Read on Twitter
learning and teaching on YouTube, a thread https://twitter.com/conaw/status/1340351181516754945
(Disclaimer: though I work on education stuff at Google, these opinions are my own as an outside observer of YouTube)
1. Backing up a bit: the availability of information on the web is still under appreciated. It has diminished authority (read The Revolt of the The Public for a full treatment) and will fundamentally change what it means to learn and live.
2. I won’t bother to look up how many videos are on YouTube, the answer is vastly beyond the scale of a human lifetime. This means that processing YouTube content as a whole is strictly in the realm of algorithms. Broadly there are two of interest: search and recommendation.
3. In other words, learning x on YouTube only happens when x is recommended or searching for a term brings up x.
This is the first thing to keep in mind with the claim that this is a democratization of information access.
This is the first thing to keep in mind with the claim that this is a democratization of information access.
4. One way of learning on YouTube is that you are in a conventional class and watch a conventional lecture on YouTube.
It’s conventional for a reason though: there’s massive value in its content- and motivation-structure that’s lacking in other scenarios.
It’s conventional for a reason though: there’s massive value in its content- and motivation-structure that’s lacking in other scenarios.
5. Sitting through an hour+ video lecture is painful. The live presence of the instructor and peers seems to help motivationally. Data about MOOC completion rates supports that we just won’t do it without that nor the incentive of a grade and degree.
6. Research has been pretty damning for the lecture format when compared to other types of instruction (ex: https://science.sciencemag.org/content/332/6031/862.abstract)
This is usually presented as "active" vs. "passive". But I think it's something a bit more subtle: attention.
This is usually presented as "active" vs. "passive". But I think it's something a bit more subtle: attention.
7. Imagine someone having a question, then going to YouTube and watching a video with the answer. I'd guess they learn something. Is that passive because they're just watching? Or is it active because they had the question first? Either way they paid attention.
8. One of the first big successes of learning on YouTube was Khan Academy. In this case students may still be in a class content/motivation structure, but they have some question that needs answering, and Khan Academy provides in a more direct and personal style than a lecture.
9. Ten years after Khan Academy was founded, Google recognized the importance of this use case and acquired Socratic in 2018. There the friction to search is further reduced by letting you take a photo of your homework problem.
10. Khan Academy has focused on practice exercises in addition to their videos. Likewise MOOCs intersperse questions within lectures. This is the right move directionally--one study found that time w/ practice had a 6x effect over watching https://dl.acm.org/doi/abs/10.1145/2724660.2724681
11. Anecdotally, the questions are often far from thoughtful. In MOOCs, questions that are too abstract or require additional background knowledge may cause people to rewatch large segments of video and still not find what they're looking for.
12. Moreover, as @andy_matuschak and @michael_nielsen pointed out in https://numinous.productions/ttft/ the questions are often distracting from the emotional experience of the video.
13. @veritasium suggested that videos like Khan Academy's could start by drawing on common misconceptions: (note: this doesn't have to be an exercise--it's about attention)
Some nice research on this is http://aaalab.stanford.edu/papers/time_for_telling.pdf
Some nice research on this is http://aaalab.stanford.edu/papers/time_for_telling.pdf
14. When it comes to both motivation and efficiency, many things would benefit from being structured by the practice rather than the video. I.e. there is a sequence of exercises, and the video comes in before/after as needed. One example: https://experiments.withgoogle.com/interplay-mode/view/
15. This is not very different from a video game, where tutorials may come in at different points, and players are typically more motivated to reach goals. Plus video games have the option of teaching you via game mechanics: https://blog.codinghorror.com/level-one-the-intro-stage/
16. There are rare examples of of making a video-based narrative that ties in well with interactive elements as in this sequence on quaternions https://eater.net/quaternions
17. How far can "just watching" take you? This tweet is fascinating https://twitter.com/patio11/status/1264757794298789890
Of course sudoku youtube is a thing!
Perhaps an important pattern is seeing what someone is seeing while hearing their thoughts. (Khan Academy fits that too.)
Of course sudoku youtube is a thing!
Perhaps an important pattern is seeing what someone is seeing while hearing their thoughts. (Khan Academy fits that too.)
18. One clue here is dual-coding theory, which suggests that simultaneous visual and verbal input are stored in distinct representations, strengthening memory.
I'd love to see a study comparing different formats of presenting the info for something like the sudoku videos.
I'd love to see a study comparing different formats of presenting the info for something like the sudoku videos.
19. There's lots to study with basic pedagogy of videos. (If you know any, hmu) Presumably we wouldn't all benefit the same from the same sudoku video.
Is there a formula for which information and how quickly to present it based on the viewer's knowledge?
Is there a formula for which information and how quickly to present it based on the viewer's knowledge?
20. Another thing that is happening with this kind of "show me doing a skill" videos is revealing tacit knowledge. Studies show that experts are unable to produce a list of every step in a complex action. But in some cases, all can be revealed in a video.
21. A few sources highlighting just how critical this tacit knowledge transfer problem is. Can video save us? We need to know! https://medium.com/@samo.burja/the-youtube-revolution-in-knowledge-transfer-cb701f82096a
https://danwang.co/how-technology-grows/
https://danwang.co/how-technology-grows/
22. How about live videos? In good cases, having simultaneous co-learners adds excitement and serves as a check to the pedagogy -- you can ask a question to get background knowledge or slow the pace (this also benefits the archived video).
23. One of my favorite channels is the archive of George Hotz's Twitch streams: https://www.youtube.com/channel/UCwgKmJM4ZJQRJ-U5NjvR2dg. He takes on a variety of deep programming projects and manages to be entertaining while doing it. Though I wish there was a better way to see highlights.
24. Something that struck me in was the balance of being able to watch him learn some basics of biology (via web search so you can read what he's reading), but him clearly having some background knowledge and the smarts to keep it fast paced.
25. Let's define a spectrum of video styles from demonstrative to didactic. Hotz is demonstrative: authentically using a skill. Something like Crash Course is more didactic: defining and explaining concepts.
26. Perhaps the greatest (& most useless) educational YouTube video of all time is It uses a nice blend of demonstrative and didactic styles. The latter by using diagrams to explain important concepts (scuttlebug region, floating point conversion, etc).
27. There are so many insanely great communities around game glitches, speed runs, Minecraft building, and general gaming skills. Gamification misses the point by a mile. The real gamechanger is when the progress itself is worth showing off, building on, and having fun with.
28. Demonstration requires fluency in a medium and directing attention. For games, you record the screen, attention is already directed to the action. In more complex games like Starcraft, using the cursor and boxes to draw attention is important. See
29. Another example is music: the notes becomes part of the language as exemplified by this hilarious comment (from )
30. This masterclass between two piano greats involves talking, playing, pointing at the music, hand gestures, and other body language: Not a pianist but I feel like the video captures it well.
31. Buckle up, it's time for a detour through TikTok.
Admittedly I don't use it myself, but I recommend following @nwilliams030 and @jacknasjaq for examples & commentary: https://twitter.com/search?q=from%3Anwilliams030%20tiktok https://twitter.com/search?q=from%3Ajacknasjaq%20tiktok
Admittedly I don't use it myself, but I recommend following @nwilliams030 and @jacknasjaq for examples & commentary: https://twitter.com/search?q=from%3Anwilliams030%20tiktok https://twitter.com/search?q=from%3Ajacknasjaq%20tiktok
32. Unlike YT, which wants to increase watch time as much as possible, TikTok makes the marginal cost of watching another video as small as possible. Videos are short, can easily be skipped, and viewing is highly driven by recommendation.
33. Thinking of TikTok as just short videos undersells how much creativity is on -- creators are incentivized to be engaging and do it fast.
I've seen very effective videos on both the didactic ( https://www.tiktok.com/@songpsych/video/6909163375415119110) and demonstrative (any dance) sides of the spectrum.
I've seen very effective videos on both the didactic ( https://www.tiktok.com/@songpsych/video/6909163375415119110) and demonstrative (any dance) sides of the spectrum.
34. Because recommendation is dominate over search, there is more of a pull to find lots of cool new things instead of progressing deeply into a skill. (Of course you're free to leave TikTok, but, hey, cool new things!)
35. I still think there's a chance to see where the format goes. For example @PBhere1 is an ARG presented as a serial narrative. It's engaging, interactive, and ultimately long form. (just not very educational afaict)
36. Also re: @PBhere1 I wonder how much the animated-self style videos are going to take off given concerns about anonymity.
Maybe instead of an infinite geography to walk around, the metaverse is just floating head avatars streaming from your camera & on your screen.
Maybe instead of an infinite geography to walk around, the metaverse is just floating head avatars streaming from your camera & on your screen.
37. Let's go back. At 16-17 I branched into "just watching". What else could be done around watching+doing?
My favorite term framing this discussion is the "assistance dilemma" via http://www.cs.cmu.edu/~aleven/Papers/2007/Koedinger_Aleven_EPR2007.pdf
My favorite term framing this discussion is the "assistance dilemma" via http://www.cs.cmu.edu/~aleven/Papers/2007/Koedinger_Aleven_EPR2007.pdf
38. The assistance dilemma is the recognition that some amount of learning happens when you try to produce information rather than getting it directly. The question is how to balance these.
It's possible to do the former in a video, e.g. stop and ask to predict what will happen.
It's possible to do the former in a video, e.g. stop and ask to predict what will happen.
39. @ddmeyer introduced a category of math lessons around this idea called 3-acts https://blog.mrmeyer.com/2011/the-three-acts-of-a-mathematical-story/
Framing an interesting problem with the answer initially withheld will motivate the mathematical concept that resolves it.
Framing an interesting problem with the answer initially withheld will motivate the mathematical concept that resolves it.
40. Dan now works at @Desmos where they have classroom activities in this spirit. One of the benefits of an interactive classroom tool is that you can collect predictions, and students can see what their classmates thought.
41. Another benefit of an interactive tool is that you can see in real time the effects of changing variables. There is something to having control of the knobs yourself. Ex: in the warmup for this activity https://teacher.desmos.com/activitybuilder/custom/5e42c7ac8be9fd0c5cfbe706
42. @worrydream has illustrated the case for using interactivity to work with concepts in various essays and talks such as http://worrydream.com/LadderOfAbstraction/, which contains interactive visualization for a car control system.
43. Do we know that this interactivity is actually good for learning? Back to the assistance dilemma:
Perhaps there should be more information, the essay should simply show some examples at each layer of abstraction.
Perhaps less: maybe you should build the simulations yourself!
Perhaps there should be more information, the essay should simply show some examples at each layer of abstraction.
Perhaps less: maybe you should build the simulations yourself!
44. I should be clear that Bret is not trying to optimize learning here. He made the point clearly in a follow up essay, http://worrydream.com/LearnableProgramming/. Should we use these interactive tools to teach JavaScript? No, programming itself should be easier to learn.
45. A strong answer to the assistance dilemma is that info should never be withheld, "direct instruction": https://mrbartonmaths.com/resourcesnew/8.%20Research/Explicit%20Instruction/Why%20minimal%20guidance%20instruction%20does%20not%20work.pdf They write that worked examples (e.g. demonstrative videos) are the "epitome of strongly guided instructions".
46. The theory behind this argument (cognitive load theory) is that working memory is extremely limited, and that all learning happens by passing through working memory into long-term memory.
Withholding information taxes working memory, so don't do it.
Withholding information taxes working memory, so don't do it.
47. I haven't done justice to the breadth of research that has compared forms of instruction, but it is clear that there are many *wrong* ways to withhold information in the realm of techniques like discovery learning, problem-based learning, inquiry learning, etc.
48. But even taking the direct instruction conclusion at face value, it isn't clear what they mean by "during instruction" and how a class or educational resource can be organized to include exploration and practice before or after instruction.
49. The assistance dilemma paper explores this question within cognitive tutors, assisted problem solving environments. They conclude that *interactive* assistance is best, i.e. based on the student's real-time performance.
50. Cognitive tutors tend to be used in STEM. I dream of a world where we can just start doing something and have real-time interactive intelligent feedback on it. Alas we aren't there yet. A cycle of trying yourself and watching YouTube may be the best general approach for now.
