(9 intermediate revisions by the same user not shown)
Line 10:
Line 10:
Across life, he accumulates knowledge about longevity and the incrementally increase in value for learning new info its value drops.
Across life, he accumulates knowledge about longevity and the incrementally increase in value for learning new info its value drops.
Wozniak sees himself as part of a whole, a network of global human intelligence. There is recyclement between brains [of different generations]. The brains come unprimed, with new models of reality, like <nowiki>[[Jarret Ye]]</nowiki>, he has a completely different view on <nowiki>[[Spaced Repetition]]</nowiki> and can be a compliment of the research he did.
Wozniak sees himself as part of a whole, a network of global human intelligence. There is recyclement between brains [of different generations]. The brains come unprimed, with new models of reality, like [[Jarret Ye]], he has a completely different view on [[Spaced Repetition]] and can be a compliment of the research he did.
There are no studies about SRS in the elderly. It is worth mentioning the difference for people who started using SRS at a young age and until elderly years. What long-term effects has SRS on the brain? What short-term effects does SRS for the elderly population who are introduced to SRS at late stages of life?
There are no studies about SRS in the elderly. It is worth mentioning the difference for people who started using SRS at a young age and until elderly years. What long-term effects has SRS on the brain? What short-term effects does SRS for the elderly population who are introduced to SRS at late stages of life?
Line 48:
Line 48:
==== Supermemo API ====
==== Supermemo API ====
[[File:Krzysztof Biedalak twit about supermemo API releasing soon.png|thumb|Krzysztof Biedalak tweet about supermemo API releasing soon]]
SuperMemo API was meant to be released first internally. Now, with [[FSRS]] being open-source and closer in performance, makes less sense to publish a closed-source paid API with just a slightly better performance.
SuperMemo API was meant to be released first internally. Now, with [[FSRS]] being open-source and closer in performance, makes less sense to publish a closed-source paid API with just a slightly better performance.
The motivation for Supermemo.com to have an internal API is to detach the code for the algorithm from the rest of the code. This way, they could run and test multiple algorithms at the same time.
The motivation for Supermemo.com to have an internal API is to detach the code for the algorithm from the rest of the code. This way, they could run and test multiple algorithms at the same time.
[[File:Krzysztof Biedalak twit about supermemo API releasing soon.png|thumb|Krzysztof Biedalak twit about supermemo API releasing soon]]
=== Pre-event text-based Q&A ===
=== Pre-event text-based Q&A ===
Line 77:
Line 78:
** 5.
** 5.
*** Note: I think 5 developers is for supermemo World company, which does not mean 5 people work on SM for windows.
*** Note: I think 5 developers is for supermemo World company, which does not mean 5 people work on SM for windows.
* Maybe I missed it. I have read it just now. I'm not very interested in continuing the comparison between FSRS and SuperMemo, because only Woz could implement SuperMemo.
* Maybe I missed it. I have read it just now. I'm not very interested in continuing the comparison between FSRS and SuperMemo, because only Woz could implement SuperMemo.
** implement "SuperMemo" sounds intimidating; anyone can implement a simple algorithm based on 3-component model of memory; only data sets are an issue
** implement "SuperMemo" sounds intimidating; anyone can implement a simple algorithm based on 3-component model of memory; only data sets are an issue
*** What do you mean by "data sets are an issue"? Can you explain the issues?
**** I loved my 2-3 years with alg-sm17 because I could play like a baby with my 30 years of data, pondering individual items and their history and how they intertwine with my personal life; without that factor, the job would get far more boring and technical. If you have a good theoretical model, all you need is a good dataset (I hear Mnemosyne has the largest, but that's not incremental reading, so it would fit better Anki-like work mode)
* In the last event Woz was quite enthusiastic about finding ways for the community to contribute more directly to SM. APIs, plugins, and even open source were discussed. What's the current state of this topic within the SM team, any progress, are they still open to/interested in the idea?
* In the last event Woz was quite enthusiastic about finding ways for the community to contribute more directly to SM. APIs, plugins, and even open source were discussed. What's the current state of this topic within the SM team, any progress, are they still open to/interested in the idea?
Line 91:
Line 96:
* I believe that it is possible to implement a functional API within SM by using the techniques you have already implemented for YouTube.htm (aka reading the <nowiki><input></nowiki> and <nowiki><option> fields). Would you be interested in providing source code for portions of SM related to YouTubeVideo.htm, and how MSHTML is used to render Elements? Based on my findings, you could provide an API without a major refactoring of the source code by adding more <input></nowiki> fields user scripts can toggle.
* I believe that it is possible to implement a functional API within SM by using the techniques you have already implemented for YouTube.htm (aka reading the <input> and <option> fields). Would you be interested in providing source code for portions of SM related to YouTubeVideo.htm, and how MSHTML is used to render Elements? Based on my findings, you could provide an API without a major refactoring of the source code by adding more <input> fields user scripts can toggle.
** I am impressed! The brain treats it like an instant subconscious boost to priority!
** I am impressed! The brain treats it like an instant subconscious boost to priority!
Line 103:
Line 108:
** yes. With your good knowledge of ML, you can forget matrices and theory; just stick to the model; for me the biggest value of alg-sm17 is not efficiency, but insights into the model itself; I can say that with a solid re-affirmation of theory, my whole life changed because see brains in a totally different way than 99.99% of the planet
** yes. With your good knowledge of ML, you can forget matrices and theory; just stick to the model; for me the biggest value of alg-sm17 is not efficiency, but insights into the model itself; I can say that with a solid re-affirmation of theory, my whole life changed because see brains in a totally different way than 99.99% of the planet
* What do you mean by "data sets are an issue"? Can you explain the issues?
** I loved my 2-3 years with alg-sm17 because I could play like a baby with my 30 years of data, pondering individual items and their history and how they intertwine with my personal life; without that factor, the job would get far more boring and technical. If you have a good theoretical model, all you need is a good dataset (I hear Mnemosyne has the largest, but that's not incremental reading, so it would fit better Anki-like work mode)
* There is the question of what is the most important thing to consider when buying flats. The answer is "the location and the location and the location". I think the most important thing in free learning is the quality (3x) of the learning material and the didactic preparation. And unfortunately, SM does not help here.
* There is the question of what is the most important thing to consider when buying flats. The answer is "the location and the location and the location". I think the most important thing in free learning is the quality (3x) of the learning material and the didactic preparation. And unfortunately, SM does not help here.
Line 131:
Line 134:
* My work involves learning such formula.. For actuarial. Science.. how can we write such formulas in SM.. That would solve all my learning issues at present
* My work involves learning such formula.. For actuarial. Science.. how can we write such formulas in SM.. That would solve all my learning issues at present
** yes. I bet supermemo is horrible for a mathematician (so says Paul R.). I flatten formulas to simple programming language or verbal descriptions. As I am not at school, I can use any notation I wish, so I can only empathize. Other users will probably tell you more fitting approaches
** yes. I bet supermemo is horrible for a mathematician (so says [[Paul Raymond-Robichaud]]). I flatten formulas to simple programming language or verbal descriptions. As I am not at school, I can use any notation I wish, so I can only empathize. Other users will probably tell you more fitting approaches
* Would there be a 4th component of long-term memory such as "learntropy" at the time of the repetition? How do you compare different levels of learntropy for the user across time? or when comparing users: the same difficulty but different learntropy? i.e. could learntrophy be estimated or computed, so weighted in the algorithm?
* Would there be a 4th component of long-term memory such as "learntropy" at the time of the repetition? How do you compare different levels of learntropy for the user across time? or when comparing users: the same difficulty but different learntropy? i.e. could learntrophy be estimated or computed, so weighted in the algorithm?
** genius question, but learntropy acts primarily on input; then you can rely on the pleasure of review which might combine usefulness and retrievability; e.g. you get happiest with "I almost forgot that gem!" perhaps also "How could I have forgotten such vital info?". Applicability changes overtime (do you know how to effectively export this discussion? 🙂
** genius question, but learntropy acts primarily on input; then you can rely on the pleasure of review which might combine usefulness and retrievability; e.g. you get happiest with "I almost forgot that gem!" perhaps also "How could I have forgotten such vital info?". Applicability changes overtime (do you know how to effectively export this discussion? 🙂
* In one of your documents you made a note that it might not be optimal for the algo to anticipate 90% forgetting index for each interval if the requested forgetting index is 90%. You argue that for mature material with a long interval, it costs more to start again from the beginning after a lapse than for material with smaller intervals. In other words, it should be considered whether SM should automatically increase the forgetting index for material with long intervals, e.g. from 90% to 95% (dynamically, of course). Have you thought about this further?
* In one of your documents you made a note that it might not be optimal for the algo to anticipate 10% forgetting index for each interval if the requested forgetting index is 10%. You argue that for mature material with a long interval, it costs more to start again from the beginning after a lapse than for material with smaller intervals. In other words, it should be considered whether SM should automatically increase the forgetting index for material with long intervals, e.g. from 10% to 5% (dynamically, of course). Have you thought about this further?
** I have a procedure for simulating that! it was never opened in SuperMemo. You revive my passion for algorithms 🙂
** I have a procedure for simulating that! it was never opened in SuperMemo. You revive my passion for algorithms 🙂
*** You simulate this manually in SM? By manually adjusting the requested FI?
*** You simulate this manually in SM? By manually adjusting the requested FI?
**** No. You optimize the algorithm by adding one extra dimension: requested forgetting index. With your question, I change my answer: Paul Robichaud suggested that improvement first time we met (Aug 2016). Now with alg-sm17 his suggestion is finally possible. So the answer to the questions: "Can we improve the algorithm further?" is ... yes. If we change the definition of the goals. Instead of having FI=10%, you may look for lifetime benefit (defined in some crafty way) 🙂
**** No. You optimize the algorithm by adding one extra dimension: requested forgetting index. With your question, I change my answer: [[Paul Raymond-Robichaud]] suggested that improvement first time we met (Aug 2016). Now with alg-sm17 his suggestion is finally possible. So the answer to the questions: "Can we improve the algorithm further?" is ... yes. If we change the definition of the goals. Instead of having FI=10%, you may look for lifetime benefit (defined in some crafty way) 🙂
* Have you analyzed the relationship between recalling time and retrievability? I guess it will take more time to recall when the retrievability is low. If we could build a model to predict it, it will help us optimize the review cost/learning speed in spaced repetition.
* Have you analyzed the relationship between recalling time and retrievability? I guess it will take more time to recall when the retrievability is low. If we could build a model to predict it, it will help us optimize the review cost/learning speed in spaced repetition.
Wozniak mentioned that he was concerned about longevity during his twenties. One tends to lose the fear of death with age, contrary to common belief. He believes that as long as one takes care of health, the brain is the least concern.
Across life, he accumulates knowledge about longevity and the incrementally increase in value for learning new info its value drops.
Wozniak sees himself as part of a whole, a network of global human intelligence. There is recyclement between brains [of different generations]. The brains come unprimed, with new models of reality, like Jarret Ye, he has a completely different view on Spaced Repetition and can be a compliment of the research he did.
There are no studies about SRS in the elderly. It is worth mentioning the difference for people who started using SRS at a young age and until elderly years. What long-term effects has SRS on the brain? What short-term effects does SRS for the elderly population who are introduced to SRS at late stages of life?
Supermemo.com does not keep track of user's age, as they follow the principle of getting the least info as possible from users. They would need to request an optional survey to use data from thousands of elderly users.
SRS for kids
Different memory properties, storage, and profile of knowledge. The process of becoming proficient in SRS takes years, I'd say I was ready in my twenties. Nowadays, information is so easily accessible, and it will take even longer. He is not even sure if Incremental Reading as it is a higher stage of development. Perhaps, remodel the process of introducing learning by prioritizing the reading and then the SRS very slowly. He theorizes that perhaps the optimum experience is the other way around he experiences it, he first developed SRS in 1987 and IR in 2000.
We need new tools to optimum curve should look like. Wozniak mentions that kids he talks to who get interested in SuperMemo, the theory and method behind it, become disappointed when they try it. No surprise, the new generation has never experienced a user interface from the nineties.
The Future of Language Learning with AI
The direction is to a better immersion experience:
Have conversations about topics of interest.
Pickup things worth remembering and a seamless way to add new flashcards in repetitions.
Implicit repetitions: you don't get bored with the same questions. This complexity is already achievable with current LLMs today.
In Supermemo.com there is a feature for roleplaying using ChatGPT API with speech recognition, TTS via Microsoft Azure.
Wozniak states that the best way to learn a new language is through Incremental Reading, he use it himself to learn some Ukrainian. He is aware he is biased on his judgment.
Wokniak mentions that AI is a dream come true from three decades ago, in the sense that AI can generate context. You can have a conversation directed by AI, and the AI invents a way to interject something unrelated that is supposed to be reviewed based on the date (due, outstanding). The repetition in the software is more similar to real-life repetition.
Supermemo algorithm vs FSRS
Wozniak is more interested in understanding how memory works for a single brain, rather than compounding from many different users which creates a lot of noise.
To no surprise, Wozniak thinks SuperMemo is best to use only on a Windows computer and sees no need to use the phone for Incremental Reading. Overwhelmingly the opposite is thought by the community.
Supermemo for Windows integration with supermemo.com
One of the options is to sync the collection from SuperMemo for Windows in supermemo.com. There are some technicalities to take into consideration, such as different templates, file size of media files, etc.
Wozniak proposed to manually tag items you would want to see on a mobile phone. That is way too tedious and inefficient. An idea I proposed is a tradeoff so the user could sync a subset of their collection such as 10% of the priority queue, or certain cherry-picked concept groups, so the user has a seamless sync experience and a limited data file size to sync. Biedalak mentioned he does not know what is stopping them from implementing a sync service between supermemo for Windows and supermemo.com.
Supermemo API
Krzysztof Biedalak tweet about supermemo API releasing soon
SuperMemo API was meant to be released first internally. Now, with FSRS being open-source and closer in performance, makes less sense to publish a closed-source paid API with just a slightly better performance.
The motivation for Supermemo.com to have an internal API is to detach the code for the algorithm from the rest of the code. This way, they could run and test multiple algorithms at the same time.
Pre-event text-based Q&A
Piotr Wozniak answered some questions before the start of the event.
When to open source SM?
Open source is a matter of community, not individuals
Yes, but if all is closed code, then the open community has to start at zero. As I understand it, the question is whether SM could offer interfaces for open source that could be built upon without giving away the secrets
The best resolution to chicken-and-egg is to start small with alg-sm2, and build momentum by proving the strength of the community; the Alg is not the most important part of such a project; with Jarette, the last excuse is gone 🙂
aware, appreciating, crossing fingers but buried in the priority queue.
Why do you hate your customers?
Best question! if no better shows up, I will start from this. (Note, he didn't).
What’s the priority algorithm of SuperMemo? Will Woz write documents for the priority algorithm like he has written for the memory algorithm?
Documenting it in detail would be very expensive, but we can address individuals (dis)likes
What is the future direction of development for SM? Better algorithms or something else?
nothing revolutionary on 5000 task tasklist; today progress feels incremental; it always does until a big thing shows up, or a little thing turns out to be a big thing
What is the main focus of woz's investment in the near future? Spring of Students? SM development? Or something else?
The biggest thing is "freedom for the brains"
How many SM-programmers do you currently employ?
5.
Note: I think 5 developers is for supermemo World company, which does not mean 5 people work on SM for windows.
Maybe I missed it. I have read it just now. I'm not very interested in continuing the comparison between FSRS and SuperMemo, because only Woz could implement SuperMemo.
implement "SuperMemo" sounds intimidating; anyone can implement a simple algorithm based on 3-component model of memory; only data sets are an issue
What do you mean by "data sets are an issue"? Can you explain the issues?
I loved my 2-3 years with alg-sm17 because I could play like a baby with my 30 years of data, pondering individual items and their history and how they intertwine with my personal life; without that factor, the job would get far more boring and technical. If you have a good theoretical model, all you need is a good dataset (I hear Mnemosyne has the largest, but that's not incremental reading, so it would fit better Anki-like work mode)
In the last event Woz was quite enthusiastic about finding ways for the community to contribute more directly to SM. APIs, plugins, and even open source were discussed. What's the current state of this topic within the SM team, any progress, are they still open to/interested in the idea?
BEST QUESTION ... social concept network is the answer. Even if this is the best Q, we should move it to the last 10 min. or Part II (this is a broad and important topic that might squeeze out oxygen from the meeting)
There is only one weak cog in the machine: Woz and his tasklists, Plans, and priorities; perfection can be crippling 🙂 ... PUSH ME!
When will the public scheduling API become available?
Can the functionality of YouTube.htm be kept the same in future versions of SM to allow users to continue to write their own incremental video scripts?
I believe that it is possible to implement a functional API within SM by using the techniques you have already implemented for YouTube.htm (aka reading the <input> and <option> fields). Would you be interested in providing source code for portions of SM related to YouTubeVideo.htm, and how MSHTML is used to render Elements? Based on my findings, you could provide an API without a major refactoring of the source code by adding more <input> fields user scripts can toggle.
I am impressed! The brain treats it like an instant subconscious boost to priority!
For what purpose do you use these arrows? ("to the beginning of the collection"; "to the end of the collection") Do you misuse it as a quick switch to favorites? Or what purpose do you have for it? I don't think I have ever used these outer arrows.
I do not use those buttons either, but they were born in 1995 following a multimedia courseware standard; for multimedia courses, they were like maximize/normal/minimize in Windows
Yeah, I agree with that. FSRS is also based on DSR model. The main bottleneck is the optimization. SuperMemo has many matrixes like R-matrix to optimize the parameters of DSR model. They are complicated.
yes. With your good knowledge of ML, you can forget matrices and theory; just stick to the model; for me the biggest value of alg-sm17 is not efficiency, but insights into the model itself; I can say that with a solid re-affirmation of theory, my whole life changed because see brains in a totally different way than 99.99% of the planet
There is the question of what is the most important thing to consider when buying flats. The answer is "the location and the location and the location". I think the most important thing in free learning is the quality (3x) of the learning material and the didactic preparation. And unfortunately, SM does not help here.
I often learn from horrible sources. If you find a golden nugget, after 20 min. of timewasting, it may still pay. With chatbots now, all sources seem digestible, however bad. All we care about is good ideas inside!
Does Woz know about the mnemonic medium? It's an augmented reading environment supported by spaced repetition. I think it has many common ground with the incremental reading of SuperMemo. Link: https://notes.andymatuschak.org/zPNoWjs5jQ7iMDpxxBtjSbg
ask Biedalak in case I mislead but ... "The dataset for the FSRS benchmark comes from 20 thousand people who use Anki, a flashcard app. In total, this dataset contains information about ~1.5 billion reviews of flashcards" ... with supermemo.com or supermemo.net there was/is a horrible bias in data: a lot of people who start and quit; those who do not stick to 20 rules; in other words, lots of noise; with Murakowski we "quarreled" a lot about this. He wanted "big data", while I insisted on going from person to person, starting with myself, and including crafty long-term users with a constellation of habits. You seem to be in Murakowski camp 🙂 I insist that lots of fun stuff start happening after 3-4 years of learning; also ... the memory model needs to extend to those long intervals (see "stabilization curve"). All in all, it is interestingly only for memory researchers, because I said in 1993 (in an interview) that further progress will not make much difference to users. They have much bigger problems to tackle (like a hungry baby in the other room). In other words, the art of perfect optimizations is only for you and me 😉 I study memory. You study algorithms. Perfect! 🙂
What are your plans after retirement. Open source the code?
Larry King: "Retire to what?" 🙂
Someone suggested opening above in a very sensible incremental way; the main inhibitor was the lack of tangible benefit for invested time; the suggestion above has a good bang/minute
Do you use WhatsApp or any kind of messaging applications.. I'm curious to know
I went to Discord for kids! I get requests to get on WhatsApp but ... they never come from kids! 🙂 ... Kids get a credit for the Round Table of education. We were looking for some expensive location with a participant vacation and the kids said: "why not Discord". We did it for free! I love kids!
How often do you communicate with ChatGPT? Do you think it would change education or learning hugely?
Use Copilots/Bing some 20-40 times per day; ChatGPT for language things (esp. Polish 🙂 and Bard/Gemini are still immature 🙂
Do you think relying on LLMs for generating Q&As or adjusting formulation will have an impact on users' recall and performance in the long-term? (as in skipping a step in the process)
I would never want anyone play with my items. There is some mnemonic magic in doing it yourself, however, I know you learn languages, so you have more room for automation there. You will have to see for yourself 🙂 (and write an article! 🙂
Is the algorithm used in SuperMemo 19 the same as in SuperMemo 18? If no, will the wiki be updated?
SM-19 might include some bug fixes. inessential
Do you think the minimum definition of IR needs some revision or updating?
Nothing comes to mind. if you create open source user-friendly software doing that minimum, you might change the world
My work involves learning such formula.. For actuarial. Science.. how can we write such formulas in SM.. That would solve all my learning issues at present
yes. I bet supermemo is horrible for a mathematician (so says Paul Raymond-Robichaud). I flatten formulas to simple programming language or verbal descriptions. As I am not at school, I can use any notation I wish, so I can only empathize. Other users will probably tell you more fitting approaches
Would there be a 4th component of long-term memory such as "learntropy" at the time of the repetition? How do you compare different levels of learntropy for the user across time? or when comparing users: the same difficulty but different learntropy? i.e. could learntrophy be estimated or computed, so weighted in the algorithm?
genius question, but learntropy acts primarily on input; then you can rely on the pleasure of review which might combine usefulness and retrievability; e.g. you get happiest with "I almost forgot that gem!" perhaps also "How could I have forgotten such vital info?". Applicability changes overtime (do you know how to effectively export this discussion? 🙂
In one of your documents you made a note that it might not be optimal for the algo to anticipate 10% forgetting index for each interval if the requested forgetting index is 10%. You argue that for mature material with a long interval, it costs more to start again from the beginning after a lapse than for material with smaller intervals. In other words, it should be considered whether SM should automatically increase the forgetting index for material with long intervals, e.g. from 10% to 5% (dynamically, of course). Have you thought about this further?
I have a procedure for simulating that! it was never opened in SuperMemo. You revive my passion for algorithms 🙂
You simulate this manually in SM? By manually adjusting the requested FI?
No. You optimize the algorithm by adding one extra dimension: requested forgetting index. With your question, I change my answer: Paul Raymond-Robichaud suggested that improvement first time we met (Aug 2016). Now with alg-sm17 his suggestion is finally possible. So the answer to the questions: "Can we improve the algorithm further?" is ... yes. If we change the definition of the goals. Instead of having FI=10%, you may look for lifetime benefit (defined in some crafty way) 🙂
Have you analyzed the relationship between recalling time and retrievability? I guess it will take more time to recall when the retrievability is low. If we could build a model to predict it, it will help us optimize the review cost/learning speed in spaced repetition.
I would rather oppose that. Let the brain optimize processing time without ever being biased that the timer is on. Yes. For low retrievability, you retrieve by "search", which may take ... 24h 🙂
Have you ever reset any collection parameters? when do you think a user will need to use this feature?
useful for messed up collections or for research; I would never use it on a living collection
Some cards have similar contents. They could intervene or remind each other, such as clozes from the same topic. Did you do some research about their memory patterns?
if you have identical items, you mess up the algorithm. interference between items will always be there. also interference with life. We can only account for it statistically. Anyway, interference with life is great. Useful knowedge sems easy for supermemo 🙂
With ML approach you can actually worry less because your goal is performance, not a true reflection of the status of memory
Well, I assume performance can be improved if we can add the impact of studying similar cards to the SInc formula
we are entering the realm of artificial intelligence. with a huge load of parameters, I would fear we get lost in complexity and start leaking performance due to ... bugs ... One day will will hand it all to AI and ask to serve the brain optimally
How should we correct the situation if we previously added too many high-priority topics, overlooking the phenomenon of priority skew? Should we uniformly lower the priority of the old topics, or do nothing?
you will never run way from priority bias 🙂 ... I say enjoy your appetite for learning! Most of the population does not have this problem. I would address it only incrementally. Say you see a topic that is really not that important, you can deprioritize its branch. If you rarely see them, so much better for your appetites 🙂 You need focus, strong will and good habits to reduce the effect. The more you struggle, the better your learn drive (I bet) 🙂
