Why effort metrics mislead
Hours studied, cards reviewed and pages covered all measure how much you did rather than what happened. They are easy to record, they feel satisfying, and they can rise steadily while ability does not.
The specific danger is that they are optimisable. It is entirely possible to increase every one of them by doing more of the comfortable activities, which are usually the ones producing the least improvement.
| Metric | Measures | Predicts improvement? |
|---|---|---|
| Hours studied | Effort | Weakly |
| Cards reviewed | Deck size | Weakly |
| Pages covered | Coverage | Weakly |
| Conversations held | Output | Well |
| Words used in speech | Active vocabulary | Well |
| Passages read unaided | Reading ability | Well |
The bottom three are harder to inflate, which is exactly why they are better measures.
Output over input
The metrics worth keeping measure what you produced rather than what you consumed. They are smaller numbers, they are less satisfying, and they move in step with actual ability.
Conversations held is the single best one for most learners, because speaking is the skill that lags and the one that responds directly to volume.
- Conversations held per week.
- Words you used in speech for the first time.
- Passages read without looking anything up.
- Corrections that stopped recurring.
- Situations handled in Japanese that you previously avoided.
Two or three numbers
Elaborate tracking systems collapse, and while they last they consume time that would otherwise be study. The systems that survive are small enough to maintain in under a minute.
Two or three numbers is enough to notice a trend, which is all tracking needs to do.
- Pick two or three metrics and ignore the rest.
- Record them in whatever you already use; no new system.
- Under a minute a day, or it will not last.
- Review monthly rather than daily.
- Change a metric only if it has stopped telling you anything.
The monthly review
Language ability moves on a monthly scale, which means daily numbers are noise. Looking at them daily encourages reacting to variation that means nothing.
A monthly review, comparing this month to last, is enough to see a trend and to act on it.
- Compare the month to the previous one, not day to day.
- Ask which activity produced the change, if any.
- Ask what you avoided doing and why.
- Adjust one thing rather than redesigning the plan.
- Record one sentence about how the month felt; it is often more informative than the numbers.
Measuring what does not count itself
Some of the most useful indicators are qualitative and are worth noting even though they are not numbers, because they capture changes the metrics miss.
A recording of yourself once a month is the strongest of these, since it captures speed, pronunciation and confidence at once and is impossible to argue with later.
- A monthly recording of the same spoken prompt.
- Whether a situation you used to avoid now feels routine.
- Whether you noticed yourself thinking in Japanese.
- Whether people switch to English less often.
- Whether you finished something you previously abandoned.
When tracking has become the problem
The signs are recognisable: the system takes longer to maintain than it saves, the numbers are being optimised rather than the ability, or the day feels successful because the boxes were ticked.
The fix is to cut the system down rather than abandon it entirely, since a small amount of tracking genuinely does help by making avoidance visible.
The most valuable thing tracking does is show you what you are quietly not doing, which is almost always speaking.
If your tracking shows plenty of study and few conversations, the tracking has done its job. Act on that rather than adding another metric.