This is the 3530th most frequent Korean word.
못했던
"Couldn't do" or "was unable to do."
In this sentence, '못했던' refers to the inability to do the homework at school due to some circumstances.
그는 학교에서 못했던 숙제를 집에서 끝냈어요.
He completed the homework he couldn't do at school at home.
Here, '못했던' is describing the exercise that could not be undertaken yesterday due to being busy.
어제는 바빠서 못했던 운동을 오늘 했어요.
I did the exercises today that I couldn't do yesterday because I was busy.
In this case, '못했던' expresses the words the speaker wasn't able to convey previously to their friend.