This mostly happens because they're probably both cotton.

As you wash them, the oils the fabric was soaked in for protection slowly get removed, and the cotton fibres are rather crinkly. They can grab onto loose and weakened other fibres. This is also why some of them "ball" on the outside (no clue what it's called in English). Now on the inside, it's the shirt pulling loose fibres off the inside of the weapons, the tiniest break-offs.

To alleviate this - you can't entirely fix it, other than wearing different combinations of materials instead of cotton on cotton - wash your sweaters inside-out, which you should do anyways, in particular on printed sweaters. Instead of the insides of the sweater rubbing together and depositing lots of loose broken fibres for the t-shirts to pull off, those will get pulled off inside the washing mashine.
Note: This increases the speed at which you get the "balling" (since now the outside rubs against itself), but on fibres that can do this you want to eventually use a fabric shaver every so often anyways if you don't already.