Gerhard summarizes his arguments for why the unbaptized (and even unborn) children of Christian parents are not damned:
In our commonplace On Baptism [Commonplace XXIII] (§§ 23[7]–2[40]) we set against this bloodthirsty and Rhadamanthine decree of the Papists: (1) The theological rule that it is not a lack of the Sacraments but contempt for them which damns. (2) The divine promise that God wants to be the God also of our seed (Gen. 17:7), that He orders the little children be brought to Him (Mark 10:14); that he wants to receive whoever comes to Him (John 6:37). Godly parents grasp this promise in faith and through their prayers offer their babies to God while they are still enclosed in the womb. (3) Divine goodness, because it is not the will of the heavenly Father that any of these little children should perish (Mark 18:14). (4) Divine power, because by extraordinary action He can kindle faith unto salvation in those babies and is Himself not absolutely bound to the means instituted for our good. (5) The universal nature of the merit offered through Christ, who wanted to be born an infant also for the sake of infants. (6) The examples of the Israelite babies who died before receiving circumcision and yet were not simply excluded from salvation…
See also On Death, § 169.