Are Household Baptisms in the New Testament an Evidence of the Baptism of Infants?

No! In the case of the Philippian jailor, all those in his house were taught the gospel and believed the gospel (Acts 16:31-34); so in agreement with our Lord's commission, the salvation offered by Paul to his house was secured by believing first and then being baptized (Mark 16:16).

No! In the case of Lydia, there is no evidence the woman was even married or had any infant children (Acts 16:14-15), for "household" can easily refer to domestic servants or other employees in distinction to children (Gen 18:19; Acts 10:7; Matt 10:25).

See Also

Infant Baptism?
Does circumcision prove infant baptism? No! Scripture teaches plainly and repeatedly that faith and repentance are conditions for baptism (Mark 16:16; Acts 8:37).