Custard should taste eggy. This is a feature, not a bug. However, from your mention of flour, I think what you're talking about is a souffle, not something like a creme brulee or along those lines. Would that be correct?
Edited based on subsequent clarifications:
Ah. Your method is sorely lacking, and your ingredients are off. You need to temper the yolks first, you need to use yolks only, and use cream instead of milk. Here is a smaller version of what I use at work for creme brulee:
- 1L 35% cream
- 11 egg yolks
- 1C sugar
- 1 vanilla bean, scraped
Bring cream to a simmer with the vanilla seeds and pod (you can use approx 1.5tsp vanilla extract, pure not artificial, instead). While it is heating, beat the yolks and sugar together until just incorporated. Pour the cream into the eggs whisking briskly to prevent curdling. Pour mixture through a fine chinois. Skim off foam, pour into ramekins. Place ramekins in a pan, add water to 1/2way up the sides. Cover pan with foil. Bake at 275 (convection) or 325 (non convection) for approx 40 minutes, until wobbly in the centre but set. Chill until set, eat.
The deflation is caused by a souffle effect coming from including the egg whites, which are never used in a custard--custards, creme anglaise, etc, are always yolks only. When you include the whites, air will be trapped inside temporarily, and will escape/collapse when the mixture cools. If you bake until fully stiff with my method, you will get hard and rubbery custard by the time it is set in the fridge.
Also, save the whites--they freeze well--for meringues or souffles.